Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28288

Audit Trail Plugin log file rotation issue

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Major
    • Resolution: Fixed
    • audit-trail-plugin
    • Jenkins 1.611 running on RHEL 6.6 using audit-trail plugin 2.1

    Description

      There is a log rotation issue with the Log File logger in the audit-trail plugin.

      I set the count parameter to 1 and the limit to 512. Yet the log file is still being rotated every time more than two lines are written:

      [jenkins ~]$ cat /data/jenkins/audit-trail.xml 
      <?xml version='1.0' encoding='UTF-8'?>
      <hudson.plugins.audit__trail.AuditTrailPlugin plugin="audit-trail@2.1">
        <pattern>.*/(?:configSubmit|doDelete|postBuildResult|cancelQueue|stop|toggleLogKeep|doWipeOutWorkspace|createItem|createView|toggleOffline)</pattern>
        <logBuildCause>true</logBuildCause>
        <loggers>
          <hudson.plugins.audit__trail.LogFileAuditLogger>
            <log>/data/jenkins/logs/audit-trail.log</log>
            <limit>512</limit>
            <count>1</count>
          </hudson.plugins.audit__trail.LogFileAuditLogger>
        </loggers>
      </hudson.plugins.audit__trail.AuditTrailPlugin>
      
      [jenkins ~]$ ls -altr /data/jenkins/logs/audit-trail.log*
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:05 /data/jenkins/logs/audit-trail.log.lck
      -rw-r--r--. 1 jenkins jenkins  97 May  7 11:06 /data/jenkins/logs/audit-trail.log
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:06 /data/jenkins/logs/audit-trail.log.1.lck
      -rw-r--r--. 1 jenkins jenkins  97 May  7 11:06 /data/jenkins/logs/audit-trail.log.1
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:06 /data/jenkins/logs/audit-trail.log.2.lck
      -rw-r--r--. 1 jenkins jenkins  97 May  7 11:06 /data/jenkins/logs/audit-trail.log.2
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:07 /data/jenkins/logs/audit-trail.log.3.lck
      -rw-r--r--. 1 jenkins jenkins  97 May  7 11:14 /data/jenkins/logs/audit-trail.log.3
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:14 /data/jenkins/logs/audit-trail.log.4.lck
      -rw-r--r--. 1 jenkins jenkins 115 May  7 11:15 /data/jenkins/logs/audit-trail.log.4
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:15 /data/jenkins/logs/audit-trail.log.5.lck
      -rw-r--r--. 1 jenkins jenkins 115 May  7 11:15 /data/jenkins/logs/audit-trail.log.5
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:15 /data/jenkins/logs/audit-trail.log.6.lck
      -rw-r--r--. 1 jenkins jenkins   0 May  7 11:15 /data/jenkins/logs/audit-trail.log.6
      

      It appears the log rotation count is not working correctly.

      Attachments

        Issue Links

          Activity

            mnejman Mark Nejman created issue -
            mnejman Mark Nejman made changes -
            Field Original Value New Value
            Description There are multiple issues with the Log File logger in the audit-trail plugin.

            The first has to do with log file rotation. I set the count parameter to 1 and the limit to 5000000000. Yet the log file is still being rotated every time more than two lines are written:

            {code}
            [jenkins ~]$ cat /data/jenkins/audit-trail.xml
            <?xml version='1.0' encoding='UTF-8'?>
            <hudson.plugins.audit__trail.AuditTrailPlugin plugin="audit-trail@2.1">
              <pattern>.*/(?:configSubmit|doDelete|postBuildResult|cancelQueue|stop|toggleLogKeep|doWipeOutWorkspace|createItem|createView|toggleOffline)</pattern>
              <logBuildCause>true</logBuildCause>
              <loggers>
                <hudson.plugins.audit__trail.LogFileAuditLogger>
                  <log>/data/jenkins/logs/audit-trail.log</log>
                  <limit>500000000</limit>
                  <count>1</count>
                </hudson.plugins.audit__trail.LogFileAuditLogger>
              </loggers>
            </hudson.plugins.audit__trail.AuditTrailPlugin>

            [jenkins ~]$ ls -altr /data/jenkins/logs/audit-trail.log*
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:05 /data/jenkins/logs/audit-trail.log.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.1.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.1
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.2.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.2
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:07 /data/jenkins/logs/audit-trail.log.3.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:14 /data/jenkins/logs/audit-trail.log.3
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:14 /data/jenkins/logs/audit-trail.log.4.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.4
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.5.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.5
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6.lck
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6
            {code}

            The next issue has to do with the limit parameter. In the Jenkins console configuration screen, it says says "Log File Size MB" whereas the JavaDoc for FileHandler says the limit parameter is "the maximum number of bytes to write to any one file". As mentioned above, I used a limit of 500000000 assuming it is actually using bytes. However since the rotation isn't working correctly, I have no way of verifying whether or not the plugin does a conversion from MB to bytes.
            There are multiple issues with the Log File logger in the audit-trail plugin.

            The first has to do with log file rotation. I set the count parameter to 1 and the limit to 500000000. Yet the log file is still being rotated every time more than two lines are written:

            {code}
            [jenkins ~]$ cat /data/jenkins/audit-trail.xml
            <?xml version='1.0' encoding='UTF-8'?>
            <hudson.plugins.audit__trail.AuditTrailPlugin plugin="audit-trail@2.1">
              <pattern>.*/(?:configSubmit|doDelete|postBuildResult|cancelQueue|stop|toggleLogKeep|doWipeOutWorkspace|createItem|createView|toggleOffline)</pattern>
              <logBuildCause>true</logBuildCause>
              <loggers>
                <hudson.plugins.audit__trail.LogFileAuditLogger>
                  <log>/data/jenkins/logs/audit-trail.log</log>
                  <limit>500000000</limit>
                  <count>1</count>
                </hudson.plugins.audit__trail.LogFileAuditLogger>
              </loggers>
            </hudson.plugins.audit__trail.AuditTrailPlugin>

            [jenkins ~]$ ls -altr /data/jenkins/logs/audit-trail.log*
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:05 /data/jenkins/logs/audit-trail.log.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.1.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.1
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.2.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.2
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:07 /data/jenkins/logs/audit-trail.log.3.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:14 /data/jenkins/logs/audit-trail.log.3
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:14 /data/jenkins/logs/audit-trail.log.4.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.4
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.5.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.5
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6.lck
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6
            {code}

            The next issue has to do with the limit parameter. In the Jenkins console configuration screen, it says says "Log File Size MB" whereas the JavaDoc for FileHandler says the limit parameter is "the maximum number of bytes to write to any one file". As mentioned above, I used a limit of 500000000 assuming it is actually using bytes. However since the rotation isn't working correctly, I have no way of verifying whether or not the plugin does a conversion from MB to bytes.

            I should note that I also did try limit values of 512 and 1024, but still am seeing the logs rotate after two lines are written.
            mnejman Mark Nejman made changes -
            Description There are multiple issues with the Log File logger in the audit-trail plugin.

            The first has to do with log file rotation. I set the count parameter to 1 and the limit to 500000000. Yet the log file is still being rotated every time more than two lines are written:

            {code}
            [jenkins ~]$ cat /data/jenkins/audit-trail.xml
            <?xml version='1.0' encoding='UTF-8'?>
            <hudson.plugins.audit__trail.AuditTrailPlugin plugin="audit-trail@2.1">
              <pattern>.*/(?:configSubmit|doDelete|postBuildResult|cancelQueue|stop|toggleLogKeep|doWipeOutWorkspace|createItem|createView|toggleOffline)</pattern>
              <logBuildCause>true</logBuildCause>
              <loggers>
                <hudson.plugins.audit__trail.LogFileAuditLogger>
                  <log>/data/jenkins/logs/audit-trail.log</log>
                  <limit>500000000</limit>
                  <count>1</count>
                </hudson.plugins.audit__trail.LogFileAuditLogger>
              </loggers>
            </hudson.plugins.audit__trail.AuditTrailPlugin>

            [jenkins ~]$ ls -altr /data/jenkins/logs/audit-trail.log*
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:05 /data/jenkins/logs/audit-trail.log.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.1.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.1
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.2.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.2
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:07 /data/jenkins/logs/audit-trail.log.3.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:14 /data/jenkins/logs/audit-trail.log.3
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:14 /data/jenkins/logs/audit-trail.log.4.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.4
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.5.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.5
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6.lck
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6
            {code}

            The next issue has to do with the limit parameter. In the Jenkins console configuration screen, it says says "Log File Size MB" whereas the JavaDoc for FileHandler says the limit parameter is "the maximum number of bytes to write to any one file". As mentioned above, I used a limit of 500000000 assuming it is actually using bytes. However since the rotation isn't working correctly, I have no way of verifying whether or not the plugin does a conversion from MB to bytes.

            I should note that I also did try limit values of 512 and 1024, but still am seeing the logs rotate after two lines are written.
            There is a log rotation issue with the Log File logger in the audit-trail plugin.

            I set the count parameter to 1 and the limit to 512. Yet the log file is still being rotated every time more than two lines are written:

            {code}
            [jenkins ~]$ cat /data/jenkins/audit-trail.xml
            <?xml version='1.0' encoding='UTF-8'?>
            <hudson.plugins.audit__trail.AuditTrailPlugin plugin="audit-trail@2.1">
              <pattern>.*/(?:configSubmit|doDelete|postBuildResult|cancelQueue|stop|toggleLogKeep|doWipeOutWorkspace|createItem|createView|toggleOffline)</pattern>
              <logBuildCause>true</logBuildCause>
              <loggers>
                <hudson.plugins.audit__trail.LogFileAuditLogger>
                  <log>/data/jenkins/logs/audit-trail.log</log>
                  <limit>512</limit>
                  <count>1</count>
                </hudson.plugins.audit__trail.LogFileAuditLogger>
              </loggers>
            </hudson.plugins.audit__trail.AuditTrailPlugin>

            [jenkins ~]$ ls -altr /data/jenkins/logs/audit-trail.log*
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:05 /data/jenkins/logs/audit-trail.log.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.1.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.1
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:06 /data/jenkins/logs/audit-trail.log.2.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:06 /data/jenkins/logs/audit-trail.log.2
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:07 /data/jenkins/logs/audit-trail.log.3.lck
            -rw-r--r--. 1 jenkins jenkins 97 May 7 11:14 /data/jenkins/logs/audit-trail.log.3
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:14 /data/jenkins/logs/audit-trail.log.4.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.4
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.5.lck
            -rw-r--r--. 1 jenkins jenkins 115 May 7 11:15 /data/jenkins/logs/audit-trail.log.5
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6.lck
            -rw-r--r--. 1 jenkins jenkins 0 May 7 11:15 /data/jenkins/logs/audit-trail.log.6
            {code}

            It appears the log rotation count is not working correctly.
            Summary Audit Trail Plugin log file issues Audit Trail Plugin log file rotation issue
            mnejman Mark Nejman added a comment -

            I see there are multiple .lck files that are opened and do not go away. From reading up on FileHandler, that typically indicates multiple instances of FileHandler being instantiated. They say that is typically caused by multiple JVMs writing to the same log file. However, I am only using one JVM (verified with ps).

            I'm wondering if the issue is in hudson.plugins.audit_trail.LogFileAuditLogger line 21:

            private transient FileHandler handler;
            

            I'm thinkning maybe the extension creates several instances of the audit-trail logger, and therefore multiple FileHandler instances trying to write to the same log file.

            Is there a reason this attribute is marked as transient? I didn't think this was a serializable object. Perhaps the solution would be to mark it as static so that multiple instances do not exist in the same container.

            mnejman Mark Nejman added a comment - I see there are multiple .lck files that are opened and do not go away. From reading up on FileHandler, that typically indicates multiple instances of FileHandler being instantiated. They say that is typically caused by multiple JVMs writing to the same log file. However, I am only using one JVM (verified with ps). I'm wondering if the issue is in hudson.plugins.audit_trail.LogFileAuditLogger line 21: private transient FileHandler handler; I'm thinkning maybe the extension creates several instances of the audit-trail logger, and therefore multiple FileHandler instances trying to write to the same log file. Is there a reason this attribute is marked as transient? I didn't think this was a serializable object. Perhaps the solution would be to mark it as static so that multiple instances do not exist in the same container.
            mnejman Mark Nejman made changes -
            Assignee Nicolas De Loof [ ndeloof ]
            mnejman Mark Nejman made changes -
            Assignee Nicolas De Loof [ ndeloof ]
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 163122 ] JNJira + In-Review [ 181107 ]
            pierrebtz Pierre Beitz added a comment -

            Fixed by JENKINS-44129

            pierrebtz Pierre Beitz added a comment - Fixed by  JENKINS-44129
            pierrebtz Pierre Beitz made changes -
            Link This issue duplicates JENKINS-44129 [ JENKINS-44129 ]
            pierrebtz Pierre Beitz made changes -
            Assignee Pierre Beitz [ pierrebtz ]
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]

            People

              pierrebtz Pierre Beitz
              mnejman Mark Nejman
              Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: