Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-35021

JobConfigHistory: Max number of days to keep history entries parameter: Does not work as expected

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • Jenkins ver. 1.642.4.2
      Job Configuration History Plugin 2.14

      When you want to set limits for "Max number of history entries to keep" and "Max number of days to keep history entries" in order to reduce the size of the "config-history"

      • "Max number of history entries to keep" works very well once you overcome the limits the oldest records are removed.
      • However, "Max number of days to keep history entries" seems not to work. It does not eliminate the records older that the specified limit.

          [JENKINS-35021] JobConfigHistory: Max number of days to keep history entries parameter: Does not work as expected

          Hi Carlos,

          could you provide a log of this (FINEST)?
          I tried it and can't reproduce it (Jenkins 2.4, JobConfigHistory 2.14). The periodic checker runs once a day.
          NOTE: The periodic check does NOT delete entries with operation "created".

          Jochen

          Jochen A. Fürbacher added a comment - Hi Carlos, could you provide a log of this (FINEST)? I tried it and can't reproduce it (Jenkins 2.4, JobConfigHistory 2.14). The periodic checker runs once a day. NOTE: The periodic check does NOT delete entries with operation "created". Jochen

          If someone can reproduce this issue, feel free to reopen the ticket.

          Jochen A. Fürbacher added a comment - If someone can reproduce this issue, feel free to reopen the ticket.

          Thanks for your feedback.

          Currently we are working on providing those logs for the Job Config History plugin as explained (https://wiki.jenkins-ci.org/display/JENKINS/Logging), including `hudson.plugins.jobConfigHistory` at level `FINEST`.

          After creating the log, we would like to test the `Max number of days to keep history entries`. For instance, setting it to 1 day and leave for 2 days to see what is printed on the log. Do you think is a good approach?

          Another question is What do you mean with The periodic check does NOT delete entries with operation "created".??

          Best regards

          Carlos Rodríguez López added a comment - Thanks for your feedback. Currently we are working on providing those logs for the Job Config History plugin as explained ( https://wiki.jenkins-ci.org/display/JENKINS/Logging ), including `hudson.plugins.jobConfigHistory` at level `FINEST`. After creating the log, we would like to test the `Max number of days to keep history entries`. For instance, setting it to 1 day and leave for 2 days to see what is printed on the log. Do you think is a good approach? Another question is What do you mean with The periodic check does NOT delete entries with operation "created".?? Best regards

          18-config-history-defect.zip

          Attached logs as requested.

          Regards,

          Carlos Rodríguez López added a comment - 18-config-history-defect.zip Attached logs as requested. Regards,

          Jochen A. Fürbacher added a comment - - edited

          Hi Carlos,

          thanks for providing the logs.

          If you check your log file Config-History-Defect.log.8 then you can see after line 2096, that some of the entries get deleted:

          {{2016-07-08 04:11:41.865+0100 [id=96] FINE h.p.j.JobConfigHistoryPurger#doRun: checking for history files to purge (max age of 1 days allowed)
          2016-07-08 04:11:41.869+0100 [id=96] FINEST h.p.j.FileHistoryDao#isCreatedEntry: historyDir: C:\Jenkins\config-history\com.cloudbees.jenkins.plugins.requestfilter.Rules\2016-07-06_15-48-59
          2016-07-08 04:11:41.869+0100 [id=96] FINEST h.p.j.FileHistoryDao#isCreatedEntry: histDescr.getOperation(): Changed
          2016-07-08 04:11:41.869+0100 [id=96] FINEST h.p.j.JobConfigHistoryPurger#purgeSystemOrJobHistory: Should delete: C:\Jenkins\config-history\com.cloudbees.jenkins.plugins.requestfilter.Rules\2016-07-06_15-48-59}}

          So these history entries get deleted and should'n get shown anymore.

          However, history entries with operation Created doesn't get deleted, even when they are older than the configuried max time. E.g.:
          You create a job on July 15th. Then you do a couple of config changed of the job. When you have JobConfigHistory configured to delete all entries after 2 days and it's for instance July 20th, then all entries of this job get deleted. But the entry, which contains the Created operation, doesn't get deleted. It stays remain in the history.

          You can test it with setting it to 1 day and leave it for 2 days. When you check your log, then you should see, that JobConfigHistory tries to deletes some entries (containing rows JobConfigHistoryPurger#doRun: checking for history files to purge (max age of 1 days allowed) and {{JobConfigHistoryPurger#purgeSystemOrJobHistory: Should delete: }})

          Jochen

          Jochen A. Fürbacher added a comment - - edited Hi Carlos, thanks for providing the logs. If you check your log file Config-History-Defect.log.8 then you can see after line 2096, that some of the entries get deleted: {{2016-07-08 04:11:41.865+0100 [id=96] FINE h.p.j.JobConfigHistoryPurger#doRun: checking for history files to purge (max age of 1 days allowed) 2016-07-08 04:11:41.869+0100 [id=96] FINEST h.p.j.FileHistoryDao#isCreatedEntry: historyDir: C:\Jenkins\config-history\com.cloudbees.jenkins.plugins.requestfilter.Rules\2016-07-06_15-48-59 2016-07-08 04:11:41.869+0100 [id=96] FINEST h.p.j.FileHistoryDao#isCreatedEntry: histDescr.getOperation(): Changed 2016-07-08 04:11:41.869+0100 [id=96] FINEST h.p.j.JobConfigHistoryPurger#purgeSystemOrJobHistory: Should delete: C:\Jenkins\config-history\com.cloudbees.jenkins.plugins.requestfilter.Rules\2016-07-06_15-48-59}} So these history entries get deleted and should'n get shown anymore. However, history entries with operation Created doesn't get deleted, even when they are older than the configuried max time. E.g.: You create a job on July 15th. Then you do a couple of config changed of the job. When you have JobConfigHistory configured to delete all entries after 2 days and it's for instance July 20th, then all entries of this job get deleted. But the entry, which contains the Created operation, doesn't get deleted. It stays remain in the history. You can test it with setting it to 1 day and leave it for 2 days. When you check your log, then you should see, that JobConfigHistory tries to deletes some entries (containing rows JobConfigHistoryPurger#doRun: checking for history files to purge (max age of 1 days allowed) and {{JobConfigHistoryPurger#purgeSystemOrJobHistory: Should delete: }}) Jochen

          Judah Greenblatt added a comment - - edited

          Reading the code of the plugin shows that the maximum number of days to keep entries is applied to Nodes, Configuration files, and to jobs AT THE TOP LEVEL (not in Folders).  This matches what I observe when configuring the maximum number of days to keep entries on several large Jenkins servers.

          The error is that method FileHistoryDao.getJobs returns all directories in directory config-history/jobs/, but does not recurse into sub-directories of the returned directories named 'jobs'.

          The problem is that fully recursing through the config-history directory hierarchy would take an excessive amount of time on our servers (with perhaps 1000 folders, 20,000 jobs, and 2 million history entries before cleanup)

          Judah Greenblatt added a comment - - edited Reading the code of the plugin shows that the maximum number of days to keep entries is applied to Nodes, Configuration files, and to jobs AT THE TOP LEVEL (not in Folders).  This matches what I observe when configuring the maximum number of days to keep entries on several large Jenkins servers. The error is that method FileHistoryDao.getJobs returns all directories in directory config-history/jobs/, but does not recurse into sub-directories of the returned directories named 'jobs'. The problem is that fully recursing through the config-history directory hierarchy would take an excessive amount of time on our servers (with perhaps 1000 folders, 20,000 jobs, and 2 million history entries before cleanup)

            stefanbrausch Stefan Brausch
            carlosrodlop Carlos Rodríguez López
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: