• Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Jenkins 1.656
       jobConfigHistory plugin 2.14

      I see exceptions similar to the following in our Jenkins system log. We are running Jenkins 1.656. I asked for help first on the mailing list but got no reply. I am not sure which component is at fault - perhaps the jobConfigHistory plugin?

      java.lang.RuntimeException: Could not create rootDir /var/lib/jenkins/config-history/jobs/Zodiac_gcc_all_branches_and_trunk/branches/trunk/2016-06-06_14-37-52
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryDir(FileHistoryDao.java:241)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.getRootDir(FileHistoryDao.java:129)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryEntry(FileHistoryDao.java:409)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryEntryAndCopyConfig(FileHistoryDao.java:259)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.saveItem(FileHistoryDao.java:270)
      	at hudson.plugins.jobConfigHistory.JobConfigHistorySaveableListener.onChange(JobConfigHistorySaveableListener.java:54)
      	at hudson.model.listeners.SaveableListener.fireOnChange(SaveableListener.java:81)
      	at hudson.model.AbstractItem.save(AbstractItem.java:513)
      	at hudson.model.Job.save(Job.java:180)
      	at hudson.model.AbstractProject.save(AbstractProject.java:305)
      	at hudson.model.AbstractProject.setScm(AbstractProject.java:1563)
      	at com.github.mjdetullio.jenkins.plugins.multibranch.AbstractMultiBranchProject._syncBranches(AbstractMultiBranchProject.java:1038)
      	at com.github.mjdetullio.jenkins.plugins.multibranch.AbstractMultiBranchProject.syncBranches(AbstractMultiBranchProject.java:948)
      	at com.github.mjdetullio.jenkins.plugins.multibranch.SyncBranchesTrigger.run(SyncBranchesTrigger.java:102)
      	at com.github.mjdetullio.jenkins.plugins.multibranch.SyncBranchesTrigger.checkTriggers(SyncBranchesTrigger.java:255)
      	at com.github.mjdetullio.jenkins.plugins.multibranch.SyncBranchesTrigger$Cron.doRun(SyncBranchesTrigger.java:230)
      	at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:50)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      

          [JENKINS-35421] Exception: "Could not create rootDir"

          David Aldrich added a comment -

          Thanks for your reply.

          That directory does not exist. The directory specified by the exceptions has many different values but they are all for June:

          /var/lib/jenkins/config-history/jobs/Zodiac_gcc_all_branches_and_trunk/branches/trunk/2016-06*
          

          and none of them exist.

          David Aldrich added a comment - Thanks for your reply. That directory does not exist. The directory specified by the exceptions has many different values but they are all for June: / var /lib/jenkins/config-history/jobs/Zodiac_gcc_all_branches_and_trunk/branches/trunk/2016-06* and none of them exist.

          David Aldrich added a comment -

          Can you help me further with this please?

          David Aldrich added a comment - Can you help me further with this please?

          The multi branch plugin has a very own kind to use job configurations and this isn't compatible with the kind how the config history plugin normally handle it.
          At them moment we have no solutions for different problems with this plugin but we work on this.

          Please be patient.

          Stefan Brausch added a comment - The multi branch plugin has a very own kind to use job configurations and this isn't compatible with the kind how the config history plugin normally handle it. At them moment we have no solutions for different problems with this plugin but we work on this. Please be patient.

          David Aldrich added a comment -

          No problem, thank you for your answer. Perhaps I should disable the job config history plugin for now.

          David Aldrich added a comment - No problem, thank you for your answer. Perhaps I should disable the job config history plugin for now.

          I tried to reproduce this issue. JobConfigHistory stores config changes on the main project as well as on all branches and I don't see any exception like that one above. All directories exist.

          You mentioned, that the problematic jobs are all from June. Did you use JobConfigHistory before June without problems?

          Does this exception only occur on multi branch jobs or on all jobs?

          Jochen A. Fürbacher added a comment - I tried to reproduce this issue. JobConfigHistory stores config changes on the main project as well as on all branches and I don't see any exception like that one above. All directories exist. You mentioned, that the problematic jobs are all from June. Did you use JobConfigHistory before June without problems? Does this exception only occur on multi branch jobs or on all jobs?

          David Aldrich added a comment -

          Thanks for investigating this issue. I have disabled the JobConfigHistory plugin and restarted the system, so I no longer have the system log.

          Yes, I used JobConfigHistory before June without problems.

          I think the exception only occurred on multi-branch jobs but I can't be sure.

          David Aldrich added a comment - Thanks for investigating this issue. I have disabled the JobConfigHistory plugin and restarted the system, so I no longer have the system log. Yes, I used JobConfigHistory before June without problems. I think the exception only occurred on multi-branch jobs but I can't be sure.

          If someone can explain or reproduce this scenario, then feel free to reopen this ticket.

          Jochen A. Fürbacher added a comment - If someone can explain or reproduce this scenario, then feel free to reopen this ticket.

          Robert Baker added a comment -

          I'm encountering a similar issue to this one in my environment:

          SLES 11 SP4
          Jenkins 2.89.4
          Job Configuration History Plugin 2.18
          Docker Plugin 1.1.4

          I've configured a Cloud using the Docker plugin and in that cloud I've defined a Jenkins slave container with a label of "SLES12SLAVEJDK8".  When I run a job on this slave, however, the slave agent is never invoked and the job fails with the following error:

          SEVERE: Unexpected uncaught exception encountered while processing agent Image of ustr-docker.na.uis.unisys.com/singularity/sles12-slave-jdk8:latest
          java.lang.RuntimeException: Could not create rootDir /root/.jenkins/config-history/nodes/docker-111b76fe6f9/2018-05-22_14-15-48
                  at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryDir(FileHistoryDao.java:266)
                  at hudson.plugins.jobConfigHistory.FileHistoryDao.getRootDir(FileHistoryDao.java:899)
                  at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryEntry(FileHistoryDao.java:906)
                  at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryEntryAndSaveConfig(FileHistoryDao.java:790)
                  at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewNode(FileHistoryDao.java:772)
                  at hudson.plugins.jobConfigHistory.ComputerHistoryListener.onAdd(ComputerHistoryListener.java:82)
                  at hudson.plugins.jobConfigHistory.ComputerHistoryListener.onConfigurationChange(ComputerHistoryListener.java:59)
                  at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:231)
                  at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1551)
                  at jenkins.model.Nodes$2.run(Nodes.java:137)
                  at hudson.model.Queue._withLock(Queue.java:1373)
                  at hudson.model.Queue.withLock(Queue.java:1250)
                  at jenkins.model.Nodes.addNode(Nodes.java:133)
                  at jenkins.model.Jenkins.addNode(Jenkins.java:2094)
                  at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:241)
                  at hudson.model.Queue._withLock(Queue.java:1373)
                  at hudson.model.Queue.withLock(Queue.java:1250)
                  at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:207)
                  at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)
                  at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:809)
                  at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)
                  at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)
                  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
                  at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
                  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
                  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
                  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                  at java.lang.Thread.run(Thread.java:748)

          Apparently the dynamic creation of the slave causes the job configuration history plugin to be invoked to save the configuration changes, however, no folder exists for the docker slave (docker-111b76fe6f9) in the node configuration history folder (/root/.jenkins/config-history/nodes).  This makes the job configuration history plugin unusable in this environment, and so I had to disable it.

          Robert Baker added a comment - I'm encountering a similar issue to this one in my environment: SLES 11 SP4 Jenkins 2.89.4 Job Configuration History Plugin 2.18 Docker Plugin 1.1.4 I've configured a Cloud using the Docker plugin and in that cloud I've defined a Jenkins slave container with a label of "SLES12SLAVEJDK8".  When I run a job on this slave, however, the slave agent is never invoked and the job fails with the following error: SEVERE: Unexpected uncaught exception encountered while processing agent Image of ustr-docker.na.uis.unisys.com/singularity/sles12-slave-jdk8:latest java.lang.RuntimeException: Could not create rootDir /root/.jenkins/config-history/nodes/docker-111b76fe6f9/2018-05-22_14-15-48         at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryDir(FileHistoryDao.java:266)         at hudson.plugins.jobConfigHistory.FileHistoryDao.getRootDir(FileHistoryDao.java:899)         at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryEntry(FileHistoryDao.java:906)         at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewHistoryEntryAndSaveConfig(FileHistoryDao.java:790)         at hudson.plugins.jobConfigHistory.FileHistoryDao.createNewNode(FileHistoryDao.java:772)         at hudson.plugins.jobConfigHistory.ComputerHistoryListener.onAdd(ComputerHistoryListener.java:82)         at hudson.plugins.jobConfigHistory.ComputerHistoryListener.onConfigurationChange(ComputerHistoryListener.java:59)         at hudson.model.AbstractCIBase.updateComputerList(AbstractCIBase.java:231)         at jenkins.model.Jenkins.updateComputerList(Jenkins.java:1551)         at jenkins.model.Nodes$2.run(Nodes.java:137)         at hudson.model.Queue._withLock(Queue.java:1373)         at hudson.model.Queue.withLock(Queue.java:1250)         at jenkins.model.Nodes.addNode(Nodes.java:133)         at jenkins.model.Jenkins.addNode(Jenkins.java:2094)         at hudson.slaves.NodeProvisioner$2.run(NodeProvisioner.java:241)         at hudson.model.Queue._withLock(Queue.java:1373)         at hudson.model.Queue.withLock(Queue.java:1250)         at hudson.slaves.NodeProvisioner.update(NodeProvisioner.java:207)         at hudson.slaves.NodeProvisioner.access$000(NodeProvisioner.java:61)         at hudson.slaves.NodeProvisioner$NodeProvisionerInvoker.doRun(NodeProvisioner.java:809)         at hudson.triggers.SafeTimerTask.run(SafeTimerTask.java:51)         at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:58)         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)         at java.lang.Thread.run(Thread.java:748) Apparently the dynamic creation of the slave causes the job configuration history plugin to be invoked to save the configuration changes, however, no folder exists for the docker slave (docker-111b76fe6f9) in the node configuration history folder (/root/.jenkins/config-history/nodes).  This makes the job configuration history plugin unusable in this environment, and so I had to disable it.

          Henrique Gontijo added a comment - - edited

          rsbaker56, I faced similar issue as yours. I'm also running Jenkins on Docker. For me, the problem was caused by a directory configured on Job Configuration history that was pointing to a host folder which was not visible on the container. That happened because I moved a Jenkins instance that was running directly on a VM to run on a Docker container. The configuration is on Jenkins > Manage Jenkins > Job Config History.

          I configured it to use the default folder and it worked.

          Henrique Gontijo added a comment - - edited rsbaker56 , I faced similar issue as yours. I'm also running Jenkins on Docker. For me, the problem was caused by a directory configured on Job Configuration history that was pointing to a host folder which was not visible on the container. That happened because I moved a Jenkins instance that was running directly on a VM to run on a Docker container. The configuration is on Jenkins > Manage Jenkins > Job Config History. I configured it to use the default folder and it worked.

          Ian Williams added a comment -

          rsbaker56, jochenafuerbacher, davida2009 I created JENKINS-66533 to track my instance of this error and found the cause. If you found "the directory did not exist", it may be a similar manifestation.

          Ian Williams added a comment - rsbaker56 , jochenafuerbacher , davida2009 I created JENKINS-66533 to track my instance of this error and found the cause. If you found "the directory did not exist", it may be a similar manifestation.

            stefanbrausch Stefan Brausch
            davida2009 David Aldrich
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: