Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-14583

Improve validation of 'builds dir' global setting

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Minor Minor
    • core
    • None

      I am using Jenkins 1.475 with the perforce plugin.
      I have several jobs using different perforce workspaces and different Jenkins workspaces.
      For some reason the build history for each job includes the builds of other jobs. This may be related to JENKINS-11853.
      The last failed and last successful builds seem to be determined by build number rather than date.

      If you look at the second and third snapshots you can see that Jenkins alternates in its decision over what builds to include in the the history on the left hand side. This may be related or a separate bug.

      I have given each job a custom workspace with a unique name to avoid conflicts. Further I have installed the set next build number plugin and given each job a next build number +1000 from the others. This seemed to work until I was forced to restart Jenkins (due to a hardware issue) after which the problem has reoccurred. This scheme would be unviable in general anyway after any one project had more than 1000 builds.

      This may be down to a misconfiguration of some kind on my part but it is not at all obvious what is wrong or how to fix it.
      A contributing factor could be that some jobs were created by copying existing jobs. Perhaps this caused something to be shared that shouldn't be?

        1. jenkins1.png
          180 kB
          Bruce Adams
        2. jenkins2.png
          175 kB
          Bruce Adams
        3. jenkins3.png
          190 kB
          Bruce Adams

          [JENKINS-14583] Improve validation of 'builds dir' global setting

          Bruce Adams added a comment -

          I note that when I create a new job from an existing one the build history remains visible on the right hand side even if a new workspace is defined. Could this be the cause or at least related?

          Bruce Adams added a comment - I note that when I create a new job from an existing one the build history remains visible on the right hand side even if a new workspace is defined. Could this be the cause or at least related?

          Bruce Adams added a comment - - edited

          Missing builds caused by working around this issue cause builder threads to crash in the latest version of Jenkins. See JENKINS-15957

          Bruce Adams added a comment - - edited Missing builds caused by working around this issue cause builder threads to crash in the latest version of Jenkins. See JENKINS-15957

          Bruce Adams added a comment -

          I am still stuck with builds for different projects and/or configurations which make it look like the job is succeeding when it is failing or failing when it is succeeding because the latest build displayed on for the dashboard is in fact a completely different job.

          Can anyone suggest a workaround to this issue?

          Bruce Adams added a comment - I am still stuck with builds for different projects and/or configurations which make it look like the job is succeeding when it is failing or failing when it is succeeding because the latest build displayed on for the dashboard is in fact a completely different job. Can anyone suggest a workaround to this issue?

          Bruce Adams added a comment -

          I asked on the mailing list and go a much more rapid response.

          For some reason I had a fixed hard-code directory
          rather than the default:

          ${ITEM_ROOTDIR}/builds

          I must have tweaked this very early on when I installed and not touched it since.
          For extra paranoia I've now changed this to:

          ${ITEM_ROOTDIR}/builds/${ITEM_FULLNAME}

          So far things look promising.

          I am therefore re-purposing this issue as a request to make the location of build histories more obvious. Perhaps a link or something in a help bubble could be added to the configuration page for a job in the section containing:

          Discard Old Builds
          Days to keep builds
          Days to keep artifacts
          Max # of builds to keep with artifacts

          This is where I would expect to see the location of the build history displayed if it were independently configured per job rather than globally (that may make sense but is not a feature I require myself and is possibly a YAGNI).

          Bruce Adams added a comment - I asked on the mailing list and go a much more rapid response. For some reason I had a fixed hard-code directory rather than the default: ${ITEM_ROOTDIR}/builds I must have tweaked this very early on when I installed and not touched it since. For extra paranoia I've now changed this to: ${ITEM_ROOTDIR}/builds/${ITEM_FULLNAME} So far things look promising. I am therefore re-purposing this issue as a request to make the location of build histories more obvious. Perhaps a link or something in a help bubble could be added to the configuration page for a job in the section containing: Discard Old Builds Days to keep builds Days to keep artifacts Max # of builds to keep with artifacts This is where I would expect to see the location of the build history displayed if it were independently configured per job rather than globally (that may make sense but is not a feature I require myself and is possibly a YAGNI).

          Daniel Beck added a comment -

          Input validation of global builds dir name does not catch this, but should.

          Interpreting this issue as request to improve global config input validation.

          Daniel Beck added a comment - Input validation of global builds dir name does not catch this, but should. Interpreting this issue as request to improve global config input validation.

          Daniel Beck added a comment -

          Daniel Beck added a comment - https://github.com/jenkinsci/jenkins/pull/1251

          Paul Becotte added a comment -

          I was having a problem with all of my build history getting repeatedly deleted - JENKINS-23067. I noticed that the buildsdir directory was non-standard. I changed it back to the default value and have not seen the issue since. In my case it was set as follows-

          <workspaceDir>${JENKINS_HOME}/workspace</workspaceDir>
          <buildsDir>${JENKINS_HOME}/workspace/builds/${ITEM_FULLNAME}</buildsDir>

          (I didn't set it up originally, so I couldn't guess as to why)
          I am not sure that this was the issue, but it also made the thinbackup plugin correctly back up the history so I won't lose it if the problem does resurface. The deletion seemed to happen very early in the morning, perhaps on a server restart (if that helps in any way)

          Paul Becotte added a comment - I was having a problem with all of my build history getting repeatedly deleted - JENKINS-23067 . I noticed that the buildsdir directory was non-standard. I changed it back to the default value and have not seen the issue since. In my case it was set as follows- <workspaceDir>${JENKINS_HOME}/workspace</workspaceDir> <buildsDir>${JENKINS_HOME}/workspace/builds/${ITEM_FULLNAME}</buildsDir> (I didn't set it up originally, so I couldn't guess as to why) I am not sure that this was the issue, but it also made the thinbackup plugin correctly back up the history so I won't lose it if the problem does resurface. The deletion seemed to happen very early in the morning, perhaps on a server restart (if that helps in any way)

          Daniel Beck added a comment -

          Paul: In case it's not obvious what these settings do:

          • You cause all jobs in your entire Jenkins to use the same workspace directory. Not "in the same directory tree", but the same directory.
          • Inside that workspace directory, you store all your build data.

          You probably have a job or two that run over night and delete their workspace (e.g. Workspace Cleanup plugin), and with it, all your build records.

          I'm sorry, but that's too messed up for me to add input validation for.

          Daniel Beck added a comment - Paul: In case it's not obvious what these settings do: You cause all jobs in your entire Jenkins to use the same workspace directory. Not "in the same directory tree", but the same directory . Inside that workspace directory , you store all your build data . You probably have a job or two that run over night and delete their workspace (e.g. Workspace Cleanup plugin), and with it, all your build records. I'm sorry, but that's too messed up for me to add input validation for.

          Paul Becotte added a comment -

          lol- as I said, I didn't set it up- I didn't realize that was something you could change until I was investigating this!

          I hadn't realized the implication of setting the workspace that way- that it forced everything to use a common workspace. I didn't use the master to build anything, so it may not have mattered (executors are set to 0)- but I do wonder if that is what wiped out the build history? Would the master ever wipe out its workspace as part of the normal order of business?

          Feel free to not add validation for that... I just wanted to contribute the information in case it was helpful.

          Paul Becotte added a comment - lol- as I said, I didn't set it up- I didn't realize that was something you could change until I was investigating this! I hadn't realized the implication of setting the workspace that way- that it forced everything to use a common workspace. I didn't use the master to build anything, so it may not have mattered (executors are set to 0)- but I do wonder if that is what wiped out the build history? Would the master ever wipe out its workspace as part of the normal order of business? Feel free to not add validation for that... I just wanted to contribute the information in case it was helpful.

          Daniel Beck added a comment -

          Paul: Right, thanks for the information in any case.

          Regarding builds on master and workspaces: There's a workspace cleanup thread that logs to hudson.model.WorkspaceCleanupThread on levels FINE and FINER. It will tell you if it's the cause of this. Disable it by setting the Java system property -Dhudson.model.WorkspaceCleanupThread.disabled=true.

          Daniel Beck added a comment - Paul: Right, thanks for the information in any case. Regarding builds on master and workspaces: There's a workspace cleanup thread that logs to hudson.model.WorkspaceCleanupThread on levels FINE and FINER. It will tell you if it's the cause of this. Disable it by setting the Java system property -Dhudson.model.WorkspaceCleanupThread.disabled=true .

          Code changed in jenkins
          User: Daniel Beck
          Path:
          core/src/main/java/jenkins/model/Jenkins.java
          http://jenkins-ci.org/commit/jenkins/0b556c0fa8dd0ebd1851c3b37025e2c0321fafe8
          Log:
          JENKINS-14583 Move replacement into shared function for reuse

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/jenkins/model/Jenkins.java http://jenkins-ci.org/commit/jenkins/0b556c0fa8dd0ebd1851c3b37025e2c0321fafe8 Log: JENKINS-14583 Move replacement into shared function for reuse

          Code changed in jenkins
          User: Daniel Beck
          Path:
          core/src/main/java/jenkins/model/Jenkins.java
          core/src/test/java/jenkins/model/JenkinsDescriptorTest.java
          http://jenkins-ci.org/commit/jenkins/50cbded7f8636b5dc745c4ee8cae23af3ee110f8
          Log:
          JENKINS-14583 Fix tests

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/jenkins/model/Jenkins.java core/src/test/java/jenkins/model/JenkinsDescriptorTest.java http://jenkins-ci.org/commit/jenkins/50cbded7f8636b5dc745c4ee8cae23af3ee110f8 Log: JENKINS-14583 Fix tests

          Code changed in jenkins
          User: Daniel Beck
          Path:
          core/src/main/java/jenkins/model/Jenkins.java
          core/src/test/java/jenkins/model/JenkinsDescriptorTest.java
          http://jenkins-ci.org/commit/jenkins/5751560fc186ff4add58f9bc7b8eab20a682b09f
          Log:
          Merge pull request #1251 from daniel-beck/JENKINS-14583

          [FIX JENKINS-14538] Improve builds dir input validation

          Compare: https://github.com/jenkinsci/jenkins/compare/e8b1a500c9b9...5751560fc186

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/jenkins/model/Jenkins.java core/src/test/java/jenkins/model/JenkinsDescriptorTest.java http://jenkins-ci.org/commit/jenkins/5751560fc186ff4add58f9bc7b8eab20a682b09f Log: Merge pull request #1251 from daniel-beck/ JENKINS-14583 [FIX JENKINS-14538] Improve builds dir input validation Compare: https://github.com/jenkinsci/jenkins/compare/e8b1a500c9b9...5751560fc186

          Daniel Beck added a comment -

          Merged towards 1.575.

          Daniel Beck added a comment - Merged towards 1.575.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3551
          JENKINS-14583 Move replacement into shared function for reuse (Revision 0b556c0fa8dd0ebd1851c3b37025e2c0321fafe8)
          JENKINS-14583 Fix tests (Revision 50cbded7f8636b5dc745c4ee8cae23af3ee110f8)

          Result = SUCCESS
          daniel-beck : 0b556c0fa8dd0ebd1851c3b37025e2c0321fafe8
          Files :

          • core/src/main/java/jenkins/model/Jenkins.java

          daniel-beck : 50cbded7f8636b5dc745c4ee8cae23af3ee110f8
          Files :

          • core/src/main/java/jenkins/model/Jenkins.java
          • core/src/test/java/jenkins/model/JenkinsDescriptorTest.java

          dogfood added a comment - Integrated in jenkins_main_trunk #3551 JENKINS-14583 Move replacement into shared function for reuse (Revision 0b556c0fa8dd0ebd1851c3b37025e2c0321fafe8) JENKINS-14583 Fix tests (Revision 50cbded7f8636b5dc745c4ee8cae23af3ee110f8) Result = SUCCESS daniel-beck : 0b556c0fa8dd0ebd1851c3b37025e2c0321fafe8 Files : core/src/main/java/jenkins/model/Jenkins.java daniel-beck : 50cbded7f8636b5dc745c4ee8cae23af3ee110f8 Files : core/src/main/java/jenkins/model/Jenkins.java core/src/test/java/jenkins/model/JenkinsDescriptorTest.java

          Hi Bruce,
          I face the same issue of dead executors with jenkins-1.532.3. Coudl you please provide insights:
          Here is the log:
          Exception in thread "Executor #0 for master" java.lang.ClassCastException: hudson.model.FreeStyleBuild cannot be cast to hudson.model.Action
          at hudson.model.Run.onLoad(Run.java:331)
          at hudson.model.RunMap.retrieve(RunMap.java:223)
          at hudson.model.RunMap.retrieve(RunMap.java:59)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:688)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:671)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:470)
          at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:219)
          at hudson.tasks.Fingerprinter$FingerprintAction.compact(Fingerprinter.java:361)
          at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:350)
          at hudson.model.Run.onLoad(Run.java:333)
          at hudson.model.RunMap.retrieve(RunMap.java:223)
          at hudson.model.RunMap.retrieve(RunMap.java:59)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:688)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:650)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:382)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:547)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:393)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:335)
          at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1089)
          at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:152)
          at hudson.model.Job.isLogUpdated(Job.java:293)
          at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1285)
          at hudson.model.AbstractProject.isBuildBlocked(AbstractProject.java:1222)
          at hudson.model.Queue.isBuildBlocked(Queue.java:1021)
          at hudson.model.Queue.maintain(Queue.java:1080)
          at hudson.model.Queue.pop(Queue.java:935)
          at hudson.model.Executor.grabJob(Executor.java:297)
          at hudson.model.Executor.run(Executor.java:211)

          Maitrey Mishra added a comment - Hi Bruce, I face the same issue of dead executors with jenkins-1.532.3. Coudl you please provide insights: Here is the log: Exception in thread "Executor #0 for master" java.lang.ClassCastException: hudson.model.FreeStyleBuild cannot be cast to hudson.model.Action at hudson.model.Run.onLoad(Run.java:331) at hudson.model.RunMap.retrieve(RunMap.java:223) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:688) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:671) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:470) at hudson.model.AbstractBuild.getPreviousBuild(AbstractBuild.java:219) at hudson.tasks.Fingerprinter$FingerprintAction.compact(Fingerprinter.java:361) at hudson.tasks.Fingerprinter$FingerprintAction.onLoad(Fingerprinter.java:350) at hudson.model.Run.onLoad(Run.java:333) at hudson.model.RunMap.retrieve(RunMap.java:223) at hudson.model.RunMap.retrieve(RunMap.java:59) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:688) at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:650) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:382) at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:547) at jenkins.model.lazy.AbstractLazyLoadRunMap.search(AbstractLazyLoadRunMap.java:393) at jenkins.model.lazy.AbstractLazyLoadRunMap.newestBuild(AbstractLazyLoadRunMap.java:335) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:1089) at hudson.model.AbstractProject.getLastBuild(AbstractProject.java:152) at hudson.model.Job.isLogUpdated(Job.java:293) at hudson.model.AbstractProject.getCauseOfBlockage(AbstractProject.java:1285) at hudson.model.AbstractProject.isBuildBlocked(AbstractProject.java:1222) at hudson.model.Queue.isBuildBlocked(Queue.java:1021) at hudson.model.Queue.maintain(Queue.java:1080) at hudson.model.Queue.pop(Queue.java:935) at hudson.model.Executor.grabJob(Executor.java:297) at hudson.model.Executor.run(Executor.java:211)

          Daniel Beck added a comment -

          maitreymishra23: Please direct any questions not directly related to wrong builds dir settings to #jenkins on Freenode or the jenkinsci-users mailing list. Unrelated discussion has no place in Jira.

          Daniel Beck added a comment - maitreymishra23 : Please direct any questions not directly related to wrong builds dir settings to #jenkins on Freenode or the jenkinsci-users mailing list. Unrelated discussion has no place in Jira.

            danielbeck Daniel Beck
            tortoise74 Bruce Adams
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: