Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22681

Last build of project reloaded when project asked for later build

      A user encountered a problem whereby a MavenModuleSetBuild would, soon after starting, be shown as completed (and failed) even though really it was still running (so for example reloading the console link sufficed to show new output, but it would not automatically refresh).

      It turned out that reload had been called on the build while it was still running, triggered by the Delivery Pipeline plugin attempting (in BuildUtil) to load a build by a number that was bigger than the actual last build number of the project. (Ultimately this was due to an UpstreamCause in a historical build of another project referring to an unrelated build of a different project that used to have the same name as the project showing the issue. Thus the upstream build link was dead.) Apparently Delivery Pipeline makes frequent and repeated calls to /view/*/api/json when a browser window is displaying the view, even if it is idle , so this would happen quite often.

      When getBuildByNumber was called with the bogus build number, this delegated to search with direction EXACT, which triggered the fix of JENKINS-19418. Unfortunately that called load on the currently highest build number—that of the running build—creating a new build object (with a state of COMPLETED and result of FAILURE, as if being loaded during a Jenkins restart after a crash), and stashing that corrupt new build object into the index. (search then returned null, correctly, since the build number did not match what was requested.) So while the original build object was fine, and ultimately wrote out the correct build.xml record, in the meantime web requests were serving a clone which claimed to have already failed. Since the console log showed ongoing progress and then success, this was a source of much confusion. (After the build really finished, the next spurious reload created a copy with the correct state/result fields, adding even more confusion since it would look like to another observer like the bug had never happened.)

      I tried to reproduce this in a clean 1.532.2 installation by creating a freestyle project; building #1; adding a shell build step to sleep; starting a build of #2, watching its console log; and then in another browser tab trying to load /job/p/3, expecting a 404. This did not cause #2 to suddenly stop updating; I am guessing that an additional required trigger condition is the presence of a plugin which causes build.xml to be written before the build finishes. (Jenkins core only writes it when the build is finalized, but plugins may call Run.save earlier, for various reasons.) If there is no build.xml, then load would return null and not update the index.

      At any rate, I did manage to reproduce the basic flaw in a unit test quite easily. After calling search with EXACT and a number larger than any existing build, subsequent calls with the number of the last build returned a different copy. Simply switching to getById fixes the problem according to this test, without regressing JENKINS-19418, at least according to its test. And with this fix, indeed reload is no longer called on existing builds (at least running ones), and the bug symptoms disappeared.

          [JENKINS-22681] Last build of project reloaded when project asked for later build

          Jesse Glick added a comment -

          Marking as a performance issue since in cases where gratuitous load is called often, this can add significant load to the system.

          Jesse Glick added a comment - Marking as a performance issue since in cases where gratuitous load is called often, this can add significant load to the system.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/Run.java
          http://jenkins-ci.org/commit/jenkins/b55c157813c0f5dc3cb06d0503834d21115aebc9
          Log:
          JENKINS-22681 Additional logging useful for diagnosing certain lazy-loading bugs.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/Run.java http://jenkins-ci.org/commit/jenkins/b55c157813c0f5dc3cb06d0503834d21115aebc9 Log: JENKINS-22681 Additional logging useful for diagnosing certain lazy-loading bugs.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          changelog.html
          core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
          core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
          core/src/test/java/jenkins/model/lazy/FakeMap.java
          http://jenkins-ci.org/commit/jenkins/b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99
          Log:
          [FIXED JENKINS-22681] Fix of JENKINS-19418 should have used getById, not load, to avoid reloading the last build gratuitously.

          Compare: https://github.com/jenkinsci/jenkins/compare/e12e40adc6c2...b6b7bfd27c9b

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: changelog.html core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java core/src/test/java/jenkins/model/lazy/FakeMap.java http://jenkins-ci.org/commit/jenkins/b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99 Log: [FIXED JENKINS-22681] Fix of JENKINS-19418 should have used getById, not load, to avoid reloading the last build gratuitously. Compare: https://github.com/jenkinsci/jenkins/compare/e12e40adc6c2...b6b7bfd27c9b

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3322
          JENKINS-22681 Additional logging useful for diagnosing certain lazy-loading bugs. (Revision b55c157813c0f5dc3cb06d0503834d21115aebc9)
          [FIXED JENKINS-22681] Fix of JENKINS-19418 should have used getById, not load, to avoid reloading the last build gratuitously. (Revision b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99)

          Result = SUCCESS
          Jesse Glick : b55c157813c0f5dc3cb06d0503834d21115aebc9
          Files :

          • core/src/main/java/hudson/model/Run.java

          Jesse Glick : b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99
          Files :

          • core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
          • core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
          • core/src/test/java/jenkins/model/lazy/FakeMap.java
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #3322 JENKINS-22681 Additional logging useful for diagnosing certain lazy-loading bugs. (Revision b55c157813c0f5dc3cb06d0503834d21115aebc9) [FIXED JENKINS-22681] Fix of JENKINS-19418 should have used getById, not load, to avoid reloading the last build gratuitously. (Revision b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99) Result = SUCCESS Jesse Glick : b55c157813c0f5dc3cb06d0503834d21115aebc9 Files : core/src/main/java/hudson/model/Run.java Jesse Glick : b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99 Files : core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java core/src/test/java/jenkins/model/lazy/FakeMap.java changelog.html

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/Run.java
          http://jenkins-ci.org/commit/jenkins/142825718be9787547973eac0fb801414c6cf39e
          Log:
          JENKINS-22681 Toning down some logging to FINER.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/Run.java http://jenkins-ci.org/commit/jenkins/142825718be9787547973eac0fb801414c6cf39e Log: JENKINS-22681 Toning down some logging to FINER.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #3347
          JENKINS-22681 Toning down some logging to FINER. (Revision 142825718be9787547973eac0fb801414c6cf39e)

          Result = SUCCESS
          Jesse Glick : 142825718be9787547973eac0fb801414c6cf39e
          Files :

          • core/src/main/java/hudson/model/Run.java

          dogfood added a comment - Integrated in jenkins_main_trunk #3347 JENKINS-22681 Toning down some logging to FINER. (Revision 142825718be9787547973eac0fb801414c6cf39e) Result = SUCCESS Jesse Glick : 142825718be9787547973eac0fb801414c6cf39e Files : core/src/main/java/hudson/model/Run.java

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/Run.java
          http://jenkins-ci.org/commit/jenkins/4c4ce9c9dd47148201bd2e573bc30dc3f7e03080
          Log:
          JENKINS-22681 Additional logging useful for diagnosing certain lazy-loading bugs.
          (cherry picked from commit b55c157813c0f5dc3cb06d0503834d21115aebc9)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/Run.java http://jenkins-ci.org/commit/jenkins/4c4ce9c9dd47148201bd2e573bc30dc3f7e03080 Log: JENKINS-22681 Additional logging useful for diagnosing certain lazy-loading bugs. (cherry picked from commit b55c157813c0f5dc3cb06d0503834d21115aebc9)

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java
          core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java
          core/src/test/java/jenkins/model/lazy/FakeMap.java
          http://jenkins-ci.org/commit/jenkins/701045e6a27bb93ba739b111505d01c238ad455e
          Log:
          [FIXED JENKINS-22681] Fix of JENKINS-19418 should have used getById, not load, to avoid reloading the last build gratuitously.
          (cherry picked from commit b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99)

          Conflicts:
          changelog.html

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/jenkins/model/lazy/AbstractLazyLoadRunMap.java core/src/test/java/jenkins/model/lazy/AbstractLazyLoadRunMapTest.java core/src/test/java/jenkins/model/lazy/FakeMap.java http://jenkins-ci.org/commit/jenkins/701045e6a27bb93ba739b111505d01c238ad455e Log: [FIXED JENKINS-22681] Fix of JENKINS-19418 should have used getById, not load, to avoid reloading the last build gratuitously. (cherry picked from commit b6b7bfd27c9b5551fde04416ee9b7b5faf16ad99) Conflicts: changelog.html

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/model/Run.java
          http://jenkins-ci.org/commit/jenkins/347a9a475bd54bdecf4c8ec15876d82c460d92bc
          Log:
          JENKINS-22681 Toning down some logging to FINER.
          (cherry picked from commit 142825718be9787547973eac0fb801414c6cf39e)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/model/Run.java http://jenkins-ci.org/commit/jenkins/347a9a475bd54bdecf4c8ec15876d82c460d92bc Log: JENKINS-22681 Toning down some logging to FINER. (cherry picked from commit 142825718be9787547973eac0fb801414c6cf39e)

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: