• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • gerrit-trigger-plugin
    • None
    • Jenkins 1.554.1
      GerritTrigger 2.11.1

      When viewing jobs, newer builds disappear from the build history list. Trying to go to the URL of a newer build results in a 404. The missing builds are located on disk and restarting Jenkins will make them reappear.

      Sometimes, the nextBuildNumber is set to a lower number and duplicate builds are created; two date-directories are created in builds/ but only the latest build has a build_number symlink.

      I wrote tool to help detect these problems on disk and even repair them (for some value of repair): https://github.com/docwhat/jenkins-job-checker

      A lot of history is on JENKINS-15156; I'll repeat some of it below.

          [JENKINS-23152] builds getting lost due to GerritTrigger

          Code changed in jenkins
          User: Scott Hebert
          Path:
          src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/ssh/BuildCompletedCommandJob.java
          src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/ssh/BuildStartedCommandJob.java
          src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/LockedDownGerritEventTest.java
          http://jenkins-ci.org/commit/gerrit-trigger-plugin/a03c0d5da9cc4d275e4984ca1c8e035f1d6056d2
          Log:
          Gerrit notifier threads should impersonate SYSTEM

          The BuildCompletedCommandJob was not impersonating SYSTEM and was
          therefore throwing permission-related exceptions.

          For completeness, BuildStartedCommandJob was updated too.

          Related to JENKINS-23152

          Change-Id: I3eb6036d8186478a615c15036b4b7f98035dd54c

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Scott Hebert Path: src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/ssh/BuildCompletedCommandJob.java src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/job/ssh/BuildStartedCommandJob.java src/test/java/com/sonyericsson/hudson/plugins/gerrit/trigger/LockedDownGerritEventTest.java http://jenkins-ci.org/commit/gerrit-trigger-plugin/a03c0d5da9cc4d275e4984ca1c8e035f1d6056d2 Log: Gerrit notifier threads should impersonate SYSTEM The BuildCompletedCommandJob was not impersonating SYSTEM and was therefore throwing permission-related exceptions. For completeness, BuildStartedCommandJob was updated too. Related to JENKINS-23152 Change-Id: I3eb6036d8186478a615c15036b4b7f98035dd54c

          Code changed in jenkins
          User: Scott Hebert
          Path:
          src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java
          src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/EventListener.java
          http://jenkins-ci.org/commit/gerrit-trigger-plugin/5604387839ef68c42f22517a262f639fd4f1f111
          Log:
          Correct infinite loops caused by CommentAdded

          CommentAdded events were not be processed with the correct
          gerritEvent() signature. This caused builds be triggered needlessly
          and sometimes caused infinite loops between Jenkins and Gerrit.

          This fix exposes the EventListener as a public class so that the Gerrit Events
          Gerrit Handler can correctly invoke the appropriate methods.

          This also corrects a NPE during a BuildStarted notification.

          JENKINS-23152

          Change-Id: Ief95fc05d5cf0a5d2e0dc95330e74bb6d52f425c

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Scott Hebert Path: src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/gerritnotifier/model/BuildMemory.java src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/EventListener.java http://jenkins-ci.org/commit/gerrit-trigger-plugin/5604387839ef68c42f22517a262f639fd4f1f111 Log: Correct infinite loops caused by CommentAdded CommentAdded events were not be processed with the correct gerritEvent() signature. This caused builds be triggered needlessly and sometimes caused infinite loops between Jenkins and Gerrit. This fix exposes the EventListener as a public class so that the Gerrit Events Gerrit Handler can correctly invoke the appropriate methods. This also corrects a NPE during a BuildStarted notification. JENKINS-23152 Change-Id: Ief95fc05d5cf0a5d2e0dc95330e74bb6d52f425c

          Katherine Elliott added a comment - - edited

          I can't tell what action has been taken on this defect recently. I installed http://repo.jenkins-ci.org/releases/com/sonyericsson/hudson/plugins/gerrit/gerrit-trigger/2.13.0-beta-2/gerrit-trigger-2.13.0-beta-2.hpi on my Jenkins server and, although it did seem to fix the "missing builds" issue, now I'm only getting one of multiple jobs reporting results back to Gerrit! When I submit a patch set to our Gerrit server, I only get one email but that email shows just one job that was triggered but indicates that the job triggered is 1 of 2. When I go to the review in Gerrit, it only shows one of the two jobs being triggered. This is what I see in Gerrit:

          jenkins
          12:32 PM

          Patch Set 1:
          Build Started http://jenkins-instance:8080/job/test-2/37/ (2/2)

          I should be seeing:

          jenkins
          12:32 PM

          Patch Set 1:
          Build Started http://jenkins-instance:8080/job/test-1/37/ (1/2)
          Build Started http://jenkins-instance:8080/job/test-2/37/ (2/2)

          I see this with 2.13 beta 2 through 5. I tried 2.13 beta 6 and it fixed this but then I ran into the missing builds defect again.

          Has anyone else seen this? I could use some suggestions. This is preventing us from putting our upgraded Jenkins instances into our production environment.

          Katherine Elliott added a comment - - edited I can't tell what action has been taken on this defect recently. I installed http://repo.jenkins-ci.org/releases/com/sonyericsson/hudson/plugins/gerrit/gerrit-trigger/2.13.0-beta-2/gerrit-trigger-2.13.0-beta-2.hpi on my Jenkins server and, although it did seem to fix the "missing builds" issue, now I'm only getting one of multiple jobs reporting results back to Gerrit! When I submit a patch set to our Gerrit server, I only get one email but that email shows just one job that was triggered but indicates that the job triggered is 1 of 2. When I go to the review in Gerrit, it only shows one of the two jobs being triggered. This is what I see in Gerrit: jenkins 12:32 PM ↩ Patch Set 1: Build Started http://jenkins-instance:8080/job/test-2/37/ (2/2) I should be seeing: jenkins 12:32 PM ↩ Patch Set 1: Build Started http://jenkins-instance:8080/job/test-1/37/ (1/2) Build Started http://jenkins-instance:8080/job/test-2/37/ (2/2) I see this with 2.13 beta 2 through 5. I tried 2.13 beta 6 and it fixed this but then I ran into the missing builds defect again. Has anyone else seen this? I could use some suggestions. This is preventing us from putting our upgraded Jenkins instances into our production environment.

          rsandell added a comment -

          @Katherine Can you provide some more information? e.g. what does your configuration look like, do you see any warnings or severe logs in the server log etc.

          rsandell added a comment - @Katherine Can you provide some more information? e.g. what does your configuration look like, do you see any warnings or severe logs in the server log etc.

          Can Oztokmak added a comment -

          after activating ntp on all slaves and master nodes, the problem seems to be resolved. maybe someone else can confirm if this workaround resolves the issue.

          Can Oztokmak added a comment - after activating ntp on all slaves and master nodes, the problem seems to be resolved. maybe someone else can confirm if this workaround resolves the issue.

          Baptiste Mathus added a comment - - edited

          FWIW, same or similar issue seen here too on Jenkins 1.593 (+Jenkins Gerrit plugin 2.12.0 and a 2.8.3 Gerrit server). It seems to have happened since we used reload from disk (which we do only few times a year) and only affects Gerrit builds.
          Even when triggered manually with the "query & trigger Gerrit Patches", we see the new build displayed on the left of the screen. But it does not appear in the job page and leads to a 404 when accessed. And the build is indeed on the disk and seems to run just fine. So more something like a GUI issue.

          Will post more informations here if we manage to reproduce the issue non randomly.

          Baptiste Mathus added a comment - - edited FWIW, same or similar issue seen here too on Jenkins 1.593 (+Jenkins Gerrit plugin 2.12.0 and a 2.8.3 Gerrit server). It seems to have happened since we used reload from disk (which we do only few times a year) and only affects Gerrit builds. Even when triggered manually with the "query & trigger Gerrit Patches", we see the new build displayed on the left of the screen. But it does not appear in the job page and leads to a 404 when accessed. And the build is indeed on the disk and seems to run just fine. So more something like a GUI issue. Will post more informations here if we manage to reproduce the issue non randomly.

          rsandell added a comment -

          And what happens if you upgrade to >= 2.13 where efforts have been made to fix this?

          rsandell added a comment - And what happens if you upgrade to >= 2.13 where efforts have been made to fix this?

          rsandell We plan to upgrade our instance quite soon, we'll see it this comes back. Thanks for the feedback.

          Baptiste Mathus added a comment - rsandell We plan to upgrade our instance quite soon, we'll see it this comes back. Thanks for the feedback.

          Can Oztokmak added a comment -

          upgrading gerrit plugin to 2.14.0 resolved our issue.

          Can Oztokmak added a comment - upgrading gerrit plugin to 2.14.0 resolved our issue.

          rsandell added a comment -

          Version 2.13.0

          rsandell added a comment - Version 2.13.0

            rsandell rsandell
            docwhat Christian Höltje
            Votes:
            7 Vote for this issue
            Watchers:
            23 Start watching this issue

              Created:
              Updated:
              Resolved: