Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28865

java.lang.IllegalStateException: while using using /git/notifyCommit in Freestyle project

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • git-plugin

      I'm trying to use Git's post-receive hook trigger with Jenkins' 'Poll SCM' feature.

      curl http://jenkins_url:9099/jenkins/git/notifyCommit?url=ssh://gitolite@git_server/TestRepo
      

      The problem is that as soon as I push some changes to Git, I notice that the Git Polling Log updates but nothing happens. Just during that time, I see "Dead " link shows up in 'Build Executor Status'.

      Here's the details that I get when I click on the Dead link:

      java.lang.IllegalStateException: cannot create a build with number 31 since that (or higher) is already in use among [38]
      	at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361)
      	at hudson.model.RunMap.put(RunMap.java:192)
      	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
      	at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011)
      	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210)
      	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
      	at hudson.model.Executor$1.call(Executor.java:328)
      	at hudson.model.Executor$1.call(Executor.java:310)
      	at hudson.model.Queue._withLock(Queue.java:1251)
      	at hudson.model.Queue.withLock(Queue.java:1189)
      	at hudson.model.Executor.run(Executor.java:310)
      more info
      
      

      I checked to see whether it was an existing issue and if it was fixed. I came across JENKINS-26739, JENKINS-26616 & JENKINS-26582 of which the latter one looked promising. I, therefore, updated matrix-project-plugin to ver. 1.5 but still I'm facing the same issue. Looks like it's similar but not the same issue, hence this JIRA.

          [JENKINS-28865] java.lang.IllegalStateException: while using using /git/notifyCommit in Freestyle project

          Mark Waite added a comment -

          Did you restart Jenkins after updating the matrix plugin to 1.5?

          Did you attempt the pre-release of the git plugin?

          The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin. Problems detected in the pre-release should be e-mailed to MarkEWaite and ndeloof.

          I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.

          I ran through the steps described in this bug report with a matrix job running across slaves on multiple versions of Windows and multiple versions of Linux (CentOS, Debian, and Ubuntu). As far as I can tell, the bug is fixed by the changes made by jglick.

          Mark Waite added a comment - Did you restart Jenkins after updating the matrix plugin to 1.5? Did you attempt the pre-release of the git plugin? The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin . Problems detected in the pre-release should be e-mailed to MarkEWaite and ndeloof . I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases. I ran through the steps described in this bug report with a matrix job running across slaves on multiple versions of Windows and multiple versions of Linux (CentOS, Debian, and Ubuntu). As far as I can tell, the bug is fixed by the changes made by jglick .

          Garbageyard added a comment -

          Yes, i did restart Jenkins.

          Actually, a lot of users will be impacted if anything breaks so i didn't attempt the pre-release plugin yet. I can give it a try this weekend.

          There is one weird observation i had few mins back. Earlier, when i used to push, only Git Polling Log updated but no build was triggered. Now, builds are getting triggered but only for each alternate pushes. For example, if i push changes 3 times, then build is triggered only for first and third pushes. Any idea?

          Garbageyard added a comment - Yes, i did restart Jenkins. Actually, a lot of users will be impacted if anything breaks so i didn't attempt the pre-release plugin yet. I can give it a try this weekend. There is one weird observation i had few mins back. Earlier, when i used to push, only Git Polling Log updated but no build was triggered. Now, builds are getting triggered but only for each alternate pushes. For example, if i push changes 3 times, then build is triggered only for first and third pushes. Any idea?

          Mark Waite added a comment -

          There appear to be some polling and change detection related bugs in the git plugin 2.3.5 that I'm still trying to characterize. The common cases I've seen were having too many builds generated, not having too few builds generated, but there have been a few reports of changes which were missed.

          You might compare behavior between the 2.3.4 and 2.3.5 versions of the plugin your case.

          Mark Waite added a comment - There appear to be some polling and change detection related bugs in the git plugin 2.3.5 that I'm still trying to characterize. The common cases I've seen were having too many builds generated, not having too few builds generated, but there have been a few reports of changes which were missed. You might compare behavior between the 2.3.4 and 2.3.5 versions of the plugin your case.

          Garbageyard added a comment -

          Tried with 2.3.4 as well. Has the exact same behaviour. Are you aware whether it was working properly in any of the earlier versions?

          Garbageyard added a comment - Tried with 2.3.4 as well. Has the exact same behaviour. Are you aware whether it was working properly in any of the earlier versions?

          Mark Waite added a comment - - edited

          Considering the area where the fix was made was made in the git plugin, I assume it was a long-standing bug and exists in many different versions. I can't explain why the bug only surfaced recently.

          Mark Waite added a comment - - edited Considering the area where the fix was made was made in the git plugin, I assume it was a long-standing bug and exists in many different versions. I can't explain why the bug only surfaced recently.

          Garbageyard added a comment -

          Ok. By the way, does the pre-release build that you asked me to try contains fix for this issue as well or does it addresses Dead issue only?

          Garbageyard added a comment - Ok. By the way, does the pre-release build that you asked me to try contains fix for this issue as well or does it addresses Dead issue only?

          Mark Waite added a comment -

          I'm not sure what you mean by the question ""does it address Dead issue only?" The pre-release contains fixes for several bugs. The list of the exact bugs it is trying to fix is in the document where I listed the tests I'm considering.

          Mark Waite added a comment - I'm not sure what you mean by the question ""does it address Dead issue only?" The pre-release contains fixes for several bugs. The list of the exact bugs it is trying to fix is in the document where I listed the tests I'm considering.

          Garbageyard added a comment -

          What i meant by Dead issue is the "Dead " link that shows up in 'Build Executor Status' (image attached in this JIRA) and the following exception that i get when i click on the Dead link:

          java.lang.IllegalStateException: cannot create a build with number 31 since that (or higher) is already in use among [38]
          	at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361)
          	at hudson.model.RunMap.put(RunMap.java:192)
          	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
          	at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011)
          	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210)
          	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
          	at hudson.model.Executor$1.call(Executor.java:328)
          	at hudson.model.Executor$1.call(Executor.java:310)
          	at hudson.model.Queue._withLock(Queue.java:1251)
          	at hudson.model.Queue.withLock(Queue.java:1189)
          	at hudson.model.Executor.run(Executor.java:310)
          more info
          
          

          My query was whether the pre-release build has fix for the above issue and the polling issue (builds getting triggered only for alternate pushes) as well?

          By the way, I couldn't find the list of bugs in the link that you have shared in your first post. The links contain the HPIs for git-client and git plugin, which i have already downloaded. Can you please provide me the exact link for the list of bugs?

          Garbageyard added a comment - What i meant by Dead issue is the "Dead " link that shows up in 'Build Executor Status' (image attached in this JIRA) and the following exception that i get when i click on the Dead link: java.lang.IllegalStateException: cannot create a build with number 31 since that (or higher) is already in use among [38] at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361) at hudson.model.RunMap.put(RunMap.java:192) at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178) at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144) at hudson.model.Executor$1.call(Executor.java:328) at hudson.model.Executor$1.call(Executor.java:310) at hudson.model.Queue._withLock(Queue.java:1251) at hudson.model.Queue.withLock(Queue.java:1189) at hudson.model.Executor.run(Executor.java:310) more info My query was whether the pre-release build has fix for the above issue and the polling issue (builds getting triggered only for alternate pushes) as well? By the way, I couldn't find the list of bugs in the link that you have shared in your first post. The links contain the HPIs for git-client and git plugin, which i have already downloaded. Can you please provide me the exact link for the list of bugs?

          Mark Waite added a comment -

          The list of bugs attempting to be fixed in that pre-release are in this test ideas list

          Mark Waite added a comment - The list of bugs attempting to be fixed in that pre-release are in this test ideas list

          Garbageyard added a comment -

          Thanks for sharing the link. Looks like JENKINS-27093 is what i'm looking for. However, i was expecting that "java.lang.IllegalStateException: cannot create a build with number" issue was actually going to be part of this pre-release-build too. Anyways, i will test the pre-release build this weekend. I'm open to suggestions for areas of testing.

          Garbageyard added a comment - Thanks for sharing the link. Looks like JENKINS-27093 is what i'm looking for. However, i was expecting that "java.lang.IllegalStateException: cannot create a build with number" issue was actually going to be part of this pre-release-build too. Anyways, i will test the pre-release build this weekend. I'm open to suggestions for areas of testing.

          Mark Waite added a comment -

          I assumed that the illegal state exception is JENKINS-26582, as fixed by jglick. However, it may be that you've found a different case which exercises a different bug with the same exception.

          Mark Waite added a comment - I assumed that the illegal state exception is JENKINS-26582 , as fixed by jglick . However, it may be that you've found a different case which exercises a different bug with the same exception.

          Daniel Beck added a comment -

          As this issue is specifically about Freestyle projects, JENKINS-26582 does not seem to apply.

          Would be interesting what the contents of the nextBuildNumber file in the job directory are when that happens, as well as what happens when you start a build manually.

          Daniel Beck added a comment - As this issue is specifically about Freestyle projects, JENKINS-26582 does not seem to apply. Would be interesting what the contents of the nextBuildNumber file in the job directory are when that happens, as well as what happens when you start a build manually.

          Jesse Glick added a comment -

          Without knowing how to reproduce this would be tricky to diagnose. Somehow Jenkins is trying to reuse build numbers.

          Jesse Glick added a comment - Without knowing how to reproduce this would be tricky to diagnose. Somehow Jenkins is trying to reuse build numbers.

          Garbageyard added a comment - - edited

          Ok. So here is my observation.

          @Daniel/Jesse: First of all, this JIRA was raised mainly for the ISE error which used to show up when i clicked on Dead link in 'Build Executor Status' section. For the last two days, i haven't noticed the Dead link. Not sure what worked since i tried downgrading to 2.3 version as well. My focus was more on seeing whether polling was working fine. To add to it, no one from my team noticed the Dead link either. Frankly, I'm not sure whether it's permanently fixed. Will have to check it for 3-4 days more before i can be sure about it.

          @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository.

          Let me explain it further: Let's assume that the polling right now shows up-to-date commid id and the build for the latest commit has been done. Now i do the following pushes, each after a wait period of few min.

          1st push: SHA - 06a018724fa50c1edd42cf...
          No build triggered and the polling log shows:

          Started on Jun 13, 2015 7:00:35 PM
          Using strategy: Default
          [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2)
           > /usr/bin/git --version # timeout=10
           > /usr/bin/git -c core.askpass=true ls-remote -h ssh://gitolite@git_server/test_repo # timeout=10
          [poll] Latest remote head revision on refs/heads/release-test_2 is: d001b3aed3455817e3de9dc9d74b387c67ce32b2 - already built by 62
          Done. Took 0.23 sec
          No changes
          

          2nd push: SHA - 25fcdc97c076abe70e63...
          Build is triggered and the polling log shows:

          Started on Jun 13, 2015 7:06:04 PM
          Using strategy: Default
          [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2)
           > /usr/bin/git --version # timeout=10
           > /usr/bin/git -c core.askpass=true ls-remote -h ssh://gitolite@git_server/test_repo # timeout=10
          [poll] Latest remote head revision on refs/heads/release-test_2 is: 06a018724fa50c1edd42cfbfce30b1c0e48f40c7
          Done. Took 0.23 sec
          Changes found
          

          Note: Although poll detected commit id of my 1st push, when the build happened, the workspace fetched both my 1st commit and my 2nd commit. This i confirmed from the Changes Summary.

          3rd push: SHA - 0b9abfbbfae0b935...
          No build triggered and the polling log shows:

          Started on Jun 13, 2015 7:12:01 PM
          Using strategy: Default
          [poll] Last Built Revision: Revision 25fcdc97c076abe70e636fb5c851895c48daa455 (origin/release-test_2)
           > /usr/bin/git --version # timeout=10
           > /usr/bin/git -c core.askpass=true ls-remote -h ssh://gitolite@git_server/test_repo # timeout=10
          [poll] Latest remote head revision on refs/heads/release-test_2 is: 25fcdc97c076abe70e636fb5c851895c48daa455 - already built by 63
          Done. Took 0.21 sec
          No changes
          

          As we can see from above, the third polling detected my 2nd commit but since that commit was already fetched in the workspace during the last build, the next build did not trigger.

          I hope this helps in understanding my issue more clearly.

          Garbageyard added a comment - - edited Ok. So here is my observation. @Daniel/Jesse: First of all, this JIRA was raised mainly for the ISE error which used to show up when i clicked on Dead link in 'Build Executor Status' section. For the last two days, i haven't noticed the Dead link. Not sure what worked since i tried downgrading to 2.3 version as well. My focus was more on seeing whether polling was working fine. To add to it, no one from my team noticed the Dead link either. Frankly, I'm not sure whether it's permanently fixed. Will have to check it for 3-4 days more before i can be sure about it. @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository. Let me explain it further: Let's assume that the polling right now shows up-to-date commid id and the build for the latest commit has been done. Now i do the following pushes, each after a wait period of few min. 1st push : SHA - 06a018724fa50c1edd42cf... No build triggered and the polling log shows: Started on Jun 13, 2015 7:00:35 PM Using strategy: Default [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2) > /usr/bin/git --version # timeout=10 > /usr/bin/git -c core.askpass= true ls-remote -h ssh: //gitolite@git_server/test_repo # timeout=10 [poll] Latest remote head revision on refs/heads/release-test_2 is: d001b3aed3455817e3de9dc9d74b387c67ce32b2 - already built by 62 Done. Took 0.23 sec No changes 2nd push : SHA - 25fcdc97c076abe70e63... Build is triggered and the polling log shows: Started on Jun 13, 2015 7:06:04 PM Using strategy: Default [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2) > /usr/bin/git --version # timeout=10 > /usr/bin/git -c core.askpass= true ls-remote -h ssh: //gitolite@git_server/test_repo # timeout=10 [poll] Latest remote head revision on refs/heads/release-test_2 is: 06a018724fa50c1edd42cfbfce30b1c0e48f40c7 Done. Took 0.23 sec Changes found Note : Although poll detected commit id of my 1st push, when the build happened, the workspace fetched both my 1st commit and my 2nd commit . This i confirmed from the Changes Summary . 3rd push : SHA - 0b9abfbbfae0b935... No build triggered and the polling log shows: Started on Jun 13, 2015 7:12:01 PM Using strategy: Default [poll] Last Built Revision: Revision 25fcdc97c076abe70e636fb5c851895c48daa455 (origin/release-test_2) > /usr/bin/git --version # timeout=10 > /usr/bin/git -c core.askpass= true ls-remote -h ssh: //gitolite@git_server/test_repo # timeout=10 [poll] Latest remote head revision on refs/heads/release-test_2 is: 25fcdc97c076abe70e636fb5c851895c48daa455 - already built by 63 Done. Took 0.21 sec No changes As we can see from above, the third polling detected my 2nd commit but since that commit was already fetched in the workspace during the last build, the next build did not trigger . I hope this helps in understanding my issue more clearly.

          Garbageyard added a comment -

          Is this resolved in the newly released-plugin?

          Garbageyard added a comment - Is this resolved in the newly released-plugin?

          Mark Waite added a comment -

          I made no additional changes for this compared to the changes already made by Jesse Glick. You said in an earlier comment:

          @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository.

          As far as I know, there were no changes to the git plugin 2.4.0 or the git client plugin 1.18.0 attempting to address the case you described in that quote.

          Mark Waite added a comment - I made no additional changes for this compared to the changes already made by Jesse Glick. You said in an earlier comment: @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository. As far as I know, there were no changes to the git plugin 2.4.0 or the git client plugin 1.18.0 attempting to address the case you described in that quote.

          Garbageyard added a comment -

          Ok. Thanks for the update Mark!

          Garbageyard added a comment - Ok. Thanks for the update Mark!

          Mark Waite added a comment -

          Closing this as resolved since the issue seems to be resolved as described in this bug report and the separate issue described by Gaurav Chhabra is already tracked as JENKINS-12545

          Mark Waite added a comment - Closing this as resolved since the issue seems to be resolved as described in this bug report and the separate issue described by Gaurav Chhabra is already tracked as JENKINS-12545

            ndeloof Nicolas De Loof
            gc Garbageyard
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: