Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-28865

java.lang.IllegalStateException: while using using /git/notifyCommit in Freestyle project

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Fixed
    • Component/s: git-plugin
    • Environment:
    • Similar Issues:

      Description

      I'm trying to use Git's post-receive hook trigger with Jenkins' 'Poll SCM' feature.

      curl http://jenkins_url:9099/jenkins/git/notifyCommit?url=ssh://gitolite@git_server/TestRepo
      

      The problem is that as soon as I push some changes to Git, I notice that the Git Polling Log updates but nothing happens. Just during that time, I see "Dead " link shows up in 'Build Executor Status'.

      Here's the details that I get when I click on the Dead link:

      java.lang.IllegalStateException: cannot create a build with number 31 since that (or higher) is already in use among [38]
      	at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361)
      	at hudson.model.RunMap.put(RunMap.java:192)
      	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
      	at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011)
      	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210)
      	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
      	at hudson.model.Executor$1.call(Executor.java:328)
      	at hudson.model.Executor$1.call(Executor.java:310)
      	at hudson.model.Queue._withLock(Queue.java:1251)
      	at hudson.model.Queue.withLock(Queue.java:1189)
      	at hudson.model.Executor.run(Executor.java:310)
      more info
      
      

      I checked to see whether it was an existing issue and if it was fixed. I came across JENKINS-26739, JENKINS-26616 & JENKINS-26582 of which the latter one looked promising. I, therefore, updated matrix-project-plugin to ver. 1.5 but still I'm facing the same issue. Looks like it's similar but not the same issue, hence this JIRA.

        Attachments

          Issue Links

            Activity

            Hide
            markewaite Mark Waite added a comment -

            Did you restart Jenkins after updating the matrix plugin to 1.5?

            Did you attempt the pre-release of the git plugin?

            The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin. Problems detected in the pre-release should be e-mailed to Mark Waite and Nicolas De Loof.

            I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases.

            I ran through the steps described in this bug report with a matrix job running across slaves on multiple versions of Windows and multiple versions of Linux (CentOS, Debian, and Ubuntu). As far as I can tell, the bug is fixed by the changes made by Jesse Glick.

            Show
            markewaite Mark Waite added a comment - Did you restart Jenkins after updating the matrix plugin to 1.5? Did you attempt the pre-release of the git plugin? The git plugin and git client plugin are being tested in hopes of releasing new versions before the end of June. If you're willing to assist with the testing, please download and install a pre-release build of the git client plugin and the git plugin . Problems detected in the pre-release should be e-mailed to Mark Waite and Nicolas De Loof . I wrote some test ideas if you would like suggestions of areas that need testing. The git plugin supports many different use cases and its automated tests only evaluate a very few of those use cases. I ran through the steps described in this bug report with a matrix job running across slaves on multiple versions of Windows and multiple versions of Linux (CentOS, Debian, and Ubuntu). As far as I can tell, the bug is fixed by the changes made by Jesse Glick .
            Hide
            gc Garbageyard added a comment -

            Yes, i did restart Jenkins.

            Actually, a lot of users will be impacted if anything breaks so i didn't attempt the pre-release plugin yet. I can give it a try this weekend.

            There is one weird observation i had few mins back. Earlier, when i used to push, only Git Polling Log updated but no build was triggered. Now, builds are getting triggered but only for each alternate pushes. For example, if i push changes 3 times, then build is triggered only for first and third pushes. Any idea?

            Show
            gc Garbageyard added a comment - Yes, i did restart Jenkins. Actually, a lot of users will be impacted if anything breaks so i didn't attempt the pre-release plugin yet. I can give it a try this weekend. There is one weird observation i had few mins back. Earlier, when i used to push, only Git Polling Log updated but no build was triggered. Now, builds are getting triggered but only for each alternate pushes. For example, if i push changes 3 times, then build is triggered only for first and third pushes. Any idea?
            Hide
            markewaite Mark Waite added a comment -

            There appear to be some polling and change detection related bugs in the git plugin 2.3.5 that I'm still trying to characterize. The common cases I've seen were having too many builds generated, not having too few builds generated, but there have been a few reports of changes which were missed.

            You might compare behavior between the 2.3.4 and 2.3.5 versions of the plugin your case.

            Show
            markewaite Mark Waite added a comment - There appear to be some polling and change detection related bugs in the git plugin 2.3.5 that I'm still trying to characterize. The common cases I've seen were having too many builds generated, not having too few builds generated, but there have been a few reports of changes which were missed. You might compare behavior between the 2.3.4 and 2.3.5 versions of the plugin your case.
            Hide
            gc Garbageyard added a comment -

            Tried with 2.3.4 as well. Has the exact same behaviour. Are you aware whether it was working properly in any of the earlier versions?

            Show
            gc Garbageyard added a comment - Tried with 2.3.4 as well. Has the exact same behaviour. Are you aware whether it was working properly in any of the earlier versions?
            Hide
            markewaite Mark Waite added a comment - - edited

            Considering the area where the fix was made was made in the git plugin, I assume it was a long-standing bug and exists in many different versions. I can't explain why the bug only surfaced recently.

            Show
            markewaite Mark Waite added a comment - - edited Considering the area where the fix was made was made in the git plugin, I assume it was a long-standing bug and exists in many different versions. I can't explain why the bug only surfaced recently.
            Hide
            gc Garbageyard added a comment -

            Ok. By the way, does the pre-release build that you asked me to try contains fix for this issue as well or does it addresses Dead issue only?

            Show
            gc Garbageyard added a comment - Ok. By the way, does the pre-release build that you asked me to try contains fix for this issue as well or does it addresses Dead issue only?
            Hide
            markewaite Mark Waite added a comment -

            I'm not sure what you mean by the question ""does it address Dead issue only?" The pre-release contains fixes for several bugs. The list of the exact bugs it is trying to fix is in the document where I listed the tests I'm considering.

            Show
            markewaite Mark Waite added a comment - I'm not sure what you mean by the question ""does it address Dead issue only?" The pre-release contains fixes for several bugs. The list of the exact bugs it is trying to fix is in the document where I listed the tests I'm considering.
            Hide
            gc Garbageyard added a comment -

            What i meant by Dead issue is the "Dead " link that shows up in 'Build Executor Status' (image attached in this JIRA) and the following exception that i get when i click on the Dead link:

            java.lang.IllegalStateException: cannot create a build with number 31 since that (or higher) is already in use among [38]
            	at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361)
            	at hudson.model.RunMap.put(RunMap.java:192)
            	at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178)
            	at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011)
            	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210)
            	at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144)
            	at hudson.model.Executor$1.call(Executor.java:328)
            	at hudson.model.Executor$1.call(Executor.java:310)
            	at hudson.model.Queue._withLock(Queue.java:1251)
            	at hudson.model.Queue.withLock(Queue.java:1189)
            	at hudson.model.Executor.run(Executor.java:310)
            more info
            
            

            My query was whether the pre-release build has fix for the above issue and the polling issue (builds getting triggered only for alternate pushes) as well?

            By the way, I couldn't find the list of bugs in the link that you have shared in your first post. The links contain the HPIs for git-client and git plugin, which i have already downloaded. Can you please provide me the exact link for the list of bugs?

            Show
            gc Garbageyard added a comment - What i meant by Dead issue is the "Dead " link that shows up in 'Build Executor Status' (image attached in this JIRA) and the following exception that i get when i click on the Dead link: java.lang.IllegalStateException: cannot create a build with number 31 since that (or higher) is already in use among [38] at jenkins.model.lazy.AbstractLazyLoadRunMap.proposeNewNumber(AbstractLazyLoadRunMap.java:361) at hudson.model.RunMap.put(RunMap.java:192) at jenkins.model.lazy.LazyBuildMixIn.newBuild(LazyBuildMixIn.java:178) at hudson.model.AbstractProject.newBuild(AbstractProject.java:1011) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1210) at hudson.model.AbstractProject.createExecutable(AbstractProject.java:144) at hudson.model.Executor$1.call(Executor.java:328) at hudson.model.Executor$1.call(Executor.java:310) at hudson.model.Queue._withLock(Queue.java:1251) at hudson.model.Queue.withLock(Queue.java:1189) at hudson.model.Executor.run(Executor.java:310) more info My query was whether the pre-release build has fix for the above issue and the polling issue (builds getting triggered only for alternate pushes) as well? By the way, I couldn't find the list of bugs in the link that you have shared in your first post. The links contain the HPIs for git-client and git plugin, which i have already downloaded. Can you please provide me the exact link for the list of bugs?
            Hide
            markewaite Mark Waite added a comment -

            The list of bugs attempting to be fixed in that pre-release are in this test ideas list

            Show
            markewaite Mark Waite added a comment - The list of bugs attempting to be fixed in that pre-release are in this test ideas list
            Hide
            gc Garbageyard added a comment -

            Thanks for sharing the link. Looks like JENKINS-27093 is what i'm looking for. However, i was expecting that "java.lang.IllegalStateException: cannot create a build with number" issue was actually going to be part of this pre-release-build too. Anyways, i will test the pre-release build this weekend. I'm open to suggestions for areas of testing.

            Show
            gc Garbageyard added a comment - Thanks for sharing the link. Looks like JENKINS-27093 is what i'm looking for. However, i was expecting that "java.lang.IllegalStateException: cannot create a build with number" issue was actually going to be part of this pre-release-build too. Anyways, i will test the pre-release build this weekend. I'm open to suggestions for areas of testing.
            Hide
            markewaite Mark Waite added a comment -

            I assumed that the illegal state exception is JENKINS-26582, as fixed by Jesse Glick. However, it may be that you've found a different case which exercises a different bug with the same exception.

            Show
            markewaite Mark Waite added a comment - I assumed that the illegal state exception is JENKINS-26582 , as fixed by Jesse Glick . However, it may be that you've found a different case which exercises a different bug with the same exception.
            Hide
            danielbeck Daniel Beck added a comment -

            As this issue is specifically about Freestyle projects, JENKINS-26582 does not seem to apply.

            Would be interesting what the contents of the nextBuildNumber file in the job directory are when that happens, as well as what happens when you start a build manually.

            Show
            danielbeck Daniel Beck added a comment - As this issue is specifically about Freestyle projects, JENKINS-26582 does not seem to apply. Would be interesting what the contents of the nextBuildNumber file in the job directory are when that happens, as well as what happens when you start a build manually.
            Hide
            jglick Jesse Glick added a comment -

            Without knowing how to reproduce this would be tricky to diagnose. Somehow Jenkins is trying to reuse build numbers.

            Show
            jglick Jesse Glick added a comment - Without knowing how to reproduce this would be tricky to diagnose. Somehow Jenkins is trying to reuse build numbers.
            Hide
            gc Garbageyard added a comment - - edited

            Ok. So here is my observation.

            @Daniel/Jesse: First of all, this JIRA was raised mainly for the ISE error which used to show up when i clicked on Dead link in 'Build Executor Status' section. For the last two days, i haven't noticed the Dead link. Not sure what worked since i tried downgrading to 2.3 version as well. My focus was more on seeing whether polling was working fine. To add to it, no one from my team noticed the Dead link either. Frankly, I'm not sure whether it's permanently fixed. Will have to check it for 3-4 days more before i can be sure about it.

            @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository.

            Let me explain it further: Let's assume that the polling right now shows up-to-date commid id and the build for the latest commit has been done. Now i do the following pushes, each after a wait period of few min.

            1st push: SHA - 06a018724fa50c1edd42cf...
            No build triggered and the polling log shows:

            Started on Jun 13, 2015 7:00:35 PM
            Using strategy: Default
            [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2)
             > /usr/bin/git --version # timeout=10
             > /usr/bin/git -c core.askpass=true ls-remote -h ssh://gitolite@git_server/test_repo # timeout=10
            [poll] Latest remote head revision on refs/heads/release-test_2 is: d001b3aed3455817e3de9dc9d74b387c67ce32b2 - already built by 62
            Done. Took 0.23 sec
            No changes
            

            2nd push: SHA - 25fcdc97c076abe70e63...
            Build is triggered and the polling log shows:

            Started on Jun 13, 2015 7:06:04 PM
            Using strategy: Default
            [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2)
             > /usr/bin/git --version # timeout=10
             > /usr/bin/git -c core.askpass=true ls-remote -h ssh://gitolite@git_server/test_repo # timeout=10
            [poll] Latest remote head revision on refs/heads/release-test_2 is: 06a018724fa50c1edd42cfbfce30b1c0e48f40c7
            Done. Took 0.23 sec
            Changes found
            

            Note: Although poll detected commit id of my 1st push, when the build happened, the workspace fetched both my 1st commit and my 2nd commit. This i confirmed from the Changes Summary.

            3rd push: SHA - 0b9abfbbfae0b935...
            No build triggered and the polling log shows:

            Started on Jun 13, 2015 7:12:01 PM
            Using strategy: Default
            [poll] Last Built Revision: Revision 25fcdc97c076abe70e636fb5c851895c48daa455 (origin/release-test_2)
             > /usr/bin/git --version # timeout=10
             > /usr/bin/git -c core.askpass=true ls-remote -h ssh://gitolite@git_server/test_repo # timeout=10
            [poll] Latest remote head revision on refs/heads/release-test_2 is: 25fcdc97c076abe70e636fb5c851895c48daa455 - already built by 63
            Done. Took 0.21 sec
            No changes
            

            As we can see from above, the third polling detected my 2nd commit but since that commit was already fetched in the workspace during the last build, the next build did not trigger.

            I hope this helps in understanding my issue more clearly.

            Show
            gc Garbageyard added a comment - - edited Ok. So here is my observation. @Daniel/Jesse: First of all, this JIRA was raised mainly for the ISE error which used to show up when i clicked on Dead link in 'Build Executor Status' section. For the last two days, i haven't noticed the Dead link. Not sure what worked since i tried downgrading to 2.3 version as well. My focus was more on seeing whether polling was working fine. To add to it, no one from my team noticed the Dead link either. Frankly, I'm not sure whether it's permanently fixed. Will have to check it for 3-4 days more before i can be sure about it. @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository. Let me explain it further: Let's assume that the polling right now shows up-to-date commid id and the build for the latest commit has been done. Now i do the following pushes, each after a wait period of few min. 1st push : SHA - 06a018724fa50c1edd42cf... No build triggered and the polling log shows: Started on Jun 13, 2015 7:00:35 PM Using strategy: Default [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2) > /usr/bin/git --version # timeout=10 > /usr/bin/git -c core.askpass= true ls-remote -h ssh: //gitolite@git_server/test_repo # timeout=10 [poll] Latest remote head revision on refs/heads/release-test_2 is: d001b3aed3455817e3de9dc9d74b387c67ce32b2 - already built by 62 Done. Took 0.23 sec No changes 2nd push : SHA - 25fcdc97c076abe70e63... Build is triggered and the polling log shows: Started on Jun 13, 2015 7:06:04 PM Using strategy: Default [poll] Last Built Revision: Revision d001b3aed3455817e3de9dc9d74b387c67ce32b2 (origin/release-test_2) > /usr/bin/git --version # timeout=10 > /usr/bin/git -c core.askpass= true ls-remote -h ssh: //gitolite@git_server/test_repo # timeout=10 [poll] Latest remote head revision on refs/heads/release-test_2 is: 06a018724fa50c1edd42cfbfce30b1c0e48f40c7 Done. Took 0.23 sec Changes found Note : Although poll detected commit id of my 1st push, when the build happened, the workspace fetched both my 1st commit and my 2nd commit . This i confirmed from the Changes Summary . 3rd push : SHA - 0b9abfbbfae0b935... No build triggered and the polling log shows: Started on Jun 13, 2015 7:12:01 PM Using strategy: Default [poll] Last Built Revision: Revision 25fcdc97c076abe70e636fb5c851895c48daa455 (origin/release-test_2) > /usr/bin/git --version # timeout=10 > /usr/bin/git -c core.askpass= true ls-remote -h ssh: //gitolite@git_server/test_repo # timeout=10 [poll] Latest remote head revision on refs/heads/release-test_2 is: 25fcdc97c076abe70e636fb5c851895c48daa455 - already built by 63 Done. Took 0.21 sec No changes As we can see from above, the third polling detected my 2nd commit but since that commit was already fetched in the workspace during the last build, the next build did not trigger . I hope this helps in understanding my issue more clearly.
            Hide
            gc Garbageyard added a comment -

            Is this resolved in the newly released-plugin?

            Show
            gc Garbageyard added a comment - Is this resolved in the newly released-plugin?
            Hide
            markewaite Mark Waite added a comment -

            I made no additional changes for this compared to the changes already made by Jesse Glick. You said in an earlier comment:

            @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository.

            As far as I know, there were no changes to the git plugin 2.4.0 or the git client plugin 1.18.0 attempting to address the case you described in that quote.

            Show
            markewaite Mark Waite added a comment - I made no additional changes for this compared to the changes already made by Jesse Glick. You said in an earlier comment: @All: Coming to polling issue, i downloaded both git and git-client pre-release build plugin and tried triggering build using the post-receive hook using /git/notifyCommit feature. It seems the issue is till there. If i do two pushes, the build triggers only when the second push is done on the given branch for a repository. As far as I know, there were no changes to the git plugin 2.4.0 or the git client plugin 1.18.0 attempting to address the case you described in that quote.
            Hide
            gc Garbageyard added a comment -

            Ok. Thanks for the update Mark!

            Show
            gc Garbageyard added a comment - Ok. Thanks for the update Mark!
            Hide
            markewaite Mark Waite added a comment -

            Closing this as resolved since the issue seems to be resolved as described in this bug report and the separate issue described by Gaurav Chhabra is already tracked as JENKINS-12545

            Show
            markewaite Mark Waite added a comment - Closing this as resolved since the issue seems to be resolved as described in this bug report and the separate issue described by Gaurav Chhabra is already tracked as JENKINS-12545

              People

              Assignee:
              ndeloof Nicolas De Loof
              Reporter:
              gc Garbageyard
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: