Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4270

SVN polling, clean builds, and long checkout equal endless loop

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • subversion-plugin
    • None
    • Platform: All, OS: All

      I was using svn polling once every minute (* * * * *) and the clean build option. My
      checkout time exceeds one minute.

      So what happens is...
      1) A build is triggered, this deletes the workspace (clean).
      2) The svn checkout begins (more than 60s elapse).
      3) Hudson svn polling tries to query the workspace to see if there are any updates, but the
      workspace is not done checking out from step 2. This causes the following message:

      Started on Aug 19, 2009 12:00:54 PM
      Workspace doesn't contain http://xxxxxxxxxxxxxxxxxxxxxxxxxx. Need a new build
      Done. Took 0.2 sec
      Changes found

      4) A new build is scheduled even though there are no new changes. Goto to step 1).

      This creates an endlessly looping build.

      A quick fix would be to suspend polling a project if a checkout/update is in progress.

      Alternately, you could query the repository directly (i.e. svn log --limit 1
      http://svnrepo/code/some/project) and compare the result to the previous change which
      triggered a build.

      You can workaround this issue by changing your poll schedule to something less frequent
      which will allow your build time to finish the checkout.

          [JENKINS-4270] SVN polling, clean builds, and long checkout equal endless loop

          tugboat_1 added a comment -

          I ran into this bug, as I moved the build server from a single core machine to a
          dual core machine.

          tugboat_1 added a comment - I ran into this bug, as I moved the build server from a single core machine to a dual core machine.

          Andrew Bayer added a comment -

          I believe this showed up because of the move of polling to the slave - I'm not
          entirely sure how that caused this, but that's what changed in the subversion
          plugin at that time. I've changed the relevant section of pollChanges to check
          whether the previous build is still building before it checks to see if all the
          module locations defined for the previous build of this job are present in the
          workspace. That's the logic that was causing this problem, so with that change,
          it won't force a build due to missing modules in the workspace while a build is
          already running.

          I'm currently verifying that that behavior will still work when the previous
          build isn't running. After that, I'll commit the change, and it'll be in
          subversion plugin 1.7 - which will hopefully be in 1.323, but I'll need to check
          with kohsuke.

          Andrew Bayer added a comment - I believe this showed up because of the move of polling to the slave - I'm not entirely sure how that caused this, but that's what changed in the subversion plugin at that time. I've changed the relevant section of pollChanges to check whether the previous build is still building before it checks to see if all the module locations defined for the previous build of this job are present in the workspace. That's the logic that was causing this problem, so with that change, it won't force a build due to missing modules in the workspace while a build is already running. I'm currently verifying that that behavior will still work when the previous build isn't running. After that, I'll commit the change, and it'll be in subversion plugin 1.7 - which will hopefully be in 1.323, but I'll need to check with kohsuke.

          Code changed in hudson
          User: : abayer
          Path:
          trunk/hudson/plugins/subversion/src/main/java/hudson/scm/SubversionSCM.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=21364
          Log:
          [FIXED JENKINS-4270] Verify there isn't already a build running before checking whether we need a whole new workspace due to module locations not matching up with previous build's revisions.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : abayer Path: trunk/hudson/plugins/subversion/src/main/java/hudson/scm/SubversionSCM.java http://fisheye4.cenqua.com/changelog/hudson/?cs=21364 Log: [FIXED JENKINS-4270] Verify there isn't already a build running before checking whether we need a whole new workspace due to module locations not matching up with previous build's revisions.

          williamleara added a comment -

          abayer, thanks for working on the issue! I'm wondering: will your fix break
          the "Execute concurrent builds" feature? If polling waits until any
          outstanding jobs finish, then they'll never be an opportunity to perform
          concurrent builds... is that true?

          williamleara added a comment - abayer, thanks for working on the issue! I'm wondering: will your fix break the "Execute concurrent builds" feature? If polling waits until any outstanding jobs finish, then they'll never be an opportunity to perform concurrent builds... is that true?

          Andrew Bayer added a comment -

          It doesn't wait 'til no build is running to poll - it only skips one specific
          chunk of behavior, where it checks to see if every module location defined in
          the job config is present in the previous build's revisions.txt and if not,
          forces a build. Polling for changes proper behaves exactly the same whether a
          build is running or not - if changes occur while a build is running or polling,
          they'll trigger a new build

          Andrew Bayer added a comment - It doesn't wait 'til no build is running to poll - it only skips one specific chunk of behavior, where it checks to see if every module location defined in the job config is present in the previous build's revisions.txt and if not, forces a build. Polling for changes proper behaves exactly the same whether a build is running or not - if changes occur while a build is running or polling, they'll trigger a new build

          tharpers added a comment -

          I updated subversion to the "fixed" version and still continue to see the issue.
          When i change polling back to once every minute, multiple builds occur endlessly
          and the polling log indicated it was because the workspace was empty. I'm also
          running Hudson ver. 1.322. I'm happy to provide any logs or any other
          diagnostic assistance.

          tharpers added a comment - I updated subversion to the "fixed" version and still continue to see the issue. When i change polling back to once every minute, multiple builds occur endlessly and the polling log indicated it was because the workspace was empty. I'm also running Hudson ver. 1.322. I'm happy to provide any logs or any other diagnostic assistance.

          tharpers added a comment -

          *also i don't currently run any slaves, this is all occuring with a single
          master system

          tharpers added a comment - *also i don't currently run any slaves, this is all occuring with a single master system

          Andrew Bayer added a comment -

          You can't actually upgrade the Subversion plugin on its own - you'll only get
          1.7 with Hudson 1.323, which will be released on Friday. The bug is definitely
          fixed in that release.

          Andrew Bayer added a comment - You can't actually upgrade the Subversion plugin on its own - you'll only get 1.7 with Hudson 1.323, which will be released on Friday. The bug is definitely fixed in that release.

          To clarify, the problem of not being able to update a bundled plugin is tracked
          as a separate issue.

          Kohsuke Kawaguchi added a comment - To clarify, the problem of not being able to update a bundled plugin is tracked as a separate issue.

          tharpers added a comment -

          Thanks for the clarification. Yeah i manually tried grabbing the 1.7 build and
          overwrote it in the plugin directory. I'll wait for the release.

          Thanks again for your hard work!

          tharpers added a comment - Thanks for the clarification. Yeah i manually tried grabbing the 1.7 build and overwrote it in the plugin directory. I'll wait for the release. Thanks again for your hard work!

            abayer Andrew Bayer
            gregan gregan
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: