Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-8173

SCM polling always fails using Perforce plugin due to "workspace offline" and causes infinite build loop

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • p4-plugin
    • None

      1. Set up an "in demand" slave.
      2. Set up a project that only builds on that slave
      3. The slave will go offline
      4. The SCM poll will see that the workspace is offline and trigger a new build
      5. The slave comes online, the build completes
      6. The slave goes back offline again as in 3.
      7. And here the infinite build loop happens since we now send up back at 4

      I'm not sure why Hudson is trying to get the content of the workspace? The perforce plugin knows the last change list number used for the last build, so it should also know by polling perforce that the last changlist number is different and so a build should be triggered?

      In our environment each slave is actually a virtual machine controlled by scripts, after a build completes its taken offline so checking the workspace will NEVER work. This completely breaks our setup because builds are only set to trigger for SCM changes.

      When a slave is brought back online the VM is reverted to its snapshot to ensure the build is "clean", so again this means checking the workspace content will always fail.

          [JENKINS-8173] SCM polling always fails using Perforce plugin due to "workspace offline" and causes infinite build loop

          Rob Petti added a comment - - edited

          It's not that simple. The plugin tells hudson/jenkins that a workspace is required so that the polling will actually take place on the slave that the job is configured for. Each job is configured for the specific environment it runs in, so it's unreasonable to assume that the same configuration will work on the master (see my comments above...)

          Until the plugin supports node-specific perforce configurations, this issue cannot be fixed.

          Rob Petti added a comment - - edited It's not that simple. The plugin tells hudson/jenkins that a workspace is required so that the polling will actually take place on the slave that the job is configured for. Each job is configured for the specific environment it runs in, so it's unreasonable to assume that the same configuration will work on the master (see my comments above...) Until the plugin supports node-specific perforce configurations, this issue cannot be fixed.

          Paul M added a comment -

          Is there another issue raised on node specific configuration options? Is there a framework in Jenkins to implement this functionality or will it require a change to the core?

          Also even if you configure the perforce binary locations in each slave that will not fix the issue because if the slave is offline then it still can't do the polling. There would still need to be an option that says either "Poll only on Master" or "Don't require workspace for polling".

          Paul M added a comment - Is there another issue raised on node specific configuration options? Is there a framework in Jenkins to implement this functionality or will it require a change to the core? Also even if you configure the perforce binary locations in each slave that will not fix the issue because if the slave is offline then it still can't do the polling. There would still need to be an option that says either "Poll only on Master" or "Don't require workspace for polling".

          Rob Petti added a comment -

          Yes there is an issue open for it: JENKINS-5120. And yes, there is a framework available.

          Once it has been implemented, it's a very simple change to allow polling to be performed on the slave or the master. No additional option is necessary.

          Rob Petti added a comment - Yes there is an issue open for it: JENKINS-5120 . And yes, there is a framework available. Once it has been implemented, it's a very simple change to allow polling to be performed on the slave or the master. No additional option is necessary.

          This worked upto version 1.1.9 which had my fix in it for this very issue - http://issues.jenkins-ci.org/browse/JENKINS-6575

          Changes made to the plugin after this, in 1.1.10 cause a null pointer issue. Been using 1.1.9 uptil now and hadnt noticed.

          I'll take a look and see if I can submit a 'patch' which will get this working again.

          martin franklin added a comment - This worked upto version 1.1.9 which had my fix in it for this very issue - http://issues.jenkins-ci.org/browse/JENKINS-6575 Changes made to the plugin after this, in 1.1.10 cause a null pointer issue. Been using 1.1.9 uptil now and hadnt noticed. I'll take a look and see if I can submit a 'patch' which will get this working again.

          Rob Petti added a comment -

          As mentioned, simply changing the output of requiresWorkspaceForPolling will not fix this in all cases. It is trivial to change it, but it will have far reaching consequences, such as polling simply not working at all in most configurations. This is because polling would then be executed on the master with the slave's perforce options, which would only match in the ideal case.

          Martin, can you please file a new issue for the NPE (and be sure to include the log dump and perforce plugin version). You might also want to try the latest version, since you are really out of date.

          Rob Petti added a comment - As mentioned, simply changing the output of requiresWorkspaceForPolling will not fix this in all cases. It is trivial to change it, but it will have far reaching consequences, such as polling simply not working at all in most configurations. This is because polling would then be executed on the master with the slave's perforce options, which would only match in the ideal case. Martin, can you please file a new issue for the NPE (and be sure to include the log dump and perforce plugin version). You might also want to try the latest version, since you are really out of date.

          Dont think it's worth it to issue a NPE given that the code I was using was so out of date.

          martin franklin added a comment - Dont think it's worth it to issue a NPE given that the code I was using was so out of date.

          Here's the patch I've created which 'gets this working' in a limited sense.

          If requiresWorkspaceForPolling returns false then you can perform a poll on the master without a node or a workspace. This is what previously worked for us, we just set the slave client format to ${basename}.

          Then the AbstractProject class calls scm.pollChanges(this, null, null, listener);

          I've changed the code so that if no buildnode is found, instead of just returning false, pollchanges further also checks to see if requiresWorkspaceForPolling returns false. If so it just proceeds to use the master as previously in 1.1.0.

          I havent completely tested this yet. But my preliminary tests seem to show that it works.

          I believe that my NPE's are related to AbstractProject invoking pollChanges with launcher and workspace set to null. If I still see these occuring I'll file a NPE issue and try to create another patch.

          Martin

          martin franklin added a comment - Here's the patch I've created which 'gets this working' in a limited sense. If requiresWorkspaceForPolling returns false then you can perform a poll on the master without a node or a workspace. This is what previously worked for us, we just set the slave client format to ${basename}. Then the AbstractProject class calls scm.pollChanges(this, null, null, listener); I've changed the code so that if no buildnode is found, instead of just returning false, pollchanges further also checks to see if requiresWorkspaceForPolling returns false. If so it just proceeds to use the master as previously in 1.1.0. I havent completely tested this yet. But my preliminary tests seem to show that it works. I believe that my NPE's are related to AbstractProject invoking pollChanges with launcher and workspace set to null. If I still see these occuring I'll file a NPE issue and try to create another patch. Martin

          if requiresWorkspaceForPolling returns false - never use a build node, so set it to null. This will force p4 to use the master without needed a workspace synced to the file system. Otherwise proceed as before.
          If use this with multiple slaves, dynamic or not, you must use the force sync option.

          martin franklin added a comment - if requiresWorkspaceForPolling returns false - never use a build node, so set it to null. This will force p4 to use the master without needed a workspace synced to the file system. Otherwise proceed as before. If use this with multiple slaves, dynamic or not, you must use the force sync option.

          Rob Petti added a comment -

          Setting buildNode to null doesn't actually force it to run on any particular node. It will still use the launcher provided by Jenkins, which could be running on anything.

          Rob Petti added a comment - Setting buildNode to null doesn't actually force it to run on any particular node. It will still use the launcher provided by Jenkins, which could be running on anything.

          Rob Petti added a comment -

          This issue is moot now that there is an option to poll only on the master. See JENKINS-9067.

          Rob Petti added a comment - This issue is moot now that there is an option to poll only on the master. See JENKINS-9067 .

            rpetti Rob Petti
            paulmoran paulmoran
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: