• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • p4-plugin
    • None
    • Jenkins LTS 1.565.2
      P4 Plugin 1.0.16

      The p4-plugin doesn't seem to support concurrent builds. I have multiple slave nodes and executors. My job is setup to launch concurrent builds. The behavior I'm seeing is only one build is run at a time across all the executors. Once the build is done, then the next build is kicked off. The plugin should be constantly polling based on polling schedule and add builds to queue.

      I'm pretty positive this is a limitation to p4-plugin as when I change to use old Perforce plugin, then concurrent builds are launched as expected.

          [JENKINS-25242] p4-plugin: Doesn't do concurrent builds

          swolk created issue -

          swolk added a comment -

          Did a bit of debugging. Checking the scm-polling.log I can see that polling gets stuck the next poll after a build has started.

          I then did a kill -3 on the main jenkins process and saw this:

          "Waiting to acquire /acc1/jenkins/workspace/test_stream-mainline-nonstop : SCM polling for hudson.model.FreeStyleProject@1b337068[test_stream-mainline-nonstop]" prio=10 tid=0x00007f6d2000a000 nid=0x46cc in Object.wait() [0x00007f6e2cd32000]
             java.lang.Thread.State: WAITING (on object monitor)
          	at java.lang.Object.wait(Native Method)
          	at java.lang.Object.wait(Object.java:503)
          	at hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:243)
          	- locked <0x00000003028dce60> (a hudson.slaves.WorkspaceList)
          	at hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:222)
          	- locked <0x00000003028dce60> (a hudson.slaves.WorkspaceList)
          	at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1441)
          	at hudson.model.AbstractProject._poll(AbstractProject.java:1420)
          	at hudson.model.AbstractProject.poll(AbstractProject.java:1331)
          	at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:477)
          	at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:506)
          	at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118)
          	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
          	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          	at java.lang.Thread.run(Thread.java:745)
          

          The polling mechanism is getting stuck trying to acquire the workspace list for some reason.

          swolk added a comment - Did a bit of debugging. Checking the scm-polling.log I can see that polling gets stuck the next poll after a build has started. I then did a kill -3 on the main jenkins process and saw this: "Waiting to acquire /acc1/jenkins/workspace/test_stream-mainline-nonstop : SCM polling for hudson.model.FreeStyleProject@1b337068[test_stream-mainline-nonstop]" prio=10 tid=0x00007f6d2000a000 nid=0x46cc in Object.wait() [0x00007f6e2cd32000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:503) at hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:243) - locked <0x00000003028dce60> (a hudson.slaves.WorkspaceList) at hudson.slaves.WorkspaceList.acquire(WorkspaceList.java:222) - locked <0x00000003028dce60> (a hudson.slaves.WorkspaceList) at hudson.model.AbstractProject.pollWithWorkspace(AbstractProject.java:1441) at hudson.model.AbstractProject._poll(AbstractProject.java:1420) at hudson.model.AbstractProject.poll(AbstractProject.java:1331) at hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:477) at hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:506) at hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:118) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) The polling mechanism is getting stuck trying to acquire the workspace list for some reason.

          swolk added a comment -

          After more digging, noticed that in AbstractProject._poll, because PerforceSCM.requiresWorkspaceForPolling() is set to return true, looks like it has to block waiting for workspace to free up. I'm going to test changing requiresWorkspaceForPolling to return false instead which is what the old perforce plugin does.

          swolk added a comment - After more digging, noticed that in AbstractProject._poll, because PerforceSCM.requiresWorkspaceForPolling() is set to return true, looks like it has to block waiting for workspace to free up. I'm going to test changing requiresWorkspaceForPolling to return false instead which is what the old perforce plugin does.

          Paul Allen added a comment -

          Are you polling on the same workspace?

          For the same workspace it should block, you don't want to be syncing new code while you are building. However it should not block while polling on other workspaces.

          Please let me know what you find; concurrency issues are some what harder to test for.

          Paul Allen added a comment - Are you polling on the same workspace? For the same workspace it should block, you don't want to be syncing new code while you are building. However it should not block while polling on other workspaces. Please let me know what you find; concurrency issues are some what harder to test for.
          Paul Allen made changes -
          Assignee New: Paul Allen [ p4paul ]

          swolk added a comment -

          Hi Paul, I setup my job using stream workspace with Workspace Name Format jenkins-${NODE_NAME}-${JOB_NAME}. In my environment, we have a master that just runs Jenkins and doesn't do any building, while we have 5 build slaves each of which has 2 executors. Being only able to run one build at a time across all nodes per job is very inefficient.

          Can't we have a separate client that only runs on the master doing the polling and checking for code changes? Of course, adding the post-commit hook would be ideal (JENKINS-25249), hint, hint

          BTW, when I changed requiresWorkspaceForPolling to return false, then concurrent builds were launched, unfortunately it keeps launching extraneous builds. For example, I made one change, then 3 differ builds got launched!

          swolk added a comment - Hi Paul, I setup my job using stream workspace with Workspace Name Format jenkins-${NODE_NAME}-${JOB_NAME}. In my environment, we have a master that just runs Jenkins and doesn't do any building, while we have 5 build slaves each of which has 2 executors. Being only able to run one build at a time across all nodes per job is very inefficient. Can't we have a separate client that only runs on the master doing the polling and checking for code changes? Of course, adding the post-commit hook would be ideal ( JENKINS-25249 ), hint, hint BTW, when I changed requiresWorkspaceForPolling to return false, then concurrent builds were launched, unfortunately it keeps launching extraneous builds. For example, I made one change, then 3 differ builds got launched!

          Paul Allen added a comment -

          Hi, you mentioned that you are using Streams with a Multi-configuration project.

          Are you using a &

          {var}

          parameter in the Stream Codeline name? Or are all your configurations using the same stream? My point being if every configuration has the same workspace view (e.g. the same files) then a change in that stream will kick a build off on all the configurations.

          Several people have asked if there is a way to prevent the master from building and I am planning to add something to support this.

          Paul Allen added a comment - Hi, you mentioned that you are using Streams with a Multi-configuration project. Are you using a & {var} parameter in the Stream Codeline name? Or are all your configurations using the same stream? My point being if every configuration has the same workspace view (e.g. the same files) then a change in that stream will kick a build off on all the configurations. Several people have asked if there is a way to prevent the master from building and I am planning to add something to support this.

          I tried to get around this by using Manual (custom view) together with a workspace name like this: Jenkins-${JOB_NAME}-${NODE_NAME}-${EXECUTOR_NUMBER}-workspace. I thought this should work since the executor number is unique for concurrent builds when running on a single node (as I did), which should make the workspace name unique.

          When running one build at a time, everything works fine, but when running several concurrent builds, I only managed to get the first of the concurrent builds to fetch files from the repository. Any other concurrent build's workspaces were empty. This happens every time when I launch a build while another build is running, even when the other build is done running its P4 commands.

          I'm using the P4 Plugin v1.1.4 on Jenkins 1.565.3 (LTS). This issue is a blocker for us and many other large companies, requiring fast feedback for frequent commits.

          David Pärsson added a comment - I tried to get around this by using Manual (custom view) together with a workspace name like this: Jenkins-${JOB_NAME}-${NODE_NAME}-${EXECUTOR_NUMBER}-workspace . I thought this should work since the executor number is unique for concurrent builds when running on a single node (as I did), which should make the workspace name unique. When running one build at a time, everything works fine, but when running several concurrent builds, I only managed to get the first of the concurrent builds to fetch files from the repository. Any other concurrent build's workspaces were empty. This happens every time when I launch a build while another build is running, even when the other build is done running its P4 commands. I'm using the P4 Plugin v1.1.4 on Jenkins 1.565.3 (LTS). This issue is a blocker for us and many other large companies, requiring fast feedback for frequent commits.

          Paul Allen added a comment -

          1. It might be the workspace root is set to the same location for both workspaces.

          2. I seem to remember in some situations Jenkins adds an @2 into the Job name. For obvious reasons this plays havoc with Perforce.

          Please let me know if either case applies.

          Paul Allen added a comment - 1. It might be the workspace root is set to the same location for both workspaces. 2. I seem to remember in some situations Jenkins adds an @2 into the Job name. For obvious reasons this plays havoc with Perforce. Please let me know if either case applies.

          For concurrent builds on the same node, Jenkins creates workspaces named workspace, workspace@2, workspace@3 and so on. For what reasons does this not work with Perforce?

          I just found out that it's possible to change the separator character using this system property: -Dhudson.slaves.WorkspaceList=SEPARATOR. I'll try this tomorrow and get back to you with the results.

          David Pärsson added a comment - For concurrent builds on the same node, Jenkins creates workspaces named workspace , workspace@2 , workspace@3 and so on. For what reasons does this not work with Perforce? I just found out that it's possible to change the separator character using this system property: -Dhudson.slaves.WorkspaceList=SEPARATOR . I'll try this tomorrow and get back to you with the results.

            p4paul Paul Allen
            swolk swolk
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: