Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-1022

Interaction between Perforce plugin and Matrix builds

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • p4-plugin
    • None
    • Platform: All, OS: All

      If I'm using the Perforce plugin and have a setting for this project
      (FPHibernate-Head) with a workspace like this:
      Single Workspace: wyliek.hudson-clients.fphibernate-head.london
      View: //fotech/shared/lib/java/FPHibernate/head/...
      Hudson Home: /scratch/betasx2/wyliek/hudson-home

      And am using a configuration matrix of jdk1.6.0_03 and jdk1.5.0_13, what seems
      to happen is this:
      1) Perforce plugin does a sync to
      /scratch/betasx2/wyliek/hudson-home/jobs/FPHibernate-Head/workspace
      2) Matrix runner creates axis directory of
      /scratch/betasx2/wyliek/hudson-home/jobs/FPHibernate-Head/workspace/jdk/jdk1.5.0_13
      3) Perforce Plugin changes the root of the workspace to
      /scratch/betasx2/wyliek/hudson-home/jobs/FPHibernate-Head/workspace/jdk/jdk1.5.0_13
      and does a p4 sync. P4 says nothing to sync, but there's nothing there.
      4) Ant doesn't find the build.xml file to run.

      Here's the output on the overall project running:
      started
      Performing sync with Perforce for: //fotech/shared/lib/java/FPHibernate/head/...
      Changing P4 Client Root to:
      /scratch/betasx2/wyliek/hudson-home/jobs/FPHibernate-Head/workspace/
      Changing P4 Client View to: //fotech/shared/lib/java/FPHibernate/head/...
      //wyliek.hudson-clients.fphibernate-head.london/...
      Last sync'd change: 76672
      Sync'ing workspace to depot.
      ForceSync flag is set, forcing: p4 sync
      //fotech/shared/lib/java/FPHibernate/head/...
      Sync complete, took 3392 MS
      Triggering jdk=jdk1.6.0_03
      Triggering jdk=jdk1.5.0_13
      finished: FAILURE

      Here's the output on the jdk1.6.0_03 axis run:
      started
      Performing sync with Perforce for: //fotech/shared/lib/java/FPHibernate/head/...
      Changing P4 Client Root to:
      /scratch/betasx2/wyliek/hudson-home/jobs/FPHibernate-Head/workspace/jdk/jdk1.6.0_03/
      Changing P4 Client View to: //fotech/shared/lib/java/FPHibernate/head/...
      //wyliek.hudson-clients.fphibernate-head.london/...
      Last sync'd change: 76672
      No changes since last build.
      [jdk1.6.0_03] $ /scratch/betasx2/wyliek/apache-ant-1.7.0/bin/ant
      -Djdk=jdk1.6.0_03 tests
      Buildfile: build.xml does not exist!
      Build failed
      Recording test results
      finished: FAILURE

      Here's the output on the jdk1.5.0_13 axis run:
      started
      Performing sync with Perforce for: //fotech/shared/lib/java/FPHibernate/head/...
      Changing P4 Client Root to:
      /scratch/betasx2/wyliek/hudson-home/jobs/FPHibernate-Head/workspace/jdk/jdk1.5.0_13/
      Changing P4 Client View to: //fotech/shared/lib/java/FPHibernate/head/...
      //wyliek.hudson-clients.fphibernate-head.london/...
      Last sync'd change: 76672
      No changes since last build.
      [jdk1.5.0_13] $ /scratch/betasx2/wyliek/apache-ant-1.7.0/bin/ant
      -Djdk=jdk1.5.0_13 tests
      Buildfile: build.xml does not exist!
      Build failed
      Recording test results
      finished: FAILURE

          [JENKINS-1022] Interaction between Perforce plugin and Matrix builds

          digerata added a comment -

          Marking this as WONTFIX because it is a symptom of behavior of the Perforce client. There isn't anything
          we can do from the Hudson plugin's standpoint.

          digerata added a comment - Marking this as WONTFIX because it is a symptom of behavior of the Perforce client. There isn't anything we can do from the Hudson plugin's standpoint.

          emmulator added a comment -

          I've just noted the same behavior in this thread on the users mailing list:

          http://www.nabble.com/Perforce-workspace-issues-in-matrix-project-td25198191.html

          I don't understand how you can say this is just the behavior of Perforce and not
          something you can fix. Hudson already has behavior specific to Perforce in
          creating and managing custom Perforce workspaces, but in this instance it's
          using them incorrectly. Hudson is trying to use the same Perforce workspace for
          two different roots without re-syncing.

          In the thread, I offered two solutions:

          1) Don't run the main job on the same node/slave as the axis jobs – this would
          also have the benefit of not blocking itself when there is only one executor on
          a slave.

          2) Create additional Perforce workspaces for the axis jobs – just as the main
          job syncs to "workspace" and the axis jobs sync to "workspace/axis", the main
          job could use Perforce workspace "x" and the axis jobs could use Perforce
          workspace "x-axis".

          Please reconsider this issue. Otherwise you should probably document on the
          matrix feature that it is not inteded to work with Perforce.

          emmulator added a comment - I've just noted the same behavior in this thread on the users mailing list: http://www.nabble.com/Perforce-workspace-issues-in-matrix-project-td25198191.html I don't understand how you can say this is just the behavior of Perforce and not something you can fix. Hudson already has behavior specific to Perforce in creating and managing custom Perforce workspaces, but in this instance it's using them incorrectly. Hudson is trying to use the same Perforce workspace for two different roots without re-syncing. In the thread, I offered two solutions: 1) Don't run the main job on the same node/slave as the axis jobs – this would also have the benefit of not blocking itself when there is only one executor on a slave. 2) Create additional Perforce workspaces for the axis jobs – just as the main job syncs to "workspace" and the axis jobs sync to "workspace/axis", the main job could use Perforce workspace "x" and the axis jobs could use Perforce workspace "x-axis". Please reconsider this issue. Otherwise you should probably document on the matrix feature that it is not inteded to work with Perforce.

          digerata added a comment -

          Reassigning to issues@hudson as I no longer work on this project.

          digerata added a comment - Reassigning to issues@hudson as I no longer work on this project.

          emmulator added a comment -

          I saw other people asking for the ability to tie the parent matrix job to a
          particular node in Issue 936, so I thought I'd try that as a workaround for this
          issue as well. I just copied the "master/slave" section from the
          AbstractItem/configure-common.jelly to the
          MatrixProject/configure-entries.jelly, so that the "Tie this project to a node"
          option would be exposed for Matrix projects as well. Then I tied the Matrix
          project itself to my Master node, and for the axis I just selected two
          "Individual Nodes" on other machines. It looks like it should be working now
          – I see three progress bars in the "Build Executor Status" section as I would
          expect. I can see from the logs that it is now using three distinct perforce
          workspaces, but one of them still doesn't sync. Here are the relevant portions
          of the build logs:

          --------------------

          Started by user anonymous
          Building on master
          Using master perforce client: experimental_boogaloo_4
          [workspace] $ p4 workspace -o experimental_boogaloo_4
          Changing P4 Client Root to: /home/cruise/hudson/jobs/experimental boogaloo
          4/workspace/
          Changing P4 Client View to: //cj/personal/jemmer/ExperimentalAndUtility/...
          //experimental_boogaloo_4/...

          Saving new client experimental_boogaloo_4
          [workspace] $ p4 -s client -i
          Last sync'd change: 95162
          [workspace] $ p4 counter change
          [workspace] $ p4 -s changes //experimental_boogaloo_4/...@95163,@95878
          Sync'ing workspace to changelist 95878 (forcing sync of unchanged files).
          [workspace] $ p4 sync -f //experimental_boogaloo_4/...@95878
          Sync complete, took 5173 ms
          Triggering build106_bugfix
          Triggering build107_bugfix
          Finished: FAILURE

          --------------------

          Started by upstream project "experimental boogaloo 4" build number 1
          Building remotely on build106_bugfix
          Using remote perforce client: experimental_boogaloo_4-build106
          Changing P4 Client Root to: /home/cruise/hudson_bugfix/workspace/experimental
          boogaloo 4/label/build106_bugfix/
          Changing P4 Client View to: //cj/personal/jemmer/ExperimentalAndUtility/...
          //experimental_boogaloo_4-build106/...

          Saving new client experimental_boogaloo_4-build106
          Last sync'd change: 0
          Sync'ing workspace to changelist 95878.
          Sync complete, took 68 ms
          [maven output complaining that there's no pom.xml]
          Finished: FAILURE

          --------------------

          Started by upstream project "experimental boogaloo 4" build number 1
          Building remotely on build107_bugfix
          Using remote perforce client: experimental_boogaloo_4-build107
          Changing P4 Client Root to: /home/cruise/hudson_bugfix/workspace/experimental
          boogaloo 4/label/build107_bugfix/
          Changing P4 Client View to: //cj/personal/jemmer/ExperimentalAndUtility/...
          //experimental_boogaloo_4-build107/...

          Saving new client experimental_boogaloo_4-build107
          Last sync'd change: 0
          Sync'ing workspace to changelist 95878.
          Sync complete, took 4880 ms
          [maven output showing successful build]
          Finished: SUCCESS

          --------------------

          If I go onto the 'build106' machine, which didn't sync anything to its
          workspace, and issue the following command:

          p4 -c experimental_boogaloo_4-build106 sync -n

          It does report that it would add all the appropriate files.

          So now I'm stumped and it seems my proposed workaround doesn't work and
          something will indeed probably have to be done in the Perforce plugin itself.

          emmulator added a comment - I saw other people asking for the ability to tie the parent matrix job to a particular node in Issue 936, so I thought I'd try that as a workaround for this issue as well. I just copied the "master/slave" section from the AbstractItem/configure-common.jelly to the MatrixProject/configure-entries.jelly, so that the "Tie this project to a node" option would be exposed for Matrix projects as well. Then I tied the Matrix project itself to my Master node, and for the axis I just selected two "Individual Nodes" on other machines. It looks like it should be working now – I see three progress bars in the "Build Executor Status" section as I would expect. I can see from the logs that it is now using three distinct perforce workspaces, but one of them still doesn't sync. Here are the relevant portions of the build logs: -------------------- Started by user anonymous Building on master Using master perforce client: experimental_boogaloo_4 [workspace] $ p4 workspace -o experimental_boogaloo_4 Changing P4 Client Root to: /home/cruise/hudson/jobs/experimental boogaloo 4/workspace/ Changing P4 Client View to: //cj/personal/jemmer/ExperimentalAndUtility/... //experimental_boogaloo_4/... Saving new client experimental_boogaloo_4 [workspace] $ p4 -s client -i Last sync'd change: 95162 [workspace] $ p4 counter change [workspace] $ p4 -s changes //experimental_boogaloo_4/...@95163,@95878 Sync'ing workspace to changelist 95878 (forcing sync of unchanged files). [workspace] $ p4 sync -f //experimental_boogaloo_4/...@95878 Sync complete, took 5173 ms Triggering build106_bugfix Triggering build107_bugfix Finished: FAILURE -------------------- Started by upstream project "experimental boogaloo 4" build number 1 Building remotely on build106_bugfix Using remote perforce client: experimental_boogaloo_4-build106 Changing P4 Client Root to: /home/cruise/hudson_bugfix/workspace/experimental boogaloo 4/label/build106_bugfix/ Changing P4 Client View to: //cj/personal/jemmer/ExperimentalAndUtility/... //experimental_boogaloo_4-build106/... Saving new client experimental_boogaloo_4-build106 Last sync'd change: 0 Sync'ing workspace to changelist 95878. Sync complete, took 68 ms [maven output complaining that there's no pom.xml] Finished: FAILURE -------------------- Started by upstream project "experimental boogaloo 4" build number 1 Building remotely on build107_bugfix Using remote perforce client: experimental_boogaloo_4-build107 Changing P4 Client Root to: /home/cruise/hudson_bugfix/workspace/experimental boogaloo 4/label/build107_bugfix/ Changing P4 Client View to: //cj/personal/jemmer/ExperimentalAndUtility/... //experimental_boogaloo_4-build107/... Saving new client experimental_boogaloo_4-build107 Last sync'd change: 0 Sync'ing workspace to changelist 95878. Sync complete, took 4880 ms [maven output showing successful build] Finished: SUCCESS -------------------- If I go onto the 'build106' machine, which didn't sync anything to its workspace, and issue the following command: p4 -c experimental_boogaloo_4-build106 sync -n It does report that it would add all the appropriate files. So now I'm stumped and it seems my proposed workaround doesn't work and something will indeed probably have to be done in the Perforce plugin itself.

          emmulator added a comment -

          I got access to the perforce server logs and can now demonstrate that the plugin
          is not issuing the commands from the correct servers. Here are all the perforce
          commands issued during two change-triggered builds:

          2009/10/13 17:21:00 pid 19985
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o
          hudson_sample_matrix-build106'
          2009/10/13 17:21:01 pid 19986
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes -m 1
          //hudson_sample_matrix-build106/...'
          2009/10/13 17:21:06 pid 19992
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o
          hudson_sample_matrix-build106'
          2009/10/13 17:21:06 pid 19993
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-client -i'
          2009/10/13 17:21:06 pid 19994
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change'
          2009/10/13 17:21:06 pid 19995
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes
          //hudson_sample_matrix-build106/...@96579,@96584'
          2009/10/13 17:21:06 pid 19996
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-describe -s 96584'
          2009/10/13 17:21:07 pid 19997
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync
          //hudson_sample_matrix-build106/...@96584'
          2009/10/13 17:21:08 pid 20002
          build.automation@hudson_sample_matrix-build108 x.x.x..83 'user-workspace -o
          hudson_sample_matrix-build108'
          2009/10/13 17:21:08 pid 20003
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-workspace -o
          hudson_sample_matrix-build107'
          2009/10/13 17:21:08 pid 20004
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change'
          2009/10/13 17:21:08 pid 20005
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o
          hudson_sample_matrix-build106'
          2009/10/13 17:21:08 pid 20006
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change'
          2009/10/13 17:21:08 pid 20007
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes
          //hudson_sample_matrix-build107/...@96581,@96584'
          2009/10/13 17:21:08 pid 20008
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes
          //hudson_sample_matrix-build108/...@96581,@96584'
          2009/10/13 17:21:08 pid 20009
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-client -i'
          2009/10/13 17:21:08 pid 20010
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync
          //hudson_sample_matrix-build107/...@96584'
          2009/10/13 17:21:08 pid 20011
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync
          //hudson_sample_matrix-build108/...@96584'
          2009/10/13 17:21:08 pid 20012
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change'
          2009/10/13 17:21:08 pid 20013
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes
          //hudson_sample_matrix-build106/...@96579,@96584'
          2009/10/13 17:21:08 pid 20014
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-describe -s 96584'
          2009/10/13 17:21:08 pid 20015
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync
          //hudson_sample_matrix-build106/...@96584'

          2009/10/13 17:31:01 pid 20878
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o
          hudson_sample_matrix-build106'
          2009/10/13 17:31:01 pid 20879
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes -m 1
          //hudson_sample_matrix-build106/...'
          2009/10/13 17:31:06 pid 20885
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o
          hudson_sample_matrix-build106'
          2009/10/13 17:31:06 pid 20886
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-client -i'
          2009/10/13 17:31:06 pid 20887
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change'
          2009/10/13 17:31:06 pid 20888
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes
          //hudson_sample_matrix-build106/...@96585,@96586'
          2009/10/13 17:31:06 pid 20889
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-describe -s 96585'
          2009/10/13 17:31:06 pid 20890
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync
          //hudson_sample_matrix-build106/...@96585'
          2009/10/13 17:31:07 pid 20892
          build.automation@hudson_sample_matrix-build108 x.x.x..83 'user-workspace -o
          hudson_sample_matrix-build108'
          2009/10/13 17:31:07 pid 20893
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-workspace -o
          hudson_sample_matrix-build107'
          2009/10/13 17:31:07 pid 20894
          build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o
          hudson_sample_matrix-build106'
          2009/10/13 17:31:07 pid 20895
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-counter change'
          2009/10/13 17:31:08 pid 20896
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-counter change'
          2009/10/13 17:31:08 pid 20897
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-client -i'
          2009/10/13 17:31:08 pid 20898
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-changes
          //hudson_sample_matrix-build108/...@96585,@96586'
          2009/10/13 17:31:08 pid 20899
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-changes
          //hudson_sample_matrix-build107/...@96585,@96586'
          2009/10/13 17:31:08 pid 20903
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-counter change'
          2009/10/13 17:31:08 pid 20904
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-sync
          //hudson_sample_matrix-build108/...@96586'
          2009/10/13 17:31:08 pid 20905
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-describe -s 96585'
          2009/10/13 17:31:08 pid 20906
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-changes
          //hudson_sample_matrix-build106/...@96585,@96586'
          2009/10/13 17:31:08 pid 20907
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-sync
          //hudson_sample_matrix-build107/...@96585'
          2009/10/13 17:31:08 pid 20908
          build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-sync
          //hudson_sample_matrix-build106/...@96586'

          The build is running on three slaves on three different machines build106
          (x.x.x.79), build107 (x.x.x.81), and build108 (x.x.x.83). You can see that the
          plugin is using the wrong client from the wrong machine most of the time.

          emmulator added a comment - I got access to the perforce server logs and can now demonstrate that the plugin is not issuing the commands from the correct servers. Here are all the perforce commands issued during two change-triggered builds: 2009/10/13 17:21:00 pid 19985 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/13 17:21:01 pid 19986 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes -m 1 //hudson_sample_matrix-build106/...' 2009/10/13 17:21:06 pid 19992 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/13 17:21:06 pid 19993 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-client -i' 2009/10/13 17:21:06 pid 19994 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change' 2009/10/13 17:21:06 pid 19995 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes //hudson_sample_matrix-build106/...@96579,@96584' 2009/10/13 17:21:06 pid 19996 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-describe -s 96584' 2009/10/13 17:21:07 pid 19997 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync //hudson_sample_matrix-build106/...@96584' 2009/10/13 17:21:08 pid 20002 build.automation@hudson_sample_matrix-build108 x.x.x..83 'user-workspace -o hudson_sample_matrix-build108' 2009/10/13 17:21:08 pid 20003 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-workspace -o hudson_sample_matrix-build107' 2009/10/13 17:21:08 pid 20004 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change' 2009/10/13 17:21:08 pid 20005 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/13 17:21:08 pid 20006 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change' 2009/10/13 17:21:08 pid 20007 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes //hudson_sample_matrix-build107/...@96581,@96584' 2009/10/13 17:21:08 pid 20008 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes //hudson_sample_matrix-build108/...@96581,@96584' 2009/10/13 17:21:08 pid 20009 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-client -i' 2009/10/13 17:21:08 pid 20010 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync //hudson_sample_matrix-build107/...@96584' 2009/10/13 17:21:08 pid 20011 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync //hudson_sample_matrix-build108/...@96584' 2009/10/13 17:21:08 pid 20012 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change' 2009/10/13 17:21:08 pid 20013 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes //hudson_sample_matrix-build106/...@96579,@96584' 2009/10/13 17:21:08 pid 20014 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-describe -s 96584' 2009/10/13 17:21:08 pid 20015 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync //hudson_sample_matrix-build106/...@96584' 2009/10/13 17:31:01 pid 20878 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/13 17:31:01 pid 20879 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes -m 1 //hudson_sample_matrix-build106/...' 2009/10/13 17:31:06 pid 20885 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/13 17:31:06 pid 20886 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-client -i' 2009/10/13 17:31:06 pid 20887 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-counter change' 2009/10/13 17:31:06 pid 20888 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-changes //hudson_sample_matrix-build106/...@96585,@96586' 2009/10/13 17:31:06 pid 20889 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-describe -s 96585' 2009/10/13 17:31:06 pid 20890 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-sync //hudson_sample_matrix-build106/...@96585' 2009/10/13 17:31:07 pid 20892 build.automation@hudson_sample_matrix-build108 x.x.x..83 'user-workspace -o hudson_sample_matrix-build108' 2009/10/13 17:31:07 pid 20893 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-workspace -o hudson_sample_matrix-build107' 2009/10/13 17:31:07 pid 20894 build.automation@hudson_sample_matrix-build106 x.x.x..79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/13 17:31:07 pid 20895 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-counter change' 2009/10/13 17:31:08 pid 20896 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-counter change' 2009/10/13 17:31:08 pid 20897 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-client -i' 2009/10/13 17:31:08 pid 20898 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-changes //hudson_sample_matrix-build108/...@96585,@96586' 2009/10/13 17:31:08 pid 20899 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-changes //hudson_sample_matrix-build107/...@96585,@96586' 2009/10/13 17:31:08 pid 20903 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-counter change' 2009/10/13 17:31:08 pid 20904 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-sync //hudson_sample_matrix-build108/...@96586' 2009/10/13 17:31:08 pid 20905 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-describe -s 96585' 2009/10/13 17:31:08 pid 20906 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-changes //hudson_sample_matrix-build106/...@96585,@96586' 2009/10/13 17:31:08 pid 20907 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-sync //hudson_sample_matrix-build107/...@96585' 2009/10/13 17:31:08 pid 20908 build.automation@hudson_sample_matrix-build107 x.x.x..81 'user-sync //hudson_sample_matrix-build106/...@96586' The build is running on three slaves on three different machines build106 (x.x.x.79), build107 (x.x.x.81), and build108 (x.x.x.83). You can see that the plugin is using the wrong client from the wrong machine most of the time.

          emmulator added a comment -

          OK, I've tracked it down to the fact that the same PerforceSCM object instance
          is being used by all the build threads, and they're all simultaneously modifying
          and using the same 'depot' instance variable. If I remove that instance
          variable, it actually works. As demonstrated by the following server log, each
          slave instance/machine is now issuing independent commands:

          2009/10/14 17:51:03 pid 28461 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-workspace -o hudson_sample_matrix-build106'
          2009/10/14 17:51:03 pid 28462 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-changes -m 1 //hudson_sample_matrix-build106/...'
          2009/10/14 17:51:10 pid 28477 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-workspace -o hudson_sample_matrix-build106'
          2009/10/14 17:51:10 pid 28478 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-counter change'
          2009/10/14 17:51:10 pid 28479 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-changes //hudson_sample_matrix-build106/...@96623,@96635'
          2009/10/14 17:51:10 pid 28480 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-describe -s 96635'
          2009/10/14 17:51:10 pid 28481 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-sync //hudson_sample_matrix-build106/...@96635'
          2009/10/14 17:51:12 pid 28483 build.automation@hudson_sample_matrix-build108
          x.x.x.83 'user-workspace -o hudson_sample_matrix-build108'
          2009/10/14 17:51:12 pid 28484 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-workspace -o hudson_sample_matrix-build106'
          2009/10/14 17:51:12 pid 28485 build.automation@hudson_sample_matrix-build107
          x.x.x.81 'user-workspace -o hudson_sample_matrix-build107'
          2009/10/14 17:51:12 pid 28486 build.automation@hudson_sample_matrix-build108
          x.x.x.83 'user-counter change'
          2009/10/14 17:51:12 pid 28487 build.automation@hudson_sample_matrix-build107
          x.x.x.81 'user-counter change'
          2009/10/14 17:51:12 pid 28488 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-counter change'
          2009/10/14 17:51:12 pid 28489 build.automation@hudson_sample_matrix-build108
          x.x.x.83 'user-changes //hudson_sample_matrix-build108/...@96623,@96635'
          2009/10/14 17:51:12 pid 28490 build.automation@hudson_sample_matrix-build107
          x.x.x.81 'user-changes //hudson_sample_matrix-build107/...@96623,@96635'
          2009/10/14 17:51:12 pid 28491 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-changes //hudson_sample_matrix-build106/...@96623,@96635'
          2009/10/14 17:51:12 pid 28492 build.automation@hudson_sample_matrix-build108
          x.x.x.83 'user-describe -s 96635'
          2009/10/14 17:51:12 pid 28493 build.automation@hudson_sample_matrix-build107
          x.x.x.81 'user-describe -s 96635'
          2009/10/14 17:51:12 pid 28494 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-describe -s 96635'
          2009/10/14 17:51:12 pid 28495 build.automation@hudson_sample_matrix-build108
          x.x.x.83 'user-sync //hudson_sample_matrix-build108/...@96635'
          2009/10/14 17:51:12 pid 28496 build.automation@hudson_sample_matrix-build107
          x.x.x.81 'user-sync //hudson_sample_matrix-build107/...@96635'
          2009/10/14 17:51:12 pid 28497 build.automation@hudson_sample_matrix-build106
          x.x.x.79 'user-sync //hudson_sample_matrix-build106/...@96635'

          I also had to change PerforceMailResolver, which was also depending on the
          instance attribute, but I'm not sure where this is used to test whether my
          change to that class worked.

          If I can get commit access to subversion (user 'emmulator') I can submit these
          changes.

          emmulator added a comment - OK, I've tracked it down to the fact that the same PerforceSCM object instance is being used by all the build threads, and they're all simultaneously modifying and using the same 'depot' instance variable. If I remove that instance variable, it actually works. As demonstrated by the following server log, each slave instance/machine is now issuing independent commands: 2009/10/14 17:51:03 pid 28461 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/14 17:51:03 pid 28462 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-changes -m 1 //hudson_sample_matrix-build106/...' 2009/10/14 17:51:10 pid 28477 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/14 17:51:10 pid 28478 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-counter change' 2009/10/14 17:51:10 pid 28479 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-changes //hudson_sample_matrix-build106/...@96623,@96635' 2009/10/14 17:51:10 pid 28480 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-describe -s 96635' 2009/10/14 17:51:10 pid 28481 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-sync //hudson_sample_matrix-build106/...@96635' 2009/10/14 17:51:12 pid 28483 build.automation@hudson_sample_matrix-build108 x.x.x.83 'user-workspace -o hudson_sample_matrix-build108' 2009/10/14 17:51:12 pid 28484 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-workspace -o hudson_sample_matrix-build106' 2009/10/14 17:51:12 pid 28485 build.automation@hudson_sample_matrix-build107 x.x.x.81 'user-workspace -o hudson_sample_matrix-build107' 2009/10/14 17:51:12 pid 28486 build.automation@hudson_sample_matrix-build108 x.x.x.83 'user-counter change' 2009/10/14 17:51:12 pid 28487 build.automation@hudson_sample_matrix-build107 x.x.x.81 'user-counter change' 2009/10/14 17:51:12 pid 28488 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-counter change' 2009/10/14 17:51:12 pid 28489 build.automation@hudson_sample_matrix-build108 x.x.x.83 'user-changes //hudson_sample_matrix-build108/...@96623,@96635' 2009/10/14 17:51:12 pid 28490 build.automation@hudson_sample_matrix-build107 x.x.x.81 'user-changes //hudson_sample_matrix-build107/...@96623,@96635' 2009/10/14 17:51:12 pid 28491 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-changes //hudson_sample_matrix-build106/...@96623,@96635' 2009/10/14 17:51:12 pid 28492 build.automation@hudson_sample_matrix-build108 x.x.x.83 'user-describe -s 96635' 2009/10/14 17:51:12 pid 28493 build.automation@hudson_sample_matrix-build107 x.x.x.81 'user-describe -s 96635' 2009/10/14 17:51:12 pid 28494 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-describe -s 96635' 2009/10/14 17:51:12 pid 28495 build.automation@hudson_sample_matrix-build108 x.x.x.83 'user-sync //hudson_sample_matrix-build108/...@96635' 2009/10/14 17:51:12 pid 28496 build.automation@hudson_sample_matrix-build107 x.x.x.81 'user-sync //hudson_sample_matrix-build107/...@96635' 2009/10/14 17:51:12 pid 28497 build.automation@hudson_sample_matrix-build106 x.x.x.79 'user-sync //hudson_sample_matrix-build106/...@96635' I also had to change PerforceMailResolver, which was also depending on the instance attribute, but I'm not sure where this is used to test whether my change to that class worked. If I can get commit access to subversion (user 'emmulator') I can submit these changes.

          Code changed in hudson
          User: : emmulator
          Path:
          trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/HudsonP4Executor.java
          trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceMailResolver.java
          trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceSCM.java
          trunk/hudson/plugins/perforce/src/test/java/hudson/plugins/perforce/PerforceSCMTest.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=22945
          Log:
          JENKINS-1022 removed 'depot' instance variable that was being modified by multiple threads simultaneously, this allows perforce commands to be issued from the correct slave nodes; also had master matrix job not alter the perforce client root, so that the child matrix job would get the update

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : emmulator Path: trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/HudsonP4Executor.java trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceMailResolver.java trunk/hudson/plugins/perforce/src/main/java/hudson/plugins/perforce/PerforceSCM.java trunk/hudson/plugins/perforce/src/test/java/hudson/plugins/perforce/PerforceSCMTest.java http://fisheye4.cenqua.com/changelog/hudson/?cs=22945 Log: JENKINS-1022 removed 'depot' instance variable that was being modified by multiple threads simultaneously, this allows perforce commands to be issued from the correct slave nodes; also had master matrix job not alter the perforce client root, so that the child matrix job would get the update

          emmulator added a comment -

          I'm not going to mark this fixed, since I realized that it probably only works
          with one axis. Since the plugin still only uses one client-spec per node, if
          the matrix includes multiple axes and two are built on the same node, the
          perforce clients will probably still interfere with each other. If I get a
          chance, I will test that.

          emmulator added a comment - I'm not going to mark this fixed, since I realized that it probably only works with one axis. Since the plugin still only uses one client-spec per node, if the matrix includes multiple axes and two are built on the same node, the perforce clients will probably still interfere with each other. If I get a chance, I will test that.

          torbent added a comment - - edited

          I suspect the mentioned changes to PerforceMailResolver has caused it to not
          work anymore - please see JENKINS-4933.

          torbent added a comment - - edited I suspect the mentioned changes to PerforceMailResolver has caused it to not work anymore - please see JENKINS-4933 .

          Rob Petti added a comment -

          Due to the complexities of dealing with multiconfiguration builds, I've changed the behavior of the plugin to always force sync when performing a matrix build. This eliminates the need of having a different client workspace for each and every combination, while still allowing matrix builds to work correctly.

          Rob Petti added a comment - Due to the complexities of dealing with multiconfiguration builds, I've changed the behavior of the plugin to always force sync when performing a matrix build. This eliminates the need of having a different client workspace for each and every combination, while still allowing matrix builds to work correctly.

            Unassigned Unassigned
            kirkwylie kirkwylie
            Votes:
            2 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: