Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-10592

Issues with P4 counters not working correctly in matrix builds

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • p4-plugin
    • None
    • Jenkins 1.424
      Perforce plugin 1.3.0

      Hi,

      I'm having problems getting the Perforce plugin counter system to work properly. This is my setup:

      JobA (matrix project): 
	P4 Counter = TOMLABEL
	Update counter to most recent changeset = ticked
	Build other projects = JobB

      	JobB (free-style project):
	P4 Counter = TOMLABEL
	Update counter to most recent changeset = not ticked

      So when I make a change in Perforce JobA is triggered. If I check the Console Output for the parent job it contains:

      Updating counter TOMLABEL to 140403
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMLABEL 140403

      Each element of JobA's matrix then builds. In the Console output for those builds I see a lot of this:

      [Win32] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Tools/program.exe@140403

      which is what I expect to see, ie. it's syncing to my counter.

      In between JobA finishing and JobB starting I check in another file which means the latest changelist Perforce knows about is now 140404.

      So JobA finally finishes building, succeeds and triggers JobB.

      All JobB does is print out the current value of TOMLABEL. So if I check the Console Output I would expect to see the value 140403. Unfortunately I don't. I get the value 140404 back.

      [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_External
18:24:53  Last build changeset: 140392
18:24:53  [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMLABEL
18:24:53  [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //Jenkins_External/...@140393,@140404
18:24:53  [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" describe -s 140404
18:24:53  [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -G where //personal/tfields/Live/Engine/External/NvTriStrip/NvTriStrip.cpp
18:24:53  [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" describe -s 140403
18:24:53  [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -G where //personal/tfields/Live/Engine/Core/Engine.cpp
18:24:54  Buildfile: C:\JCI\jobs\External\workspace\build-jci.xml
18:24:54  
18:24:54  PrintInfo:
18:24:54       [exec] TOMLABEL = 140404

      Any ideas why this is happening. It seems like a bug to me.

      Regards,
      Tom.

      – Posted from Bugbox for Android

        1. PerforceSCM.java
          83 kB
        2. PerforceSCM.java
          83 kB

          [JENKINS-10592] Issues with P4 counters not working correctly in matrix builds

          Thomas Fields added a comment -

          I've attached a possible fix for this issue. Can you check it please?

          Regards,
          Tom.

          Thomas Fields added a comment - I've attached a possible fix for this issue. Can you check it please? Regards, Tom.

          Rob Petti added a comment -

          I don't see the reason for disabling the counter for sub jobs so long as each sub job is syncing to the same changeset. What issues did you run into when you tested my fix?

          Rob Petti added a comment - I don't see the reason for disabling the counter for sub jobs so long as each sub job is syncing to the same changeset. What issues did you run into when you tested my fix?

          Thomas Fields added a comment -

          When testing your fix the P4 counter was being updated in the sub job when it shouldn't have been.

          JobA setup:
          Polling time is * * * * *.
          Build action is to call a Ant script containing: <sleep seconds="90"/>

          1) I check a file in to trigger a build of JobA (parent):

          Started by an SCM change
Using master perforce client: Jenkins_test
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test
Last build changeset: 141384
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter change
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141385,@141395
Sync'ing workspace to changelist 141395 (forcing sync of unchanged files).
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141395
Sync complete, took 654 ms
Updating counter TOMTOMTEST to 141395
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141395
Triggering CL,Dt,R
Triggering SNC,Dt,Q
Finished: SUCCESS

          2) CL,Dt,R sub job builds:

          Started by upstream project "test" build number 32
Using master perforce client: Jenkins_test-C-CL-N-Dt-P-R
[R] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-CL-N-Dt-P-R
Last build changeset: 141384
[R] $ "C:\\Program Files\\Perforce\\p4.exe" counter change
Sync'ing workspace to changelist 141395 (forcing sync of unchanged files).
[R] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141395
Sync complete, took 710 ms
Updating counter TOMTOMTEST to 141395
[R] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141395
[R] $ cmd.exe /C '"ant.bat -file buildTest.xml -DP=R -DC=CL -DN=Dt delay3 && exit %%ERRORLEVEL%%"'

delay3:

BUILD SUCCESSFUL
Total time: 1 minute 30 seconds
Finished: SUCCESS

          3) As soon as the CL,Dt,R sub job builds I check in another file. JobA will be added to the job queue but not start building until after the original JobA build has completed because I've only got 1 build executor thread.

          4) SNC,Dt,Q sub job builds:

          Started by upstream project "test" build number 32
Using master perforce client: Jenkins_test-C-SNC-N-Dt-P-Q
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-SNC-N-Dt-P-Q
Last build changeset: 141384
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter change
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141396,@141398
Sync'ing workspace to changelist 141398 (forcing sync of unchanged files).
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141398
Sync complete, took 638 ms
Updating counter TOMTOMTEST to 141398
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141398
[Q] $ cmd.exe /C '"ant.bat -file buildTest.xml -DP=Q -DC=SNC -DN=Dt delay3 && exit %%ERRORLEVEL%%"'

delay3:

BUILD SUCCESSFUL
Total time: 1 minute 30 seconds
Finished: SUCCESS

          In step 4 why is the counter being updated? It should still be 141395 shouldn't it?

          Regards,
          Tom.

          Thomas Fields added a comment - When testing your fix the P4 counter was being updated in the sub job when it shouldn't have been. JobA setup: Polling time is * * * * *. Build action is to call a Ant script containing: <sleep seconds="90"/> 1) I check a file in to trigger a build of JobA (parent): Started by an SCM change Using master perforce client: Jenkins_test [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test Last build changeset: 141384 [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter change [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141385,@141395 Sync'ing workspace to changelist 141395 (forcing sync of unchanged files). [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141395 Sync complete, took 654 ms Updating counter TOMTOMTEST to 141395 [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141395 Triggering CL,Dt,R Triggering SNC,Dt,Q Finished: SUCCESS 2) CL,Dt,R sub job builds: Started by upstream project "test" build number 32 Using master perforce client: Jenkins_test-C-CL-N-Dt-P-R [R] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-CL-N-Dt-P-R Last build changeset: 141384 [R] $ "C:\\Program Files\\Perforce\\p4.exe" counter change Sync'ing workspace to changelist 141395 (forcing sync of unchanged files). [R] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141395 Sync complete, took 710 ms Updating counter TOMTOMTEST to 141395 [R] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141395 [R] $ cmd.exe /C ' "ant.bat -file buildTest.xml -DP=R -DC=CL -DN=Dt delay3 && exit %%ERRORLEVEL%%" ' delay3: BUILD SUCCESSFUL Total time: 1 minute 30 seconds Finished: SUCCESS 3) As soon as the CL,Dt,R sub job builds I check in another file. JobA will be added to the job queue but not start building until after the original JobA build has completed because I've only got 1 build executor thread. 4) SNC,Dt,Q sub job builds: Started by upstream project "test" build number 32 Using master perforce client: Jenkins_test-C-SNC-N-Dt-P-Q [Q] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-SNC-N-Dt-P-Q Last build changeset: 141384 [Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter change [Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141396,@141398 Sync'ing workspace to changelist 141398 (forcing sync of unchanged files). [Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141398 Sync complete, took 638 ms Updating counter TOMTOMTEST to 141398 [Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141398 [Q] $ cmd.exe /C ' "ant.bat -file buildTest.xml -DP=Q -DC=SNC -DN=Dt delay3 && exit %%ERRORLEVEL%%" ' delay3: BUILD SUCCESSFUL Total time: 1 minute 30 seconds Finished: SUCCESS In step 4 why is the counter being updated? It should still be 141395 shouldn't it? Regards, Tom.

          Rob Petti added a comment -

          The fix was never meant to prevent sub jobs from updating the counter. In most cases, the only thing that will be getting executed will be the sub jobs (unless you are running a touchstone build first). We will always want the counter to be updated.

          The fix was supposed to sync every sub job to the same changeset.

          I will take another look at it when I have the time.

          Rob Petti added a comment - The fix was never meant to prevent sub jobs from updating the counter. In most cases, the only thing that will be getting executed will be the sub jobs (unless you are running a touchstone build first). We will always want the counter to be updated. The fix was supposed to sync every sub job to the same changeset. I will take another look at it when I have the time.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Rob Petti
          Path:
          src/main/java/hudson/plugins/perforce/PerforceSCM.java
          http://jenkins-ci.org/commit/perforce-plugin/03595c0e90be9daa53aa27a725de0b368992804c
          Log:
          JENKINS-10592 adding more logging for matrix builds, and fixing incorrect behavior

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Rob Petti Path: src/main/java/hudson/plugins/perforce/PerforceSCM.java http://jenkins-ci.org/commit/perforce-plugin/03595c0e90be9daa53aa27a725de0b368992804c Log: JENKINS-10592 adding more logging for matrix builds, and fixing incorrect behavior

          dogfood added a comment -

          Integrated in plugins_perforce #126
          JENKINS-10592 adding more logging for matrix builds, and fixing incorrect behavior

          Rob Petti :
          Files :

          • src/main/java/hudson/plugins/perforce/PerforceSCM.java

          dogfood added a comment - Integrated in plugins_perforce #126 JENKINS-10592 adding more logging for matrix builds, and fixing incorrect behavior Rob Petti : Files : src/main/java/hudson/plugins/perforce/PerforceSCM.java

          Rob Petti added a comment -

          Snapshot has been updated with a potential fix and some additional logging. Can you give it another try, please?

          Rob Petti added a comment - Snapshot has been updated with a potential fix and some additional logging. Can you give it another try, please?

          Thomas Fields added a comment -

          Hi Rob,

          Thanks for the new update. That looks a lot better so I now get:

          1) I check a file in to trigger a build of JobA (parent):

          Started by an SCM change
Using master perforce client: Jenkins_test
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test
Last build changeset: 141604
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter change
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141605,@141611
Sync'ing workspace to changelist 141611 (forcing sync of unchanged files).
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141611
Sync complete, took 1087 ms
Updating counter TOMTOMTEST to 141611
[workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141611
Triggering SNC,Dt,Q
Triggering CL,Dt,R
Finished: SUCCESS

          2) CL,Dt,R sub job builds:

          Started by upstream project "test" build number 36
Using master perforce client: Jenkins_test-C-CL-N-Dt-P-R
[R] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-CL-N-Dt-P-R
Last build changeset: 141604
[R] $ "C:\\Program Files\\Perforce\\p4.exe" counter change
This is a matrix run, trying to use change number from parent/siblings...
Latest change from parent is: 0
[R] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141605,@141611
Sync'ing workspace to changelist 141611 (forcing sync of unchanged files).
[R] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141611
Sync complete, took 815 ms
Updating counter TOMTOMTEST to 141611
[R] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141611
[R] $ cmd.exe /C '"ant.bat -file buildTest.xml -DP=R -DC=CL -DN=Dt delay3 && exit %%ERRORLEVEL%%"'

delay3:

BUILD SUCCESSFUL
Total time: 1 minute 30 seconds
Finished: SUCCESS

          3) As soon as the CL,Dt,R sub job builds I check in another file. JobA will be added to the job queue but not start building until after the original JobA build has completed because I've only got 1 build executor thread.

          4) SNC,Dt,Q sub job builds:

          Started by upstream project "test" build number 36
Using master perforce client: Jenkins_test-C-SNC-N-Dt-P-Q
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-SNC-N-Dt-P-Q
Last build changeset: 141604
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter change
This is a matrix run, trying to use change number from parent/siblings...
Latest change from parent is: 0
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141605,@141611
Sync'ing workspace to changelist 141611 (forcing sync of unchanged files).
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141611
Sync complete, took 1064 ms
Updating counter TOMTOMTEST to 141611
[Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141611
[Q] $ cmd.exe /C '"ant.bat -file buildTest.xml -DP=Q -DC=SNC -DN=Dt delay3 && exit %%ERRORLEVEL%%"'

delay3:

BUILD SUCCESSFUL
Total time: 1 minute 30 seconds
Finished: SUCCESS

          Thanks a lot.

          Tom.

          Thomas Fields added a comment - Hi Rob, Thanks for the new update. That looks a lot better so I now get: 1) I check a file in to trigger a build of JobA (parent): Started by an SCM change Using master perforce client: Jenkins_test [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test Last build changeset: 141604 [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter change [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141605,@141611 Sync'ing workspace to changelist 141611 (forcing sync of unchanged files). [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141611 Sync complete, took 1087 ms Updating counter TOMTOMTEST to 141611 [workspace] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141611 Triggering SNC,Dt,Q Triggering CL,Dt,R Finished: SUCCESS 2) CL,Dt,R sub job builds: Started by upstream project "test" build number 36 Using master perforce client: Jenkins_test-C-CL-N-Dt-P-R [R] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-CL-N-Dt-P-R Last build changeset: 141604 [R] $ "C:\\Program Files\\Perforce\\p4.exe" counter change This is a matrix run, trying to use change number from parent/siblings... Latest change from parent is: 0 [R] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141605,@141611 Sync'ing workspace to changelist 141611 (forcing sync of unchanged files). [R] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141611 Sync complete, took 815 ms Updating counter TOMTOMTEST to 141611 [R] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141611 [R] $ cmd.exe /C ' "ant.bat -file buildTest.xml -DP=R -DC=CL -DN=Dt delay3 && exit %%ERRORLEVEL%%" ' delay3: BUILD SUCCESSFUL Total time: 1 minute 30 seconds Finished: SUCCESS 3) As soon as the CL,Dt,R sub job builds I check in another file. JobA will be added to the job queue but not start building until after the original JobA build has completed because I've only got 1 build executor thread. 4) SNC,Dt,Q sub job builds: Started by upstream project "test" build number 36 Using master perforce client: Jenkins_test-C-SNC-N-Dt-P-Q [Q] $ "C:\\Program Files\\Perforce\\p4.exe" workspace -o Jenkins_test-C-SNC-N-Dt-P-Q Last build changeset: 141604 [Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter change This is a matrix run, trying to use change number from parent/siblings... Latest change from parent is: 0 [Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s changes //personal/tfields/Live/Engine/Build/...@141605,@141611 Sync'ing workspace to changelist 141611 (forcing sync of unchanged files). [Q] $ "C:\\Program Files\\Perforce\\p4.exe" -s sync -f //personal/tfields/Live/Engine/Build/...@141611 Sync complete, took 1064 ms Updating counter TOMTOMTEST to 141611 [Q] $ "C:\\Program Files\\Perforce\\p4.exe" counter TOMTOMTEST 141611 [Q] $ cmd.exe /C ' "ant.bat -file buildTest.xml -DP=Q -DC=SNC -DN=Dt delay3 && exit %%ERRORLEVEL%%" ' delay3: BUILD SUCCESSFUL Total time: 1 minute 30 seconds Finished: SUCCESS Thanks a lot. Tom.

          SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Rob Petti
          Path:
          src/main/java/hudson/plugins/perforce/PerforceSCM.java
          http://jenkins-ci.org/commit/perforce-plugin/4850a5aff2fd99eed237b5a5c35dc8b0218210f0
          Log:
          JENKINS-10592 fixing cosmetic log problem

          the log would always output '0' for the changeset number from the parent job when running a matrix build. it'll should now output the value that's actually used

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Rob Petti Path: src/main/java/hudson/plugins/perforce/PerforceSCM.java http://jenkins-ci.org/commit/perforce-plugin/4850a5aff2fd99eed237b5a5c35dc8b0218210f0 Log: JENKINS-10592 fixing cosmetic log problem the log would always output '0' for the changeset number from the parent job when running a matrix build. it'll should now output the value that's actually used

          dogfood added a comment -

          Integrated in plugins_perforce #127
          JENKINS-10592 fixing cosmetic log problem

          Rob Petti :
          Files :

          • src/main/java/hudson/plugins/perforce/PerforceSCM.java

          dogfood added a comment - Integrated in plugins_perforce #127 JENKINS-10592 fixing cosmetic log problem Rob Petti : Files : src/main/java/hudson/plugins/perforce/PerforceSCM.java

            rpetti Rob Petti
            tfields Thomas Fields
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: