Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50340

P4 Plugin : Changes / Recent Changes / changelogs empty in multi-node environment due to syncID calculation

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • p4-plugin
    • None
    • Jenkins 2.89.3
      P4 Plugin 1.8.5
      Declarative Pipelines

      Recently we converted our Jenkins jobs to Declarative Pipelines. In our environment there is 1 Jenkins master and 16 slaves.

      However, Changes / Recent Changes suddenly was empty for most builds (and as a consequence, failure notifications were no longer e-mailed etc.).

      Investigation showed that the issue lies in the way the "syncID" is calculated.

      In order to calculate the changes between two checkouts in 2 builds, the checkout in the new build needs to look up which changelist was checked out in the previous build. Since a single build could potentially contain several different checkouts, a checkout step in a build job internally is assigned a "syncID" which identifies the checkout step within the build.

      Currently this syncID is calculated by using the workspace name as input, and removing occurrences of $NODE_NAME and $EXECUTOR_NUMBER.

      These variables are often used to generate a workspace name that is sufficiently unique for Perforce to deal with : Perforce does not allow you to create 2 workspaces with the same name on different machines, so $NODE_NAME is used here to ensure that if the same job gets executed on different nodes, the workspaces have different names.

      Since the syncID should depend on the job and on the checkout step, but not on the node or executor number, the plugin removes occurrences of "$NODE_NAME" in the workspace name declaration of the checkout step, but this obviously only works in case the name declaration uses these variables explicitly at that point.

      If for example the script calculates the workspace name elsewhere (e.g. to ensure workspace name is only calculated once in the script, for maintainability), and uses that result, the plugin cannot detect the fact that in the workspace name string, the node name is present.

      Therefore, in many cases, the current syncID calculation will break.

      An example may make this clearer.

      The following fragment should work : workspace.name is defined as 'job_$NODE_NAME', and this literal is passed onto the plugin, which strips out the $NODE_NAME from the literal which is passed as workspace name. Any other environment variables are resolved afterwards.

       

       

      pipeline {
       agent { node { label '!master' } }
      
       stages {
         stage('P4') {
           steps {
             checkout perforce(
               [...]
               workspace: manualSpec(
                 [...]
                 name: 'job_$NODE_NAME',
                 pinHost: false,
                 spec: clientSpec(
                   [...]
                   view: "//myPath/... //job_${NODE_NAME}/..."
                 ),
                 syncID: "${JOB_BASE_NAME}"
               )
             )
           }
         }
         [...]

       

      The following fragment does the same thing, but more maintainable (from the script point of view), by calculating P4CLIENT once, and reusing it. That way, if the workspace name must be changed, there's only one place in the script where to calculate it. Unfortunately, the result is that the plugin now gets the calculated name as value for the workspace name, so it cannot strip out $NODE_NAME, and the resulting syncID now depends on the node :

       

       

      pipeline {
       agent { node { label '!master' } }
      
       environment {
         P4CLIENT="job_${NODE_NAME}"
       }
      
       stages {
         stage('P4') {
           steps {
             checkout perforce(
               [...]
               workspace: manualSpec(
                 [...]
                 name: "$env.P4CLIENT",
                 pinHost: false,
                 spec: clientSpec(
                   [...]
                   view: "//myPath/... //${env.P4CLIENT}/..."
                 )
               )
             )
           }
         }
         [...]

       

       

          [JENKINS-50340] P4 Plugin : Changes / Recent Changes / changelogs empty in multi-node environment due to syncID calculation

          phrx, thanks for raising this issue.

          W Basu Perforce added a comment - phrx , thanks for raising this issue.

          Philip Arickx added a comment -

          I forgot to add information on the workaround - which is simple enough : explicitly define the syncID yourself for each workspace in a p4 sync operation.

          Any static String will do as long as it is unique within the Jenkins instance, and doesn't depend on the node name, executor number or build number (in other words, it has to remain constant across builds).

          Generally speaking, $JOB_BASE_NAME should work quite well as basis for the value (provided you don't have jobs with the same name in different folders - which may be allowed by Jenkins), but when the job is renamed there may be a single build where the changes will be empty. Subsequent builds will have proper changelists again.

          $JOB_NAME might work too, but will produce a build with empty changes when the job is renamed or moved to a different folder. Also, since the String is a path, you might want to clean up special characters such as slashes to be safe, although I'm not running into any issues ATM.

          This translates into the checkout step as follows : (in this case, using only $JOB_BASE_NAME, there's only a single checkout in this script)

          checkout perforce(
            credential: 'myCreds',
            populate: autoClean( ... ),
            workspace: manualSpec(
              charset: 'utf8',
              name: "$env.P4CLIENT",
              pinHost: false,
              spec: clientSpec( ... ),
              syncID: "${JOB_BASE_NAME}"
            )
          )

           

          Philip Arickx added a comment - I forgot to add information on the workaround - which is simple enough : explicitly define the syncID yourself for each workspace in a p4 sync operation. Any static String will do as long as it is unique within the Jenkins instance, and doesn't depend on the node name, executor number or build number (in other words, it has to remain constant across builds). Generally speaking, $JOB_BASE_NAME should work quite well as basis for the value (provided you don't have jobs with the same name in different folders - which may be allowed by Jenkins), but when the job is renamed there may be a single build where the changes will be empty. Subsequent builds will have proper changelists again. $JOB_NAME might work too, but will produce a build with empty changes when the job is renamed or moved to a different folder. Also, since the String is a path, you might want to clean up special characters such as slashes to be safe, although I'm not running into any issues ATM. This translates into the checkout step as follows : (in this case, using only $JOB_BASE_NAME, there's only a single checkout in this script) checkout perforce( credential: 'myCreds',   populate: autoClean( ... ),   workspace: manualSpec(     charset: 'utf8',     name: "$env.P4CLIENT",     pinHost: false,     spec: clientSpec( ... ),     syncID: "${JOB_BASE_NAME}"   ) )  

            p4paul Paul Allen
            phrx Philip Arickx
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: