Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65048

Option to resolve text files as part of p4publish step

      In my jenkins job after build and test I do npm package version update and do publish using p4publish. All this process takes 5-7 minutes. But p4publish reverts the changes published during this 5-7 minutes and publishes the version upgrades.

       

      {{p4publish(
      credential: 'jenkins',
      publish: submit(
      delete: false,
      modtime: false,
      onlyOnSuccess: false,
      reopen: false,
      description: 'update package version',
      ),
      workspace: streamSpec(
      pinHost: false,
      charset: 'utf8',
      format: 'jenkins-${NODE_NAME}${JOB_NAME}-agent${EXECUTOR_NUMBER}',
      streamName: '//project/mystream'
      )
      )}}

      Is there a way I can avoid reverting the changes and possibly sync the changes and resolve any conflicts before publishing?

          [JENKINS-65048] Option to resolve text files as part of p4publish step

          Karl Wirth added a comment -

          Hi ramganesht - Are you saying that P4Publish overwrites anything submitted in the past 7 minutes by other jobs or that is is reveryting some of the work that was done in this job?

          Karl Wirth added a comment - Hi ramganesht - Are you saying that P4Publish overwrites anything submitted in the past 7 minutes by other jobs or that is is reveryting some of the work that was done in this job?

          Ram added a comment -

          Hi p4karl, p4publish overwrites anything thats submitted in past 7 minutes(i.e., the time between the job polling is done and p4publish)

          Ram added a comment - Hi  p4karl , p4publish overwrites anything thats submitted in past 7 minutes(i.e., the time between the job polling is done and p4publish)

          Karl Wirth added a comment -

          HI ramganesht - Sorry I missed this. Need to check why I'm not getting email updates. :-/

          I will repro this here and  let you know next steps.

          Karl Wirth added a comment - HI ramganesht - Sorry I missed this. Need to check why I'm not getting email updates. :-/ I will repro this here and  let you know next steps.

          Karl Wirth added a comment -

          Hi ramganesht.

          Thanks. I've easily reproduced this and modified this to an enhancement request.

          As you mention at the moment there is no ability to resolve files on publish. It just assumes that is only one build running at a time and this job is the only thing publishing to it or it doesnt matter if you overwrite For a build artifact such as an executable binary or a ray traced video that is valid. It doesnt matter if the binary created in the second job overwrites the first because you cant merge the content.

           

          For text artifacts where the build is adding something to them P4Publish would need to have a resolve -am then resolve -at action.

           

          The two workarounds I can think of are:

          (1) Use P4Groovy to do the resolve and publish. This may or may not work and you may hit unexpected behavior as mentioned in JENKINS-59888 so you will probably need a lot of defensive coding to pickup errors and act on them. More on P4Groovy cab be found in:

              https://www.perforce.com/manuals/jenkins/Content/P4Jenkins/p4groovy.html?Highlight=P4Groovy

          (2) Similar to above use a script/program outside P4Jenkins to do the publish. For example using the same workspace perform p4 edit, p4 sync, p4 resolve and p4 submit operations in a batch file or shell script

           

          Reproduction steps:

          (1) Add a file called 'test.txt' to your depot path.

          (2) Create a freestyle job that syncs files to a linux build host.

          (3) In the Freestyle job have a build step that runs the following shell code:

          chmod +w test.txt
          date >> test.txt
          sleep 120
          

          (4) Include a publish step to publish the contents of the workspace.

          (5) Run the build job.

          (6) During the 120 second pause, submit two new versions of test.txt outside of Jenkins that contain recognizable edits.

          (7) When the build job completes look at the latest contents of test.txt. It will not contain the new edits you made in step (6).

           

          Karl Wirth added a comment - Hi ramganesht . Thanks. I've easily reproduced this and modified this to an enhancement request. As you mention at the moment there is no ability to resolve files on publish. It just assumes that is only one build running at a time and this job is the only thing publishing to it or it doesnt matter if you overwrite For a build artifact such as an executable binary or a ray traced video that is valid. It doesnt matter if the binary created in the second job overwrites the first because you cant merge the content.   For text artifacts where the build is adding something to them P4Publish would need to have a resolve -am then resolve -at action.   The two workarounds I can think of are: (1) Use P4Groovy to do the resolve and publish. This may or may not work and you may hit unexpected behavior as mentioned in JENKINS-59888 so you will probably need a lot of defensive coding to pickup errors and act on them. More on P4Groovy cab be found in:     https://www.perforce.com/manuals/jenkins/Content/P4Jenkins/p4groovy.html?Highlight=P4Groovy (2) Similar to above use a script/program outside P4Jenkins to do the publish. For example using the same workspace perform p4 edit, p4 sync, p4 resolve and p4 submit operations in a batch file or shell script   Reproduction steps: (1) Add a file called 'test.txt' to your depot path. (2) Create a freestyle job that syncs files to a linux build host. (3) In the Freestyle job have a build step that runs the following shell code: chmod +w test.txt date >> test.txt sleep 120 (4) Include a publish step to publish the contents of the workspace. (5) Run the build job. (6) During the 120 second pause, submit two new versions of test.txt outside of Jenkins that contain recognizable edits. (7) When the build job completes look at the latest contents of test.txt. It will not contain the new edits you made in step (6).  

          Ram added a comment -

          thanks p4karl, do you have have an ETA for this?

          Also, do you have an example for the work arounds either using P4Groovy or script?

          Ram added a comment - thanks p4karl , do you have have an ETA for this? Also, do you have an example for the work arounds either using P4Groovy or script?

          Karl Wirth added a comment -

          Hi ramganesht - I cant provide an ETA sorry.

          I have not tested it but possibly something like this may work.

           

          echo("P4 SYNC:" + p4.run('sync').toString())
          echo("P4 RESOLVE: " + p4.run('resolve','-am','//depot/target/...').toString())
          echo("P4 RESOLVED: " + p4.run('resolved').toString())

          You need to sync the latest version of the file to setup the resolve and then perform the resolve.

          IMPORTANT NOTE - "resolve -am" only works on the most basic cases. If there is any form of conflict the resolve will fail and you will always need a human to make the resolve descision of which lines to keep and which to discard which is why build systems should not really be performing a resolve.

           

          Karl Wirth added a comment - Hi ramganesht - I cant provide an ETA sorry. I have not tested it but possibly something like this may work.   echo( "P4 SYNC:" + p4.run( 'sync' ).toString()) echo( "P4 RESOLVE: " + p4.run( 'resolve' , '-am' , ' //depot/target/...' ).toString()) echo( "P4 RESOLVED: " + p4.run( 'resolved' ).toString()) You need to sync the latest version of the file to setup the resolve and then perform the resolve. IMPORTANT NOTE - "resolve -am" only works on the most basic cases. If there is any form of conflict the resolve will fail and you will always need a human to make the resolve descision of which lines to keep and which to discard which is why build systems should not really be performing a resolve.  

            naveenboni Naveen Boni
            ramganesht Ram
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: