Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43378

Exceptions when performing post build Perforce tasks on concurrent builds

    • Icon: Bug Bug
    • Resolution: Postponed
    • Icon: Critical Critical
    • p4-plugin
    • Jenkins 2.53
      P4 plugin 1.5.1

      I have a job that can perform concurrent builds.

      As post-build steps I have both Perforce: Publish assets and Perforce: Cleanup.

      The Publish assets action on the concurrent build results in:
      P4: Unable to setup workspace: com.perforce.p4java.exception.RequestException: Error in client specification.
      Embedded special characters (*, %%, #, @) not allowed in 'C:\Program Files (x86)\Jenkins\workspace\test-project-assets@2'.

      ERROR: Step ‘Perforce: Publish assets’ failed: P4: Unable to setup workspace: com.perforce.p4java.exception.RequestException: Error in client specification.
      Embedded special characters (*, %%, #, @) not allowed in 'C:\Program Files (x86)\Jenkins\workspace\test-project-assets@2'.
       

      If I remove the Publish assets step and only perform the Cleanup step I get:
      P4: Template workspace not found: jenkins-master-test-project-assets_TEST
      P4: Unable to setup workspace: hudson.AbortException: P4: Undefined workspace: jenkins-master-test-project-assets_TEST.clone2
      ERROR: Step ‘Perforce: Cleanup’ failed: P4: Unable to setup workspace: hudson.AbortException: P4: Undefined workspace: jenkins-master-test-project-assets_TEST.clone2
       

      The exceptions do NOT occur on the first concurrent build.

          [JENKINS-43378] Exceptions when performing post build Perforce tasks on concurrent builds

          Paul Allen added a comment -

          Please open a support case at support@perforce.com reference the JIRA issue JENKINS-43378 and our support staff can help you upload the full log or any confidential data relating to the issue.

          I have not been able to get the .clone2.clone2 issue to occur, but it could be a timing issue.

          Paul Allen added a comment - Please open a support case at support@perforce.com  reference the JIRA issue  JENKINS-43378 and our support staff can help you upload the full log or any confidential data relating to the issue. I have not been able to get the .clone2.clone2 issue to occur, but it could be a timing issue.

          Paul Allen added a comment -

          I was looking at the logs and confused to see that the clean up happens first?  In addition I do not recognise the line:

          [WS-CLEANUP] Deleting project workspace...

          Is this a plugin or advanced Jenkins option?

          I thought you were using the 'Perforce:Cleanup' step as a Post Build action.

          Paul Allen added a comment - I was looking at the logs and confused to see that the clean up happens first?  In addition I do not recognise the line: [WS-CLEANUP] Deleting project workspace... Is this a plugin or advanced Jenkins option? I thought you were using the 'Perforce:Cleanup' step as a Post Build action.

          David Hartley added a comment -

          The 
          [WS-CLEANUP] Deleting project workspace...
          message is the vanilla Jenkins workspace cleanup that deletes the local files.

          This step occurs at both the start and end because I have "Delete workspace before build starts" ticked in the "Build Environment" section and also "Delete workspace when build is done" as the very last step in the "Post-build Actions".

          David Hartley added a comment - The  [WS-CLEANUP] Deleting project workspace... message is the vanilla Jenkins workspace cleanup that deletes the local files. This step occurs at both the start and end because I have "Delete workspace before build starts" ticked in the "Build Environment" section and also "Delete workspace when build is done" as the very last step in the "Post-build Actions".

          Paul Allen added a comment -

          It looks like you are using the ws-cleanup plugin (not included by default).

          I have not tested this plugin's compatibility with the p4-plugin, please try remove the 'before build' cleanup.

          Paul Allen added a comment - It looks like you are using the ws-cleanup  plugin (not included by default). I have not tested this plugin's compatibility with the p4-plugin, please try remove the 'before build' cleanup.

          Paul Allen added a comment -

          Sorry it is included by default, but I still can't find it for pipeline only freestyle?

          Paul Allen added a comment - Sorry it is included by default, but I still can't find it for pipeline only freestyle?

          David Hartley added a comment -

          This is a freestyle job.

          I've un-ticked every unnecessary box and removed all build and post-build steps to reduce the possibility of any conflicts.

          When I run a single build I'm now left with a single dds image in the asset-generation workspace folder.

          When I run 2 builds, 2 folders are created but only 1 of them contains a dds image.

           

          In my test I have echoed the filename it should be mapping for each build and this is correct. But if I look at the mapping of the workspace in P4V both of the workspaces are mapped to the same file.

           

          //JenkinsTest/Games/test-project/assets/bottom_game/barrel.dds //jenkins-master-asset-generation-populate/barrel.dds

          //JenkinsTest/Games/test-project/assets/bottom_game/barrel.dds //jenkins-master-asset-generation-populate.clone2/barrel.dds

           

          which it generated from:

           

          ${DEPOT_DIR}/${FILENAME} //jenkins-${NODE_NAME}-${JOB_NAME}-populate/${FILENAME}

           

          ${FILENAME} was backdrop.dds on build 1 and barrel.dds on build 2.

           

          David Hartley added a comment - This is a freestyle job. I've un-ticked every unnecessary box and removed all build and post-build steps to reduce the possibility of any conflicts. When I run a single build I'm now left with a single dds image in the asset-generation workspace folder. When I run 2 builds, 2 folders are created but only 1 of them contains a dds image.   In my test I have echoed the filename it should be mapping for each build and this is correct. But if I look at the mapping of the workspace in P4V both of the workspaces are mapped to the same file.   //JenkinsTest/Games/test-project/assets/bottom_game/barrel.dds //jenkins-master-asset-generation-populate/barrel.dds //JenkinsTest/Games/test-project/assets/bottom_game/barrel.dds //jenkins-master-asset-generation-populate.clone2/barrel.dds   which it generated from:   ${DEPOT_DIR}/${FILENAME} //jenkins-${NODE_NAME}-${JOB_NAME}-populate/${FILENAME}   ${FILENAME} was backdrop.dds on build 1 and barrel.dds on build 2.  

          Paul Allen added a comment - - edited

          When the workspace is cloned during a concurrent build the 'expanded' view is copied from the parent.  So I would have expected:

          Executor 1:  workspace(ws), view(//depot/A.dds //ws/A.dds)

          Executor 2:  workspace(ws.clone2), view(//depot/A.dds //ws/A.dds)

          You might find removing ${FILENAME} and using a view of ${DEPOT_DIR}/....dds might work better.  Your use of ${FILENAME} is an interesting variation I had not considered. 

          Paul Allen added a comment - - edited When the workspace is cloned during a concurrent build the 'expanded' view is copied from the parent.  So I would have expected: Executor 1:  workspace(ws), view(//depot/A.dds //ws/A.dds) Executor 2:  workspace(ws.clone2), view(//depot/A.dds //ws/A.dds) You might find removing ${FILENAME} and using a view of ${DEPOT_DIR}/....dds might work better.  Your use of ${FILENAME} is an interesting variation I had not considered. 

          David Hartley added a comment -

          Just done some testing and if I change my view mapping to:

          ${DEPOT_DIR}/....dds

          or 

          ${DEPOT_DIR}/...

          build 1 does not download any files but build 2 downloads correctly. I'm not using a specific file anywhere now so both instances should be identical.

          David Hartley added a comment - Just done some testing and if I change my view mapping to: ${DEPOT_DIR}/....dds or  ${DEPOT_DIR}/... build 1 does not download any files but build 2 downloads correctly. I'm not using a specific file anywhere now so both instances should be identical.

          David Hartley added a comment -

          Issue is currently down to the workspace directory location being null.

          Should be c:/program files/jenkins/workspace/etc/

           
          P4 Task: reverting all pending and shelved revisions.
          ... p4 revert null/... +
           
          ... rm [abandoned files]
          duration: (62ms)

          P4 Task: cleaning workspace to match have list.
          ... p4 reconcile -w -f null/... +
           
          duration: (85ms)

          P4 Task: syncing files at change: 775
          ... p4 sync -q null/...@775 +
           
          duration: (59ms)

          David Hartley added a comment - Issue is currently down to the workspace directory location being null. Should be c:/program files/jenkins/workspace/etc/   P4 Task: reverting all pending and shelved revisions. ... p4 revert null/... +   ... rm [abandoned files] duration: (62ms) P4 Task: cleaning workspace to match have list. ... p4 reconcile -w -f null/... +   duration: (85ms) P4 Task: syncing files at change: 775 ... p4 sync -q null/...@775 +   duration: (59ms)

          Paul Allen added a comment -

          The concurrent logic has changed since 1.6.x.  Issue Postponed.

          (Please open a new issue and reference this one if required)

          Paul Allen added a comment - The concurrent logic has changed since 1.6.x.  Issue Postponed. (Please open a new issue and reference this one if required)

            p4paul Paul Allen
            d_hartley David Hartley
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: