Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70693

local workspace is used instead of external workspace

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • Jenkins 2.392 (also 2.390)
      pipeline 590.v6a_d052e5a_a_b_5
      Pipeline: Nodes and Processes Version 1234.v019404b_3832a

      This happens since updating some plugins today (27.02.2023).
      I have a feeling that it might be related to JENKINS-70528 (just a guess).

      For testing I used following pipeline snippled (the job's name is "work_ServerCheck"):

      def exwsAll = exwsAllocate diskPoolId: 'ArchivedBuilds', path: "unmerged/work_ServerCheck"
      exws  (exwsAll) {
          exwsPath=   pwd()    
          println "exwsPath="+exwsPath{}
          sh 'pwd'
      }

      This produces following output (I marked unexpected values red - they used to be equal to the green value in previous versions):
      10:02:42  Running in /media/Builds/unmerged/work_ServerCheck
      10:02:42  [Pipeline] {
      10:02:42  [Pipeline] pwd
      10:02:42  [Pipeline] echo
      10:02:42  exwsPath=/rsync/JenkinsJobs/workspace/work_ServerCheck
      10:02:42  [Pipeline] sh
      10:02:42  + pwd
      10:02:42  /rsync/JenkinsJobs/workspace/work_ServerCheck
      10:02:42  [Pipeline]
      10:02:42  [Pipeline] // exws
      10:02:42  [Pipeline] echo

          [JENKINS-70693] local workspace is used instead of external workspace

          Roman Zwi added a comment -

          I'm sorry, I don't have any knowledge about Jenkins plugin development.

          But I managed to get my jobs running without using this plugin, so I disabled it.

          Roman Zwi added a comment - I'm sorry, I don't have any knowledge about Jenkins plugin development. But I managed to get my jobs running without using this plugin, so I disabled it.

          Chris added a comment - - edited

          I have exactly the same issue on my side respectively in my Jenkins environmet. We have also updated some plugins and the Jenkins from version 2.375.2 to 2.375.3 on February 27th and noticed same results as Roman. Strangely a rollback of all updated plugins and the Jenkins version back to 2.375.2 did not help either.

          I was also able to reproduce the behavior by using the simple example from the GitHub README (same job and one node).

          Chris added a comment - - edited I have exactly the same issue on my side respectively in my Jenkins environmet. We have also updated some plugins and the Jenkins from version 2.375.2 to 2.375.3 on February 27th and noticed same results as Roman. Strangely a rollback of all updated plugins and the Jenkins version back to 2.375.2 did not help either. I was also able to reproduce the behavior by using the simple example from the GitHub README (same job and one node).

          Ian added a comment -

          We also began experiencing this issue on feburary 17th. We use kubernetes and nodes are ephemeral, so pulling the repo on one node is supposed to go to a shared external workspace, but instead only clones to the node's workspace and so follow-on node workspaces are empty, which completely breaks our builds.

          Ian added a comment - We also began experiencing this issue on feburary 17th. We use kubernetes and nodes are ephemeral, so pulling the repo on one node is supposed to go to a shared external workspace, but instead only clones to the node's workspace and so follow-on node workspaces are empty, which completely breaks our builds.

          Roman Zwi added a comment -

          jglick do you have any thoughts about my guess about a possible relation to JENKINS-70528?

          Roman Zwi added a comment - jglick do you have any thoughts about my guess about a possible relation to JENKINS-70528 ?

          Jesse Glick added a comment -

          I have heard of this plugin but am not familiar with the details. Offhand I would guess that the exws step needs to call a new FilePathDynamicContext.createContextualObject(FilePath, FlowNode) method to interoperate correctly with other steps.

          If anyone is interested in adopting this plugin, I would strongly recommend it be added to bom so that changes to critical dependencies such as Pipeline plugins get cross-tested against it. I am presuming that it does have some functional test coverage which can run without external dependencies (i.e., mvn test in a generic container will skip few if any tests), though it does not currently seem to have any tests that run pwd() or even sh so it is not obvious that a regression like this would even be flagged without more work.

          Jesse Glick added a comment - I have heard of this plugin but am not familiar with the details. Offhand I would guess that the exws step needs to call a new FilePathDynamicContext.createContextualObject(FilePath, FlowNode) method to interoperate correctly with other steps. If anyone is interested in adopting this plugin, I would strongly recommend it be added to bom so that changes to critical dependencies such as Pipeline plugins get cross-tested against it. I am presuming that it does have some functional test coverage which can run without external dependencies (i.e., mvn test in a generic container will skip few if any tests), though it does not currently seem to have any tests that run pwd() or even sh so it is not obvious that a regression like this would even be flagged without more work.

          Jesse Glick added a comment -

          I checked and ExwsExecution does call the now-deprecated overload. Fixing this in code would be straightforward (passing the FlowNode found in the step context) but depending on a current release of workflow-durable-task-step to access the new API would require more changes: currently it is on a very old Jenkins baseline dependency and an ancient parent POM. To work with new stuff the toolchains need to be refreshed. And, as above, to prove that the fix is effective might require at least a little new test coverage.

          I could probably make the deprecated overload claim its depth is some large number rather than zero, which might restore compatibility with exws in typical cases, but things would still be broken in some cases (such as dir inside exws) and changes here are pretty risky so I would rather recommend a proper fix.

          Another possible hotfix that would not require metadata changes to the plugin: calling the new overload via reflection when the dep is new enough. Could only be interactively verified (automated tests run against the declared dependency version).

          Jesse Glick added a comment - I checked and ExwsExecution does call the now-deprecated overload. Fixing this in code would be straightforward (passing the FlowNode found in the step context) but depending on a current release of workflow-durable-task-step to access the new API would require more changes: currently it is on a very old Jenkins baseline dependency and an ancient parent POM. To work with new stuff the toolchains need to be refreshed . And, as above, to prove that the fix is effective might require at least a little new test coverage. I could probably make the deprecated overload claim its depth is some large number rather than zero, which might restore compatibility with exws in typical cases, but things would still be broken in some cases (such as dir inside exws ) and changes here are pretty risky so I would rather recommend a proper fix. Another possible hotfix that would not require metadata changes to the plugin: calling the new overload via reflection when the dep is new enough. Could only be interactively verified (automated tests run against the declared dependency version).

          Jesse Glick added a comment -

          Jesse Glick added a comment - See if https://github.com/jenkinsci/external-workspace-manager-plugin/pull/76 helps.

          I've released the version 1.3.0 that should fix this issue.

          Thank you jglick and timja for the quick help resolving this!

          Alexandru Somai added a comment - I've released the version 1.3.0 that should fix this issue. Thank you jglick and timja for the quick help resolving this!

          Chris added a comment -

          Everything seems to be ok on my side again after an update to version 1.3.0 of external-workspace-manager-plugin.

          Good job and many thanks to all!

           

          Chris added a comment - Everything seems to be ok on my side again after an update to version 1.3.0 of external-workspace-manager-plugin. Good job and many thanks to all!  

          Ian added a comment -

          updated to 1.3.0 and problem is resolved. thanks for the quick turn around!

          Ian added a comment - updated to 1.3.0 and problem is resolved. thanks for the quick turn around!

            timja Tim Jacomb
            romanz Roman Zwi
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: