Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38706

Workspace directory names mangled in multibranch pipeline

      Apparently after a recent update of plugins, our Multibranch Pipeline jobs started putting their workspaces in mangled locations. They used to be, for example, in D:\jenkins\cm-jenkins-09\my-pipline-job\my-branch-name and now they're like D:\jenkins\cm-jenkins-09\ine-job_my-banch-name-5P5URKHPJXWGGVIWDQOJBB3N7RJECAQJGFTDCVOPY3PABO7LNTIQ

      There are several problems with this:

      1. They're difficult to read with all that garbage at the end, and the beginning of the name sometimes cut off
      2. They're not organized within the parent job like they used to be
      3. This is causing files within them to have extremely long paths such that Windows won't actually let us delete them (Jenkins, CMD, and Windows Explorer all fail to delete them)

      Let me know if I can provide any other information to help solve this. This is a private Jenkins install so I can't point you to it.

          [JENKINS-38706] Workspace directory names mangled in multibranch pipeline

          Tyrel Haveman created issue -
          Tyrel Haveman made changes -
          Description Original: Apparently after a recent update of plugins, our Multibranch Pipeline jobs started putting their workspaces in mangled locations. They used to be, for example, in {{D:\jenkins\cm-jenkins-09\my-pipline-job\my-branch-name}} and now they're like {{D:\jenkins\cm-jenkins-09\ine-job_my-banch-name-5P5URKHPJXWGGVIWDQOJBB3N7RJECAQJGFTDCVOPY3PABO7LNTIQ}}

          There are several problems with this:
          # They're difficult to read with all that garbage at the end, and the beginning of the name sometimes cut off
          # They're not organized within the parent job like they used to be
          # This is causing files within them to have extremely long paths such that Windows won't actually let us delete them (Jenkins, CMD, and Windows Explorer all fail to delete them)
          # It looks like, at least sometimes, jobs are getting a brand new workspace each time they run, causing disk space usage to climb very quickly

          Let me know if I can provide any other information to help solve this. This is a private Jenkins install so I can't point you to it.

          New: Apparently after a recent update of plugins, our Multibranch Pipeline jobs started putting their workspaces in mangled locations. They used to be, for example, in {{D:\jenkins\cm-jenkins-09\my-pipline-job\my-branch-name}} and now they're like {{D:\jenkins\cm-jenkins-09\ine-job_my-banch-name-5P5URKHPJXWGGVIWDQOJBB3N7RJECAQJGFTDCVOPY3PABO7LNTIQ}}

          There are several problems with this:
          # They're difficult to read with all that garbage at the end, and the beginning of the name sometimes cut off
          # They're not organized within the parent job like they used to be
          # This is causing files within them to have extremely long paths such that Windows won't actually let us delete them (Jenkins, CMD, and Windows Explorer all fail to delete them)

          Let me know if I can provide any other information to help solve this. This is a private Jenkins install so I can't point you to it.

          Greg Smith added a comment -

          This just happened to us as well.

          Not only are the names super long with that UUID like value on them, but the first character of the branch name is missing too.

          If you look at Tyrel's example, his directory name is "ine-job_my-branch...." – that was probably supposed to be "line" or something else. We are also seeing the first character removed.

          Greg Smith added a comment - This just happened to us as well. Not only are the names super long with that UUID like value on them, but the first character of the branch name is missing too. If you look at Tyrel's example, his directory name is "ine-job_my-branch...." – that was probably supposed to be "line" or something else. We are also seeing the first character removed.

          Tyrel Haveman added a comment - - edited

          gregcovertsmith Yeah, characters are being removed from the beginning of the name. It's not always just the first character, though; it seems to depend on the length of the build name. Longer names have more characters removed.

          We do have a workaround, but it's annoying: Inside the node element of our Jenkinsfile we wrap the entire build also with:

          ws("workspace/${env.JOB_NAME}") {
            // ...
          }
          

          Tyrel Haveman added a comment - - edited gregcovertsmith Yeah, characters are being removed from the beginning of the name. It's not always just the first character, though; it seems to depend on the length of the build name. Longer names have more characters removed. We do have a workaround, but it's annoying: Inside the node element of our Jenkinsfile we wrap the entire build also with: ws( "workspace/${env.JOB_NAME}" ) { // ... }

          Greg Smith added a comment -

          I would even vote for raising this to severe.

          I don't know what is worse – giant long workspace names, or randomly removing characters from the workspace names.

          I did go through and try to downlevel all of the plugins I had just updated – the problem never went away (I did it one by one – unfortunately, I was a couple of weeks behind, so there were a lot of updates)

          I just reached a point where my Jenkins install wouldn't start (too many cross dependencies), and I had to move back up to latest to get it started again.

          Greg Smith added a comment - I would even vote for raising this to severe. I don't know what is worse – giant long workspace names, or randomly removing characters from the workspace names. I did go through and try to downlevel all of the plugins I had just updated – the problem never went away (I did it one by one – unfortunately, I was a couple of weeks behind, so there were a lot of updates) I just reached a point where my Jenkins install wouldn't start (too many cross dependencies), and I had to move back up to latest to get it started again.

          Tyrel Haveman added a comment -

          I was looking a bit and Jenkins source trying to find out where this bug might be, and now I'm getting a terrible feeling that this might actually be intentional.

          Check out this chunk of a unit test in WorkflowBranchProjectFactoryTest.java:

          story.j.assertLogContains("branch=dev/main", b1);
          story.j.assertLogContains("workspace=dev_main-ZFNHWJSHKH4HUVOQUPOQV6WFX7XUPIKIAQAQ3DV7CCAGIXQW7YSA", b1);
          

          Tyrel Haveman added a comment - I was looking a bit and Jenkins source trying to find out where this bug might be, and now I'm getting a terrible feeling that this might actually be intentional . Check out this chunk of a unit test in WorkflowBranchProjectFactoryTest.java: story.j.assertLogContains( "branch=dev/main" , b1); story.j.assertLogContains( "workspace=dev_main-ZFNHWJSHKH4HUVOQUPOQV6WFX7XUPIKIAQAQ3DV7CCAGIXQW7YSA" , b1);

          Tyrel Haveman added a comment - - edited

          Indeed, this was done to fix JENKINS-30744. Encoding the name gets rid of %2F which was there before in place of / in a branch name. For our organization, %2F wasn't causing any problems (Maven doesn't seem to care) and I prefer it to this messy situation.

          Tyrel Haveman added a comment - - edited Indeed, this was done to fix JENKINS-30744 . Encoding the name gets rid of %2F which was there before in place of / in a branch name. For our organization, %2F wasn't causing any problems (Maven doesn't seem to care) and I prefer it to this messy situation.

          Ing. Christoph Obexer added a comment - - edited

          I vote for an option like with Matrix projects "Use custom child workspace". If multibranch projects had such an option it could be used to solve this issue and the feature branch disk space problem at the same time.

          The default value could be "${PROJECT_NAME}/${BRANCH_NAME}" and people with / in their branch names would simply get subdirectories (probably called features or releases) no one would get %2F in their folder names. Those with larger workspaces could set it to "${PROJECT_NAME}" and enjoy saving lots of gigabytes of disk space.

          Ing. Christoph Obexer added a comment - - edited I vote for an option like with Matrix projects "Use custom child workspace". If multibranch projects had such an option it could be used to solve this issue and the feature branch disk space problem at the same time. The default value could be "${PROJECT_NAME}/${BRANCH_NAME}" and people with / in their branch names would simply get subdirectories (probably called features or releases) no one would get %2F in their folder names. Those with larger workspaces could set it to "${PROJECT_NAME}" and enjoy saving lots of gigabytes of disk space.
          Edward Easton made changes -
          Environment Original: Windows 2012 x64, Java 1.8, Jenkins 2.7.4, Pipeline: Multibranch 2.9, Pipeline: API 2.4 New: Windows 2012 x64, Java 1.8, Jenkins 2.7.4, Pipeline: Multibranch 2.9, Pipeline: API 2.4, Linux Python

          Edward Easton added a comment -

          Hi,
          This change was very bad for all Python development. Super-long shebang paths like this cause all standard Python entry points in virtualenv to fail due to the Linux operating system shebang length. This includes 'pip' and 'easy_install' !
          I appreciate fixing the '%2f' was needed, but this is not the best way to do it I think. It's taken days and days fixing every single call to a subprocess with the standard workaround for this which is to run the programs as 'venv/bin/python venv/bin/entrypoint <args>'.

          Edward Easton added a comment - Hi, This change was very bad for all Python development. Super-long shebang paths like this cause all standard Python entry points in virtualenv to fail due to the Linux operating system shebang length. This includes 'pip' and 'easy_install' ! I appreciate fixing the '%2f' was needed, but this is not the best way to do it I think. It's taken days and days fixing every single call to a subprocess with the standard workaround for this which is to run the programs as 'venv/bin/python venv/bin/entrypoint <args>'.

            Unassigned Unassigned
            tyrelh Tyrel Haveman
            Votes:
            76 Vote for this issue
            Watchers:
            102 Start watching this issue

              Created:
              Updated:
              Resolved: