Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-34564

Give the ability to choose how the multibranch subprojects will be named.

      The %2F encoding of / breaks many software (msbuild for example) and can't be changed.
      Being able to provide a rule to build the subproject names from the branch name instead of just encoding things would permit to give the user the power to work around their problems with this kind of software.

      This would be an advanced option in which one can write something like a sed regex.

          [JENKINS-34564] Give the ability to choose how the multibranch subprojects will be named.

          Tyrel Haveman added a comment - - edited

          Our workaround involves creating workspace paths based on the parent job and simplified branch name, with a limit on length, and using a small hash if the length is too long.

          For example, in the job example-core-pipeline, we might have a branch named feature/adding-something-important ends up going into the path D:\jenkins\workspace\example-core-pipeline\adding-somB5, and a branch named feature/goodbye-bugs goes to D:\jenkins\workspace\example-core-pipeline\goodbye-bugs.

          Trimming off the branch prefix (feature/ in these cases) helps to shorten the path somewhat. We chose to limit the rest of the branch name when used as a folder name to 12 characters. If it is longer, it gets trimmed to 10 characters and the rest of the name is hashed to a simple 2-character hash.

          We consider it extremely unlikely that we would have two branches with similar enough names that the first 10 characters plus a 2-character hash will create a collision. On our very active 18-member development team we've been using this since just after I reported JENKINS-38706 with high success. Through many dozens of branches there have been no collisions, the paths are human-readable, and they are not too long for our builds.

          Tyrel Haveman added a comment - - edited Our workaround involves creating workspace paths based on the parent job and simplified branch name, with a limit on length, and using a small hash if the length is too long. For example, in the job example-core-pipeline , we might have a branch named feature/adding-something-important ends up going into the path D:\jenkins\workspace\example-core-pipeline\adding-somB5 , and a branch named feature/goodbye-bugs goes to D:\jenkins\workspace\example-core-pipeline\goodbye-bugs . Trimming off the branch prefix ( feature/ in these cases) helps to shorten the path somewhat. We chose to limit the rest of the branch name when used as a folder name to 12 characters. If it is longer, it gets trimmed to 10 characters and the rest of the name is hashed to a simple 2-character hash. We consider it extremely unlikely that we would have two branches with similar enough names that the first 10 characters plus a 2-character hash will create a collision. On our very active 18-member development team we've been using this since just after I reported JENKINS-38706 with high success. Through many dozens of branches there have been no collisions, the paths are human-readable, and they are not too long for our builds.

          Sorin Sbarnea added a comment -

          I was hit by this bug again and it really drives me insane. I find really disappointing to see how users feedback is dismissed with remarks that aim to imply that the problem is not caused by this plugin and that instead is floating around somewhere between linux kernel and other tools using virtual environments (virtualenv,...).

          The reality is that this change is causing outages to 90% of the users, we cannot rely on dirty hacks forever just because someone puts "safety" (huge hash) before system uptime or usability.

          Let's be a bit pragmatic here, let's fix something that is affecting 9/10 users of this plugin instead of trying to fix a non-problem: hash collisions (more likely to be hit by an asteroid than having someone using this approach to compromise your jenkins instance, they are tons other more accessible).

          Now let me go back to changing jenkins home directory and patching the config file

          Sorin Sbarnea added a comment - I was hit by this bug again and it really drives me insane. I find really disappointing to see how users feedback is dismissed with remarks that aim to imply that the problem is not caused by this plugin and that instead is floating around somewhere between linux kernel and other tools using virtual environments (virtualenv,...). The reality is that this change is causing outages to 90% of the users, we cannot rely on dirty hacks forever just because someone puts "safety" (huge hash) before system uptime or usability. Let's be a bit pragmatic here, let's fix something that is affecting 9/10 users of this plugin instead of trying to fix a non-problem: hash collisions (more likely to be hit by an asteroid than having someone using this approach to compromise your jenkins instance, they are tons other more accessible). Now let me go back to changing jenkins home directory and patching the config file

          I'm also a bit disappointed. I have the workspace folder on C:\ and still can't install the nuget packages in the root of my repo since one nuget package has a long name. Using the long path trick on windows 10 isn't helping since the application that writes the files (in my case nuget) does not support it and throws errors. So no workaround possible for my situation.
          Is there any perspective on this?

          Marco Bachmann added a comment - I'm also a bit disappointed. I have the workspace folder on C:\ and still can't install the nuget packages in the root of my repo since one nuget package has a long name. Using the long path trick on windows 10 isn't helping since the application that writes the files (in my case nuget) does not support it and throws errors. So no workaround possible for my situation. Is there any perspective on this?

          James Dumay added a comment - - edited

          In the past I've mounted subdirectories as virtual drives to get around windows 260 character path limitations. With some experimentation there could be decent workaround in it.

          James Dumay added a comment - - edited In the past I've mounted subdirectories as virtual drives to get around windows 260 character path limitations. With some experimentation there could be decent workaround in it.

          but how would you do that in a multibranch scenario? Each mount must be dynamically created and unique based on the branch beeing built

          Marco Bachmann added a comment - but how would you do that in a multibranch scenario? Each mount must be dynamically created and unique based on the branch beeing built

          David Aldrich added a comment -

          What is the current status of this issue please? I am still running plugin 'Multi-Branch Project Plugin' v.0.3 (the latest is 0.5.1) so that my msbuild jobs run correctly on Windows. I don't understand where the current fix is applied or whether I should wait for a new fix. Please will someone clarify the situation?

          David Aldrich added a comment - What is the current status of this issue please? I am still running plugin 'Multi-Branch Project Plugin' v.0.3 (the latest is 0.5.1) so that my msbuild jobs run correctly on Windows. I don't understand where the current fix is applied or whether I should wait for a new fix. Please will someone clarify the situation?

          Jesse Glick added a comment -

          Current plugin releases fix the originally reported problem, that some characters are unsupported in workspace names. It also fixes the problem of workspace names of unbounded length, for example due to long branch names. The new bound is customizable via system property, and the new behavior can be turned off entirely via that same system property, as detailed in the release notes for this plugin. A more comprehensive solution that would satisfy a larger proportion of use cases out of the box needs to wait for an overhauled workspace handling system as described in JENKINS-2111.

          Jesse Glick added a comment - Current plugin releases fix the originally reported problem, that some characters are unsupported in workspace names. It also fixes the problem of workspace names of unbounded length, for example due to long branch names. The new bound is customizable via system property, and the new behavior can be turned off entirely via that same system property, as detailed in the release notes for this plugin. A more comprehensive solution that would satisfy a larger proportion of use cases out of the box needs to wait for an overhauled workspace handling system as described in JENKINS-2111 .

          Bruce Rust added a comment -

          1. You mention the "customizable via system property" detailed in the release notes. But I cannot find those release notes anywhere. I think I found what I needed on a different but related issue that somebody commented on. Can you provide a link to the necessary information so I have the full listing of what is available and the release notes, etc.?

          Bruce Rust added a comment - 1. You mention the "customizable via system property" detailed in the release notes. But I cannot find those release notes anywhere. I think I found what I needed on a different but related issue that somebody commented on. Can you provide a link to the necessary information so I have the full listing of what is available and the release notes, etc.?

          jglick As I mentioned in another issue, simple link can help a lot to people in order to find information about changes done or planned to be done. Currently I am just going trough an endless loop between dependent but 'Resolved as duplicate' issues trying to find the origin where the actual work is done.

          Dmitriy Belyaev added a comment - jglick As I mentioned in another issue, simple link can help a lot to people in order to find information about changes done or planned to be done. Currently I am just going trough an endless loop between dependent but 'Resolved as duplicate' issues trying to find the origin where the actual work is done.

          David Aldrich added a comment -

          I agree with Dmitriy Belyaev. It's very hard to find out what is going on with this issue. Also, I think that quite a while back there were comments that the issue shouldn't be fixed in the multibranch project plugin but in some other part of the system. It would be very helpful if this could all be clarified. Jesse mentioned:

          Current plugin releases fix the originally reported problem...

          Is he referring to the multibranch project plugin?

          David Aldrich added a comment - I agree with Dmitriy Belyaev. It's very hard to find out what is going on with this issue. Also, I think that quite a while back there were comments that the issue shouldn't be fixed in the multibranch project plugin but in some other part of the system. It would be very helpful if this could all be clarified. Jesse mentioned: Current plugin releases fix the originally reported problem... Is he referring to the multibranch project plugin?

          Paul Kordes added a comment -

          Yes, PLEASE! Which plugin and what version fixes this and where do you specify the change. I have a bunch of plugins that all refer to Pipeline and I don't see this option anywhere.

          Paul Kordes added a comment - Yes, PLEASE! Which plugin and what version fixes this and where do you specify the change. I have a bunch of plugins that all refer to Pipeline and I don't see this option anywhere.

          Hans Schulz added a comment -

          The change was introduced in version 1.11 of the Branch API Plugin (branch-api).

          Hans Schulz added a comment - The change was introduced in version 1.11 of the Branch API Plugin (branch-api).

          John Smith added a comment -

          I submitted a Pull Request to reduce the minimum length of workspace directories from 54 characters to 12 characters when

          -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX
          

          is set to a value between 1 and 54. The pull request is at https://github.com/jenkinsci/branch-api-plugin/pull/62.

          John Smith added a comment - I submitted a Pull Request to reduce the minimum length of workspace directories from 54 characters to 12 characters when -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX is set to a value between 1 and 54. The pull request is at https://github.com/jenkinsci/branch-api-plugin/pull/62 .

          Code changed in jenkins
          User: Stephen Connolly
          Path:
          src/main/java/jenkins/branch/WorkspaceLocatorImpl.java
          src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java
          http://jenkins-ci.org/commit/branch-api-plugin/6df91ed6911e49df85d3af2efd777ab330d5c497
          Log:
          Merge pull request #62 from creste/jenkins-34564

          JENKINS-34564 Allow workspace paths to be less than 54 characters.

          Compare: https://github.com/jenkinsci/branch-api-plugin/compare/e856fdf42eda...6df91ed6911e

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Stephen Connolly Path: src/main/java/jenkins/branch/WorkspaceLocatorImpl.java src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java http://jenkins-ci.org/commit/branch-api-plugin/6df91ed6911e49df85d3af2efd777ab330d5c497 Log: Merge pull request #62 from creste/jenkins-34564 JENKINS-34564 Allow workspace paths to be less than 54 characters. Compare: https://github.com/jenkinsci/branch-api-plugin/compare/e856fdf42eda...6df91ed6911e

          David Aldrich added a comment -

          Hi Stephen

          Please will you summarize the state of this issue?

          Does the merge of the pull request mean that this change is now in Jenkins?

          Does it mean that the concerns about long path names are mitigated? Where would the user set the path length?

          Sorry for the many questions but this issue spans lots of Jira issues and is hard to track by the uninformed.

          Thanks

          David

          David Aldrich added a comment - Hi Stephen Please will you summarize the state of this issue? Does the merge of the pull request mean that this change is now in Jenkins? Does it mean that the concerns about long path names are mitigated? Where would the user set the path length? Sorry for the many questions but this issue spans lots of Jira issues and is hard to track by the uninformed. Thanks David

          Jesse Glick added a comment -

          See release notes for this plugin.

          Jesse Glick added a comment - See release notes for this plugin.

          Tyrel Haveman added a comment -

          You are incredibly unhelpful jglick. Your comment does not summarize the state of this issue, it does not indicate whether this change is now in Jenkins, it does indicate that this mitigated, it does not tell the user how to set a path length.

          The changelog for workflow-api plugin, which I had to dig around for because you seem to be unable to link to, has not been updated since you checked in your changes. https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+API+Plugin – So it doesn't answer any of these questions either.

          Please just answer people's questions.

          Tyrel Haveman added a comment - You are incredibly unhelpful jglick . Your comment does not summarize the state of this issue, it does not indicate whether this change is now in Jenkins, it does indicate that this mitigated, it does not tell the user how to set a path length. The changelog for workflow-api plugin, which I had to dig around for because you seem to be unable to link to, has not been updated since you checked in your changes. https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+API+Plugin – So it doesn't answer any of these questions either. Please just answer people's questions.

          Jesse Glick added a comment -

          https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin is the wiki for the plugin named in this issue.

          Jesse Glick added a comment - https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin is the wiki for the plugin named in this issue.

          Tyrel Haveman added a comment -

          Thank you for the link. That is very helpful.

          As far as I can see, it still doesn't answer some of davida2009's questions:

          Does it mean that the concerns about long path names are mitigated? Where would the user set the path length?

          Tyrel Haveman added a comment - Thank you for the link. That is very helpful. As far as I can see, it still doesn't answer some of davida2009 's questions: Does it mean that the concerns about long path names are mitigated? Where would the user set the path length?

          John Smith added a comment -

          Hi tyrelh, please see https://issues.jenkins-ci.org/browse/JENKINS-34564?focusedCommentId=281910&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-281910. You can try setting the "jenkins.branch.WorkspaceLocatorImpl.PATH_MAX" environment variable to the maximum length Jenkins should use for workspace paths. There are several ways you can set environment variables in Jenkins so I won't explain that here but Google should have instructions if you need assistance.

          Note that the release notes at https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin probably need to be updated. Right now, the release notes for Version 1.11 still say "The default value is 80; values as low as 54 (but no lower) are possible". The release notes for Version 2.0.7 should probably say that the lower limit is now 12.

          John Smith added a comment - Hi tyrelh , please see https://issues.jenkins-ci.org/browse/JENKINS-34564?focusedCommentId=281910&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-281910 . You can try setting the "jenkins.branch.WorkspaceLocatorImpl.PATH_MAX" environment variable to the maximum length Jenkins should use for workspace paths. There are several ways you can set environment variables in Jenkins so I won't explain that here but Google should have instructions if you need assistance. Note that the release notes at https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin probably need to be updated. Right now, the release notes for Version 1.11 still say "The default value is 80; values as low as 54 (but no lower) are possible". The release notes for Version 2.0.7 should probably say that the lower limit is now 12.

          Jason MCDev added a comment -

          There is so much info here.  I am hitting the original encoding issue with a freestyle multibranch project using BitBucket project.   We use Visual Studio, Nuget, msbuild which are all dying on the the %2n.

           

          What do I have to do to get around that encoding being used?  Please help.

           

          Jason MCDev added a comment - There is so much info here.  I am hitting the original encoding issue with a freestyle multibranch project using BitBucket project.   We use Visual Studio, Nuget, msbuild which are all dying on the the %2n.   What do I have to do to get around that encoding being used?  Please help.  

          What plugin versions are you on jsnmcdev if you are on all the latest, you should not have this as a problem

          Stephen Connolly added a comment - What plugin versions are you on jsnmcdev if you are on all the latest, you should not have this as a problem

          Jason MCDev added a comment -

          That's what I thought @Stephen Connolly.  

          I have the following or (tried the following):
          Bitbucket Branch Source Plugin: 2.1.0
          Branch API Plugin: 2.0.8
          Multi-Branch Project Plugin: 0.6
          Jenkins: 2.32.2

          Thanks!

          Jason MCDev added a comment - That's what I thought @Stephen Connolly.   I have the following or (tried the following): Bitbucket Branch Source Plugin: 2.1.0 Branch API Plugin: 2.0.8 Multi-Branch Project Plugin: 0.6 Jenkins: 2.32.2 Thanks!

          Patrick Wolf added a comment -

          Wait. Did you say you are hitting this on a Freestyle multibranch project, jsnmcdev? If that is true, that is an entirely different plugin.  

          Multibranch freestyle jobs are implemented in this plugin: 

          https://wiki.jenkins-ci.org/display/JENKINS/Multi-Branch+Project+Plugin

          Patrick Wolf added a comment - Wait. Did you say you are hitting this on a Freestyle multibranch project, jsnmcdev ? If that is true, that is an entirely different plugin.   Multibranch freestyle jobs are implemented in this plugin:  https://wiki.jenkins-ci.org/display/JENKINS/Multi-Branch+Project+Plugin

          Jason MCDev added a comment -

          I have tried the Bitbucket team/project and freestyle multi-branch project.

          This is what I have installed - I need to use Bitbucket and handle building feature branches.  I didn't see a "Freestyle multibranch" plugin as listed as installed in the admin section.

           

          Jason MCDev added a comment - I have tried the Bitbucket team/project and freestyle multi-branch project. This is what I have installed - I need to use Bitbucket and handle building feature branches.  I didn't see a "Freestyle multibranch" plugin as listed as installed in the admin section.  

          Jason MCDev added a comment -

          I was able to get the freestyle multi-branch project working with the Bitbucket plugin after upgrading Jenkins and going through upgrading every plugin involved.  The long paths still cause issues on Windows. I had to upgrade my build agents to Windows Server 2016 and enable long paths.  Some tools in the MS stack still have issues ( nuget restore ).

          This gets me closer, but I would really love to see a shorter path.

          Jason MCDev added a comment - I was able to get the freestyle multi-branch project working with the Bitbucket plugin after upgrading Jenkins and going through upgrading every plugin involved.  The long paths still cause issues on Windows. I had to upgrade my build agents to Windows Server 2016 and enable long paths.  Some tools in the MS stack still have issues ( nuget restore ). This gets me closer, but I would really love to see a shorter path.

          Stephen Connolly added a comment - jsnmcdev did you try setting PATH_MAX (which was why this issue was marked as closed) :  https://github.com/jenkinsci/branch-api-plugin/blob/master/src/main/java/jenkins/branch/WorkspaceLocatorImpl.java#L59

          David Aldrich added a comment -

          stephenc How does one set PATH_MAX?

          David Aldrich added a comment - stephenc How does one set PATH_MAX?

          To set it from start-up add a system property:

          -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=...

          To change it at run-time, in the groovy console run:

          jenkins.branch.WorkspaceLocatorImpl.PATH_MAX=...

          Try not to change it often as you will litter the old workspaces every time you change it

          Stephen Connolly added a comment - To set it from start-up add a system property: -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=... To change it at run-time, in the groovy console run: jenkins.branch.WorkspaceLocatorImpl.PATH_MAX=... Try not to change it often as you will litter the old workspaces every time you change it

          David Aldrich added a comment -

          stephenc Thanks, and please will you clarify how to get the change that you merged? Is it in Jenkins 2.50?

          David Aldrich added a comment - stephenc Thanks, and please will you clarify how to get the change that you merged? Is it in Jenkins 2.50?

          Upgrade branch-api to 2.0.8 (which should work on any Jenkins from 1.642.3 upwards)

          Stephen Connolly added a comment - Upgrade branch-api to 2.0.8 (which should work on any Jenkins from 1.642.3 upwards)

          Sheamus Roche added a comment -

          Hi stephenconnolly, does 2.0.8 fix handling "/" in jenkins job name or not? I've just created my first multi-branch pipeline and coming across this issue in job generated from feature/development branch for the first time...

          Sheamus Roche added a comment - Hi stephenconnolly , does 2.0.8 fix handling "/" in jenkins job name or not? I've just created my first multi-branch pipeline and coming across this issue in job generated from feature/development branch for the first time...

          sroche this all depends on your definition of fix.

          In Jenkins, job names cannot contain a / ever.

          In 2.0.8 (and some earlier versions), we ensure that the job names never contain a / by encoding the / as %2F which fixes the job name. There are also fixed any places where we found links to jobs that were not correctly encoding links to the job (as to work correctly the URL needs to be double encoded so you get %2F -> %252F (which the browsers "helpfully" display as %2F so you might not realise that the URL has been double-encoded)

          Now on-disk, there are different rules that govern the on-disk filename, so we instead apply an encoding to map job names into 32-char max names with only a safe character set used and when necessary a hash of the full name is used to abbreviate the job name to ensure it fits into 32-char max.

          Next there are issues with workspace paths for jobs, so there is another encoding of the job workspace (I'd like to switch that to use the same mapping as for the job name, but that is a different feature) this means that the job workspace on agents will be another name that is controlled by -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=... 

          Finally the env.BRANCH_NAME should be the unescaped un-modified name of the branch.

          All the above is basically functioning as intended... so the question is "what was your 'issue' and - assuming it is the workspace path length one - did you change the system property and find your issue fixed?"

           

          Stephen Connolly added a comment - sroche this all depends on your definition of fix. In Jenkins, job names cannot contain a / ever . In 2.0.8 (and some earlier versions), we ensure that the job names never contain a / by encoding the / as %2F which fixes the job name. There are also fixed any places where we found links to jobs that were not correctly encoding links to the job (as to work correctly the URL needs to be double encoded so you get %2F -> %252F (which the browsers "helpfully" display as %2F so you might not realise that the URL has been double-encoded) Now on-disk, there are different rules that govern the on-disk filename, so we instead apply an encoding to map job names into 32-char max names with only a safe character set used and when necessary a hash of the full name is used to abbreviate the job name to ensure it fits into 32-char max. Next there are issues with workspace paths for jobs, so there is another encoding of the job workspace (I'd like to switch that to use the same mapping as for the job name, but that is a different feature) this means that the job workspace on agents will be another name that is controlled by -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=...   Finally the env.BRANCH_NAME should be the unescaped un-modified name of the branch. All the above is basically functioning as intended... so the question is "what was your 'issue' and - assuming it is the workspace path length one - did you change the system property and find your issue fixed?"  

          Varun Reddy added a comment -

          After setting the PATH_MAX = 0 the issue with long path has been resolved. I am still having issues with / as my branch name is release/1.0.0. Is there a solution for this problem?

          Varun Reddy added a comment - After setting the PATH_MAX = 0 the issue with long path has been resolved. I am still having issues with / as my branch name is release/1.0.0. Is there a solution for this problem?

          stephenconnolly - if you read the actual description of this issue (and large parts of the discussion) you will find that the actual issue has not been resolved:

          • Replacing slashes from branch names with %2F causes lots of tools to not work within the jenkins workspace (see discussion above for examples
          • Using a custom workspace in the pipeline script can be a workaround, but it has its limitations. For example, for us it breaks builds for pull requests with Bitbucket server.
          • Slashes in branch names are very common since they are used in git flow, the most widespread branching concept in the git world, so this is not an obscure problem.
          • Somehow, this issue's discussion at some point shifted the focus towards maximum workspace path length - which was indeed an issue as well, but a separate one. (Still, I am very thankful that you fixed that one!)

          I think this is what varun7447 is referring to and as you can see from many, many comments above, it is affecting the work of lots of people. Therefore I think this issue should be reopened.

          Jonas Hanschke added a comment - stephenconnolly - if you read the actual description of this issue (and large parts of the discussion) you will find that the actual issue has not been resolved: Replacing slashes from branch names with %2F causes lots of tools to not work within the jenkins workspace (see discussion above for examples Using a custom workspace in the pipeline script can be a workaround, but it has its limitations. For example, for us it breaks builds for pull requests with Bitbucket server. Slashes in branch names are very common since they are used in git flow, the most widespread branching concept in the git world, so this is not an obscure problem. Somehow, this issue's discussion at some point shifted the focus towards maximum workspace path length - which was indeed an issue as well, but a separate one. (Still, I am very thankful that you fixed that one!) I think this is what varun7447 is referring to and as you can see from many, many comments above, it is affecting the work of lots of people. Therefore I think this issue should be reopened.

          Varun Reddy added a comment -

          Thanks stephenconnolly yes that is exactly what I am trying to communicate. As there are two issues and only one is fixed(MAX PATH). This is a breaking change for users of git-flow and also who name the branch name contains /

          Varun Reddy added a comment - Thanks stephenconnolly yes that is exactly what I am trying to communicate. As there are two issues and only one is fixed(MAX PATH). This is a breaking change for users of git-flow and also who name the branch name contains /

          Michael Neale added a comment -

          I am pretty sure long ago the % was dropped from the file name on disk and another scheme was used (and then the other length problems result). I use git flow all the time and don’t see % on disk since over a year.

          Michael Neale added a comment - I am pretty sure long ago the % was dropped from the file name on disk and another scheme was used (and then the other length problems result). I use git flow all the time and don’t see % on disk since over a year.

          The file system hasn't used % for some time (since Jesse's fixes in 2016) - but it also can't be mapped back to the branch name.

          The way I've been using to get the branch name (e.g. to publish artefacts to S3 in a branch-specific folder) is to use `env.BRANCH_NAME`

          This works for me - I don't really care where on disk my build is located, but I do care where I'm going to put it.

          Robert Watkins added a comment - The file system hasn't used % for some time (since Jesse's fixes in 2016) - but it also can't be mapped back to the branch name. The way I've been using to get the branch name (e.g. to publish artefacts to S3 in a branch-specific folder) is to use `env.BRANCH_NAME` This works for me - I don't really care where on disk my build is located, but I do care where I'm going to put it.

          The env.JOB_NAME still has escaped characters. Example usage in my pipelines:

           

          def notifySuccessful() {
            hipchatSend color: 'GREEN', credentialId: 'HipChat', message: "<a href='${env.RUN_DISPLAY_URL}'>${env.JOB_NAME.replace('%2F', '/')} ${env.BRANCH_NAME} ${env.BUILD_NUMBER}</a> Successful", notify: true, room: 'Build Server Messages', sendAs: 'Jenkins', server: '', v2enabled: false
          }
          

           

          Robert Watkins added a comment - The env.JOB_NAME still has escaped characters. Example usage in my pipelines:   def notifySuccessful() {   hipchatSend color: 'GREEN' , credentialId: 'HipChat' , message: "<a href= '${env.RUN_DISPLAY_URL}' >${env.JOB_NAME.replace( '%2F' , '/' )} ${env.BRANCH_NAME} ${env.BUILD_NUMBER}</a> Successful" , notify: true , room: 'Build Server Messages' , sendAs: 'Jenkins' , server: '', v2enabled: false }  

          Michael Neale added a comment -

          no idea what this ticket is now about

          The original "solution" proposed was to tweak how the generated "jobs" are named, but was it really about what is on disk? 

          Michael Neale added a comment - no idea what this ticket is now about The original "solution" proposed was to tweak how the generated "jobs" are named, but was it really about what is on disk? 

          michaelneale - sorry if I am totally off track here, but yeah, I though it was all about what is on disk. The very first comment to this issue, made by you , actually states this, if I don't get it all wrong:

          This also breaks tools like "npm run" as well (even on filesystems where it is ok).

          "%" is not recommended as a path name on any platform, even if it is tolerated in some cases.
          I think the %2F causes more problems that people realise, so picking something else would make more sense vs making it an "advanced" option.

          However, after reading that you and twasink state that you have no % character in your workspace paths, I started digging deeper. And it turns out that the problem was, that I (long time ago) had installed the "Short Workspace Path" plugin. I was this that controlled my path names. While keeping the paths very short indeed, it still kept the %2F in the name.

          I have now removed the plugin, and resorted to limiting path length through MAX_PATH=54. This works, and also eliminates the %2F, since it only includes the very last character of the branch name in the folder name.

          So, thank you for putting me on the right path to solve this long-term problem on our server! And apologies for incorrectly reopening this issue, I will close it now again.

          Jonas Hanschke added a comment - michaelneale - sorry if I am totally off track here, but yeah, I though it was all about what is on disk. The very first comment to this issue, made by you , actually states this, if I don't get it all wrong: This also breaks tools like "npm run" as well (even on filesystems where it is ok). "%" is not recommended as a path name on any platform, even if it is tolerated in some cases. I think the %2F causes more problems that people realise, so picking something else would make more sense vs making it an "advanced" option. However, after reading that you and twasink state that you have no % character in your workspace paths, I started digging deeper. And it turns out that the problem was, that I (long time ago) had installed the "Short Workspace Path" plugin. I was this that controlled my path names. While keeping the paths very short indeed, it still kept the %2F in the name. I have now removed the plugin, and resorted to limiting path length through MAX_PATH=54. This works, and also eliminates the %2F, since it only includes the very last character of the branch name in the folder name. So, thank you for putting me on the right path to solve this long-term problem on our server! And apologies for incorrectly reopening this issue, I will close it now again.

          Michael Neale added a comment -

          Thanks for the update. Yes makes sense. Early on the names of job and disk were the same (but that changed about 18 months ago, hence confusion!)

          Michael Neale added a comment - Thanks for the update. Yes makes sense. Early on the names of job and disk were the same (but that changed about 18 months ago, hence confusion!)

          Varun Reddy added a comment -

          The issue I am talking about is below/attached. Because of the %2F the msbuild does not work. The actual branch name is release/18.22.

          Varun Reddy added a comment - The issue I am talking about is below/attached. Because of the %2F the msbuild does not work. The actual branch name is release/18.22.

          Michael Neale added a comment -

          varun7447 the path should not have %2F on disk if it is runnign the latest etc - hasn't for a long time, so not sure what is causing it to name things that way (perhaps some path limit?)

          Michael Neale added a comment - varun7447 the path should not have %2F on disk if it is runnign the latest etc - hasn't for a long time, so not sure what is causing it to name things that way (perhaps some path limit?)

          Varun Reddy added a comment -

          michaelneale I have Jenkins (If that is what you are referring) 2.107.2. I think the latest is 2.107.3. Is the issue fixed in the latest 2.107.3 version? There should not be an issue with the path. I have added -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 to Jenkins Master (Linux). I am using Windows Jenkins slave, do I need to add -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 to the slave machine too?

          Varun Reddy added a comment - michaelneale I have Jenkins (If that is what you are referring) 2.107.2. I think the latest is 2.107.3. Is the issue fixed in the latest 2.107.3 version? There should not be an issue with the path. I have added -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 to Jenkins Master (Linux). I am using Windows Jenkins slave, do I need to add -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 to the slave machine too?

          Michael Neale added a comment -

          varun7447 I think PATH_MAX=0 is probably what is causing it: it falls back to using URI encoding in the path names (which msbuild and other tools like npm don't like). If you made that a larger value or not set - it could use the non encoded name (at the risk of it being longer though)

          Michael Neale added a comment - varun7447 I think PATH_MAX=0 is probably what is causing it: it falls back to using URI encoding in the path names (which msbuild and other tools like npm don't like). If you made that a larger value or not set - it could use the non encoded name (at the risk of it being longer though)

          Varun Reddy added a comment - - edited

          michaelneale is it ok to set this like

          -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=10
          

           

           

          Varun Reddy added a comment - - edited michaelneale is it ok to set this like -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=10    

          Michael Neale added a comment -

          I am not sure how that works, never used it, but I know the new "default" is to not use URI encoding and I think that PATH_MAX=0 forces it to not use the default - if you can try it would be good. 

          Michael Neale added a comment - I am not sure how that works, never used it, but I know the new "default" is to not use URI encoding and I think that PATH_MAX=0 forces it to not use the default - if you can try it would be good. 

          Varun Reddy added a comment -

          I have completely removed and now the workspace folders names are large.

          I think that was the main reason I have added PATH_MAX=0. Now Is there a way I can avoid long path

          Varun Reddy added a comment - I have completely removed and now the workspace folders names are large. I think that was the main reason I have added PATH_MAX=0. Now Is there a way I can avoid long path

          Michael Neale added a comment -

          varun7447 if you set it to a value that is non zero - does it make it shorter without resorting to the URI encoding? 

          like set it to say 20 (it needs enough to make it not collide, as it will be a hash basically)

          Michael Neale added a comment - varun7447 if you set it to a value that is non zero - does it make it shorter without resorting to the URI encoding?  like set it to say 20 (it needs enough to make it not collide, as it will be a hash basically)

          Varun Reddy added a comment -

          michaelneale Yeah I tried different combinations. The encoding seems to be fixed. But the problem is the workspace names are messed up. If I have to troubleshoot anything it is hard by looking at the workspace name, if the workspace belongs to which job.

          Varun Reddy added a comment - michaelneale Yeah I tried different combinations. The encoding seems to be fixed. But the problem is the workspace names are messed up. If I have to troubleshoot anything it is hard by looking at the workspace name, if the workspace belongs to which job.

          diewu qin added a comment -

          Anyone can tell me if this issue fixed? I use parameter " -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=20 " but did not work, and my jenkins version is 2.89.4.

          diewu qin added a comment - Anyone can tell me if this issue fixed? I use parameter " -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=20 " but did not work, and my jenkins version is 2.89.4.

          Nick Jones added a comment -

          Attempting to use '\\?\' as a path prefix – in my case, setting '\\?\C:\Jenkins' as the "Remote root directory" for my JNLP-connected Windows agent – resulted in a "java.nio.file.InvalidPathException: Illegal character [?] in path" exception being thrown, which similar to https://bugs.openjdk.java.net/browse/JDK-8165748 (although I'm using the Oracle JRE8). It would appear that my only options with Windows agents to reduce overall path length (until JENKINS-2111 is implemented) is the use of the '-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX' parameter, which given the merge of https://github.com/jenkinsci/branch-api-plugin/pull/62 would appear to support generated paths as short as 12 characters, far less than the 80 being generated by default.

          Nick Jones added a comment - Attempting to use '\\?\' as a path prefix – in my case, setting '\\?\C:\Jenkins' as the "Remote root directory" for my JNLP-connected Windows agent – resulted in a "java.nio.file.InvalidPathException: Illegal character [?] in path" exception being thrown, which similar to https://bugs.openjdk.java.net/browse/JDK-8165748 (although I'm using the Oracle JRE8). It would appear that my only options with Windows agents to reduce overall path length (until JENKINS-2111 is implemented) is the use of the '-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX' parameter, which given the merge of https://github.com/jenkinsci/branch-api-plugin/pull/62 would appear to support generated paths as short as 12 characters, far less than the 80 being generated by default.

          Uma Shankar added a comment - - edited

          Hey guys, this seems to be still a problem with latest LTS Jenkins, I have Jenkins on Windows with a slave. I have used starting parameter as -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=2 works well on master, however builds on Node is still creating %2F folder which is breaking all the MSBuild projects

           

          Folder on Master:

           

          Folder on Slaves:

           

          Jenkins Version: Jenkins ver. 2.138.2

          Branch API version: 2.0.20

           

          Is there some specific configuration for slaves?

           

           

          Uma Shankar added a comment - - edited Hey guys, this seems to be still a problem with latest LTS Jenkins, I have Jenkins on Windows with a slave. I have used starting parameter as -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=2 works well on master, however builds on Node is still creating %2F folder which is breaking all the MSBuild projects   Folder on Master:   Folder on Slaves:   Jenkins Version:  Jenkins ver. 2.138.2 Branch API version: 2.0.20   Is there some specific configuration for slaves?    

          Jesse Glick added a comment -

          Please do not reopen. If you encounter bugs, file them separately with complete steps to reproduce from scratch. Or just try my proposed implementation of JENKINS-2111, which reduces path lengths considerably.

          Jesse Glick added a comment - Please do not reopen. If you encounter bugs, file them separately with complete steps to reproduce from scratch. Or just try my proposed implementation of JENKINS-2111 , which reduces path lengths considerably.

          Uma Shankar added a comment -

          jglick Apologies for reopening this issue. Regarding proposed implementation in JENKINS-2111, are you suggesting to use branch-api 2.0.21 ? What does it have different than 2.0.20, would like to know before putting it on production Jenkins instance?

           

          Also, JENKINS-2111 looks more about cleanup than fixing multi branch workspace folder name.

          Uma Shankar added a comment - jglick Apologies for reopening this issue. Regarding proposed implementation in JENKINS-2111 , are you suggesting to use branch-api 2.0.21 ? What does it have different than 2.0.20, would like to know before putting it on production Jenkins instance?   Also,  JENKINS-2111  looks more about cleanup than fixing multi branch workspace folder name.

          Jesse Glick added a comment -

          are you suggesting to use branch-api 2.0.21 ?

          When released. Or you can install a build from my PR today.

          What does it have different than 2.0.20

          A completely different system for selecting and managing workspace paths. Best discussed in the other issue or PR.

          Jesse Glick added a comment - are you suggesting to use branch-api 2.0.21 ? When released. Or you can install a build from my PR today. What does it have different than 2.0.20 A completely different system for selecting and managing workspace paths. Best discussed in the other issue or PR.

          Gerard added a comment - - edited

          I'm seeing this issue again (slashes coming up as "%2F" instead of "_") after my company upgraded from Jenkins 2.387.3 to 2.452.3 (and branch-api-plugin 2.1105 to 2.1178). Has anyone else seen it? It's causing lots of issues across many pipelines.

          Edit: It turns out a new plugin (ws-ws-replacement) was pulled in as a part of our upgrade. That plugin was also implementing WorkspaceLocator and beating the branch-api-plugin's implementation to the punch. It was only substituting spaces with underscores, hence leaving the "%2F" there and causing problems.

          Gerard added a comment - - edited I'm seeing this issue again (slashes coming up as "%2F" instead of "_") after my company upgraded from Jenkins 2.387.3 to 2.452.3 (and branch-api-plugin 2.1105 to 2.1178). Has anyone else seen it? It's causing lots of issues across many pipelines. Edit: It turns out a new plugin (ws-ws-replacement) was pulled in as a part of our upgrade. That plugin was also implementing WorkspaceLocator and beating the branch-api-plugin's implementation to the punch. It was only substituting spaces with underscores, hence leaving the "%2F" there and causing problems.

            jglick Jesse Glick
            rkutic Kevin Grandemange
            Votes:
            59 Vote for this issue
            Watchers:
            84 Start watching this issue

              Created:
              Updated:
              Resolved: