Replacing "/" with "" instead of "%2" is a good and quick solution but it might be still problematic for some users using "" and "/" in their branching schemes.

An alternative way might be to replace "/" with hierarchy. For example if the branch name is "feature/xyz", the workspace will be located in "C:\Jenkins\workspace\feature\xyz".

I also agree with @Michael that %2 behavior should not be the default as it causes errors that are hard to figure out for a user that is unaware that % in the path might be problematic for its tools.

I think there are 2 unsolved problems still:

1) what is the exact scheme to use that works best

• encodes most of the branch name in a non opaque way
• doesn't use path name elements which are known problematic

2) How to migrate existing jobs once #1 is solved.

I think it may be reasonable if #1 can be solved for new jobs, but it is not clear to me how automatic to make migration. This is paths on disk, so migration means moving things around. To me it would be acceptable if both old and new worked but new branches defaulted to the "corrected" encoding, but I imagine this would be messy and also problematic.

To michaelneale:

encodes most of the branch name in a non opaque way

The choice of encoding is a bit subtle, since you want to ensure that if two jobs have distinct fullName, they must also use distinct workspaces on a given node. This rules out simply replacing / with _, for example; you would have to more cautiously replace _ with __, etc.

There have also been reports of Windows users running up against the infamous 260-character path limit, and even on Linux users of the sshagent step in conjunction with docker.Image.inside will hit a 108-character limit (UNIX_PATH_MAX), so it may be necessary to use a hybrid scheme: an SHA-256 hash of the Item.fullName in Base64, plus up to a couple dozen informational scrubbed characters from the tail of the fullName to make it easier for humans to recognize the directory. For example, the following algorithm produces unique, safe, pretty recognizable workspace names no longer than 80 characters:

static String ws(String name) {
    return name.replaceAll("(%[0-9A-F]{2}|[^a-zA-Z0-9-_.])+", "_").replaceFirst(".*?(.{0,36}$)", "$1") + "-" +
        Base64.encode(Hashing.sha256().hashString(name).asBytes()).replace('/', '_').replace('+', '.').replaceFirst("=+$", "");
}

it is not clear to me how automatic to make migration

Migration is a nonissue since this is only about workspace names, and workspaces are dispensable. (Yes WorkspaceCleanupThread will lose track of the old directory, but this is already true for all sorts of more common cases such as job deletion so there is little sense worrying about it here; as a last resort we would need to just blow away the whole workspace root directory once a year or so.)

To jamesdumay:

I don't think making this configurable is going to help the situation

I never suggested making it configurable, except perhaps via system property as an escape hatch. Perhaps the original reporter of this issue assumed that would be the solution.

To myself:

you can trivially customize the workspace you use from within Jenkinsfile itself, using the ws step

Some people have since pointed out that there are instances of this problem affecting the initial checkout, due to buggy SCMs (or SCM plugins). Implementations of JENKINS-33273 would help in that regard, by avoiding the need for the @script workspace to begin with, but in the meantime these use cases are truly blocked.

My current proposal is to fix this in the core code for default workspace selection

It might be better to just implement a WorkspaceLocator in branch-api active on branch projects, as this could work even on current cores, and leave matrix-project to its own devices. For a Slave, it could refer to getWorkspaceRoot(). For Jenkins, I would not bother supporting the archaic customizable Jenkins.workspaceDir on master (there is no equivalent for agents and anyway you should be doing builds on agents); should suffice to hardcode $JENKINS_HOME/workspace. (See JENKINS-21942 for the sordid story of why this is not necessarily even the default location.)

Thanks jglick, good point in migration. I like your suggestion for a hybrid approach. I would have thought the first part of the name but you are right, the last part is more likely to be human recognisable/identifiable.

jglick ahh sorry for misattributing that to you!

To James Dumay:

I don't think making this configurable is going to help the situation

I never suggested making it configurable, except perhaps via system property as an escape hatch. Perhaps the original reporter of this issue assumed that would be the solution.

Yes, I was the one that gave this possible workaround. This didn't solve the real underlying problem, but as a user my problem was that the path was not good for some uses

I thought that if there were others with the same problem, the problem would be treated in depth. And it happened very quickly ! Thanks a lot !

Well, I appreciate the desire to treat the problem in depth, but given the nature of the issue, I'd think it warrants a quick workaround first, such that a better solution can be developed without so much pressure. I think the obvious workaround is to use "" instead of "%" in escapes and screen "" with another "_"; this is as close to the current scheme as possible and will keep the bi-directionality of the mapping. (Actually, I personally would have used a modified PunyCode scheme in the first place, but ...). Most build systems don't have problems with underscores, the changes required are minimal and one can then take time to think of and properly implement a better scheme.

Currently, multibranch subprojects do not work with the following build systems:

• Make (it interprets % in rule targets as a wildcard, and no escaping that I've tried seems to work)
• Boost Jam (fails with obscure error messages, have not identified the underlying problem yet)
• CMake (as reported in related bugs)
• MSBuild (as reported in related bugs)
• Apparently, a number of other build systems / application containers, etc.
• Large number of plugins (cppcheck, cifs, ...)

Of course, one could rightfully argue that all this software is broken, but I hope that some practical solution can be found quickly. This issue is absolutely critical to the usability of the multibranch stuff.

For the benefit of other affected users, I'm posting a workaround suggested by @rrodriguez__ on #jenkins IRC based on the comment in a linked ticket ( https://issues.jenkins-ci.org/browse/JENKINS-30744?focusedCommentId=247893#comment-247893 ):

node(agent) {

    def workspace_orig = pwd()
    def workspace_sane = workspace_orig.replaceAll("%", "_")

    ws(workspace_sane) {
        // ...
    }

}

I've also filed a bug report against Boost: https://svn.boost.org/trac/boost/ticket/12448 , but I'm not sure if one should really expect them to fix this anytime soon.

I'd think it warrants a quick workaround first, such that a better solution can be developed without so much pressure.

Well I suspect the solution I proposed would be an hour’s work for me to write, I just have not prioritized this yet, but it seems it has a lot of votes.

jglick yes it seems a lot of tools don't cope with funky names, so yeah, lots of votes - cest la vie.

Matrix projects have such an option called "Use custom child workspace". If multibranch projects had such an option it could be used to solve this issue and the feature branch disk space problem at the same time.

I hope "My current proposal is to fix this in the core code for default workspace selection, since I have heard no reports of the project name per se being a problem" means that this will then simply be the default? aka all branches building in the same folder called after the project?

Please fix.. this defect renders Jenkins at my current location as "not the way forward".

I just tried the work around above by Yury Zaytsev ( explained more in JENKINS-30744)... this works... Thank you!!!
However, it would still be nice to have an actual fix and get this out of the jenkinFile.

Code changed in jenkins
User: Jesse Glick
Path:
src/main/java/jenkins/branch/WorkspaceLocatorImpl.java
src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java
src/test/java/jenkins/branch/harness/BranchProjectFactoryImpl.java
http://jenkins-ci.org/commit/branch-api-plugin/040578616a1a7f66605203ee15ed2e9445f171ba
Log:
[FIXED JENKINS-34564] For branch projects, pick workspace names which are bounded in length and avoid unusual characters.

Awesome, thank you very much! Could you please tell in which plugin / release it will become available? Many thanks!

Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-multibranch-plugin/7c908dc9409eda22ead2885af3616f435b13ec27
Log:
JENKINS-34564 Adjusted integration test to match modified behavior.

Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-multibranch-plugin/dba7f3b9732ecf01b8a6e99f6ac10826394edd55
Log:
Merge pull request #33 from jglick/encoded-workspace-names-JENKINS-34564

JENKINS-34564 Adjusted integration test to match modified behavior

Compare: https://github.com/jenkinsci/workflow-multibranch-plugin/compare/34c7574b0cdc...dba7f3b9732e

The current fix (Just updated to Pipeline+Multibranch+Plugin 2.9) is still causing problems on Windows.

It is nice that the %2F is gone - but there is now another problem - i am hitting the 260-character path length limit.
I have a fairly typical .Net application - on my slave the workspace becomes c:\Jenkins\workspace\AS_hotfix_unit-test-failure-DZA4W4JGIWQ64M7NMWU3XPVUCYQQSZIABUVZWCGG373RLGNCFODQ which is ok - but very long.

But when running tests, I hit the error:

c:\Jenkins\workspace\AS_hotfix_unit-test-failure-DZA4W4JGIWQ64M7NMWU3XPVUCYQQSZIABUVZWCGG373RLGNCFODQ\MYPROJ.Net.Tests\\bin\Debug\\MYPROJ.Net.Tests.dll
System.IO.FileLoadException: The path is too long after being fully qualified.  Make sure the full path is less than 260 characters and the directory name is less than 248 characters.
File name: 'nunit.framework, Version=3.2.1.0, Culture=neutral, PublicKeyToken=2638cd05610744eb' ---> System.IO.FileLoadException: The path is too long after being fully qualified.  Make sure the full path is less than 260 characters and the directory name is less than 248 characters.
File name: 'nunit.framework'

A little strange - the path is well within the 248char limit, but my build fails anyway.

I solve it with this workaround:

def ensureSafeWorkspace(Closure block) {
	def maxWorkspacePathLen = 60;
	def isUnsafe = pwd().contains("%") || (pwd().length()>maxWorkspacePathLen);
	if (isUnsafe) { // Then we will request a new workspace...
		ws(safePath(env.JOB_NAME).take(maxWorkspacePathLen)) {
			block();
		}		
	} else { // Just call the closure in the current workspace (Avoid unnecessary master@2 workspace for example)
		block();
	}
}

Could we not just get rid of the excessively long string? Jenkins already seems to have features for preventing workspace sharing...

As espenalb has already commented:

on Windows, my build breaks, because the path is too long, because the workspace now contains this ridiculously long strings.
Can we somehow get rid of them?

It seems that fixing this issue can't be too easy apparently the default 80-char limit was way too generous for Windows slaves. But does it really warrant decreasing it? Or rather advice users to not to bury workspace too deep in the hierarchy?

bherfurth, in light of all problems with Windows path length limitations, why wouldn't you just give Jenkins agent a reasonable workspace path, e.g. C:\Jenkins\workspace <--- this should work.

my jenkins workspace is located under:

d:\install\jenkins\workspace

does not work ... since the directory structure is quite deep. (java, javascript, npm ... etc)

Maybe we could make this a configurable option?

I sincerely respect the work done, but I think it can't be left in the current state.

Before the change we were able to give projects and branches meaningful names. Only things to avoid were non-alphanumeric characters. It was an annoying issue that we couldn't use some special characters, I agree. However, in my opinion after the change we have a bigger one - in order to fit into the path length limit the only things we can do are:

• Place Jenkins home dir into C:\J or something like that. Weird, but not a big deal.
• I would also place workspace directory into one with a short name, BUT because of this line the setting is ignored so it's not possible. It's a separate issues, but should be fixed I believe.
• Give projects short/one-letter names.
• Give branches short/one-letter names.

Obviously, the last two are much worse limitations than just avoiding special chars. Of course, we could have some workarounds inside Jenkins file, but I think it's ideologically wrong to have workarounds for CI server or its plugins in source code or pipeline definition.

I'm not a java developer, but am happy to contribute once we all agree on the best approach. I'll have a play with the code and will submit a PR if I get any ideas.

Does it matter what the name is on disk? Should the JENKINS_HOME be as transparent as this? I am curious as to what people are doing with it (this change only impacts how it appears on disk if I understand it correctly)

espenalb bherfurth perhaps open a new ticket for the path limit on windows? as it may be able to be reduced.

I note that windows 10 claims to have removed that limit: https://mspoweruser.com/ntfs-260-character-windows-10/

The new solution has multiple issues which makes it unusable for us:

1. You have no simple way to get workspaces from org/project/branch/build which you can use in e.g. scripts.
2. The workspaces are neither reused nor cleaned up, causing a "filesystem leak". No simple way to write a cleanup script because of #1
3. You cannot use them on the console, not even with shell expansion, due to some starting with a "-" (ex. rm -r * in JENKINS_HOME/workspace fails)
4. They do not reside in the job's directory anymore, so deleting a job does not delete the workspaces (and again, no simple mapping possible)
5. The filename is very long. While not being an issue for us, many users have reported hitting MAXPATH with their projects on Windows.
   Why was just changing the escaping strategy to more sane characters not an option?

The new workspace naming schema can be disabled by setting the system property jenkins.branch.WorkspaceLocatorImpl.PATH_MAX to 0. Other values should control max length of generated directory name in new schema (but I didn't verified it).
To be effective, this system property need to be set during Jenkins startup (e.g. by adding -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 as a param).

So, in addition to causing massive problems on Windows, this also causes trouble on Unix, if the workspace contains shell scripts which use binaries residing in the workspace as interpreters. That is, something like what follows:

$ cat foo
#!/home/jenkins/agent/workspace/xxx-O7T6IFSVMNBO5RYGRVNFUGEOPAZAIQ4GQRK47UUDIOZMEN3K2OAA/source/lib/python-san/install/bin/python-dbg

One might think this ought to happen very rarely, but that's not the case! Basically, when you install any Python modules in a local prefix tree, it hardcodes the absolute path to the interpreter into the scripts and are you are screwed

See here for the details on `#!` maximum length limits (127 bytes on Linux):

http://www.in-ulm.de/~mascheck/various/shebang/

bq, You have no simple way to get workspaces from org/project/branch/build which you can use in e.g. scripts.

You should not be doing so.

The workspaces are neither reused

Yes they are.

nor cleaned up

If the job is deleted while the agent is online, the workspace is removed. Jenkins has more general issues here.

You cannot use them on the console, not even with shell expansion, due to some starting with a "-" (ex. rm -r * in JENKINS_HOME/workspace fails)

In general you should use --. Patches to ensure that the trimmed name does not start with - will be entertained of course (follow-up issues).

The filename is very long. While not being an issue for us, many users have reported hitting MAXPATH with their projects on Windows.

Yes; in general there is no foolproof solution for operating systems which impose path length limitations.

(although in the case of Windows there are workarounds IIRC involving a magic path prefix that switches paths to UNC syntax internally)

I am reopening this issue because clearly the new current behaviour is even worse than it was when the bug was opened in the first place. If we generate directory names that are ~80 characters long we can expect trouble. Not to mention that putting all of them in a single folder could generate some other scalability issues.

Until we have a proper change of the default behaviour (what we really should have!) people encountering this can do add this to /etc/default/jenkins:
-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=40 – probably it would be safe to put even a shorter value, like 20. The default number seems to be ridiculous high, somewhere between 80-90.

This feature did broke out Jenkins which was already tuned to workaround the shebang PATH limitations by moving the home directory of Jenkins user to /j folder, as opposed to /var/lib/jenkins. This was enough for us but now is not anymore.

If we generate directory names that are ~80 characters long we can expect trouble.

Depends entirely on what you are doing inside those directories. 80 characters is not really that long, and prior to the change directories prefixes could be much longer—unbounded, in fact (as noted in some complaints prior to the fix).

(Note that you can customize the prefix length via system property.)

PATH_MAX=40

No, anything less than 54 will not work at all, I just neglected to have that issue a warning.

already tuned to workaround the shebang PATH limitations

This really sounds like a separate issue, possibly in durable-task plugin, which should be filed separately with steps to reproduce from scratch—unless it is a limitation in the tools you are using: all I have seen so far is

when you install any Python modules in a local prefix tree, it hardcodes the absolute path to the interpreter into the scripts

which sounds like a bug in Python: it should use /usr/bin/env.

> which sounds like a bug in Python: it should use /usr/bin/env.

Just for the record: this isn't a bug by any means. They do it on purpose, because installed modules should be bound to a specific installing interpreter for many reasons. The question is, can they apply some hacks on their side to work around limited shebang path length issue or not, but it's a tangential one, and, in any case, the latest status was that if one could think of something, they'd welcome patches.

I updated my plugins recently and my builds stopped working. I tried Jesse Glick's suggestion and it worked, but branches names are now stripped (I'm ok with that).

For the record, in my case, it was failing to install packages in a python virtualenv inside working directory (path too long), during pip install -r requirements.txt:

bad interpreter: Permission denied

Before plugin update (was working, 56 characters):

/var/lib/jenkins/workspace/project/project/master@script

After update (failing, 109 characters):

/var/lib/jenkins/workspace/project_project_master-0GR3K62UHM3ITQD3DYGVNX7HHRXYWN3NP946WC282P28PH4B061E@script

Adding the following line to /etc/default/jenkins:

JAVA_ARGS="-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=30"

Results in this (working, 88 characters):

/var/lib/jenkins/workspace/r-MQDDZ79AS4LC4V4O49CTXGW52GDS8XDTMLMAIS05Q0ZHJ2NKK19N@script

Only one letter is kept from my branch but I don't really care as long as it works. Thanks for this. At this point, I would only keep generated hash.

In case it doesn't solve for other people, Using a shorter path for workspace sounds a good idea too:

/j folder, as opposed to /var/lib/jenkins.

But not quite sure what's the best way to achieve this.
Related SO Questions:
http://stackoverflow.com/questions/17310959/is-there-any-way-to-change-the-working-directory-of-a-jenkins-maven-build
http://stackoverflow.com/questions/21839538/change-jenkins-home-on-red-hat-linux

jglick

80 characters is not really that long

It's a third of the total possible path length on Windows. Some "special" tools on Linux are also shitty about this.

When I was running on Windows, I set the agent home to D:\J, and the hudson.model.Slave.workspaceRoot to ws, to waste as few chars as possible. This change would have wrecked my builds because Java devs routinely use up ~200 chars in deeply nested, verbosely named folder hierarchies.

perhaps its time to have a shorter hash-only thing to not eat the path length... (given many people are setting it to shorter and don't miss the job name)

IIRC there is some trick to make Windows behave properly here: prepending

\\?\

or the like to the agent FS root.

In general we cannot protect you from the consequences of having deep workspace path hierarchies. The former behavior (using raw Item.fullName) was shorter in some cases, but also unbounded in length and so caused problems for people with, for example, long branch names; the current behavior at least guarantees a bound on the part Jenkins controls.

Yes as per: http://stackoverflow.com/questions/1880321/why-does-the-260-character-path-length-limit-exist-in-windows
\\?\ is prefix to use.

This seems to imply that on windows 10 you can just change the registry and you don't have to do anything else: http://betanews.com/2016/05/29/long-paths-windows-10/

Why cant the hash just be shortened?
It worked for all of us before the change. Now it does not work anymore ==> change it back please.

Why cant the hash just be shortened?

It is already as short as it can be while guaranteeing that there is no collision between jobs.

Is the length limit documented somewhere?
There's usually no collision with git sha hash and it's 41 characters long. Jenkins seems to use a 53 characters long hash.

This is using SHA-256 encoded in Base32, it is different.

This is using SHA-256 encoded in Base32

So optionally use SHA-256 encoded in RFC-4648-flavoured Base64 - it'll be half the length.

Hmm. No, you'll have potential problems with case-insensitive file systems.

You could just use a UUID. The canonical form is 36 characters long, and you can express one in Base32 in just 16 characters.

I find ironic the fact that this fix that was supposed to help is creating more issues. Let's face it: the probability of causing build failures due to path limitations is 1000... times bigger than the one of getting checksum collisions.

I have seen LOTS of build failing due to PATH length related issues (for both job and workspace) but never a checksum collision, even for very short ones.

• I am sure that someone with more time could compute the real probabilities

The technique that I proposed when I added a configurable encoding to the branch names is as follows:

/**
     * Create a clean version of the name, suitable for use as file paths
     * @return the cleaned name
     */
    private String stripName() {
        String name = getName().replaceAll("_", "___"); // Escape all underscores

        return Util.rawEncode(name).
                replaceAll("%2F", "_"). // Replace forward slash with underscore
                replaceAll("%", "__"); // Replace remaining % with double underscore
    }

This maintains a unique, human readable form that doesn't (typically) add many characters to the branch name. Perhaps we could go with something similar?

it'll be half the length

Base64 is only about 20% shorter than Base32 (you forgot the logarithm here). Anyway

you'll have potential problems with case-insensitive file systems

Right, it can result in collisions on Windows and Mac.

You could just use a UUID.

No, because these are not deterministic from the job.

the one of getting checksum collisions

The probability of checksum collisions is not my concern. It is malicious collisions, which are easily constructed with weakened hashes.

Escape all underscores

Still vulnerable to collisions. There are plenty of variants which are not, but these also expose builds to failures due to branch name length, which was also a reported problem—not just charset restrictions.

The real fix is to maintain a per-node database of workspaces, as discussed in JENKINS-2111. Then each workspace can be given a guaranteed unique, human-readable name of guaranteed maximum length—as well as solving other longstanding issues not limited to branch projects.

Code changed in jenkins
User: Stephen Connolly
Path:
pom.xml
http://jenkins-ci.org/commit/branch-api-plugin/035c7e29193bdefc16a569047661471a2249ad82
Log:
Merge pull request #55 from jglick/compatibleSinceVersion

JENKINS-34564 compatibleSinceVersion=1.11

Compare: https://github.com/jenkinsci/branch-api-plugin/compare/81e61a4cb5a7...035c7e29193b

Code changed in jenkins
User: Stephen Connolly
Path:
pom.xml
http://jenkins-ci.org/commit/branch-api-plugin/dba46877d0c12703c8c812f769acc30d65030659
Log:
Merge pull request #55 from jglick/compatibleSinceVersion

JENKINS-34564 compatibleSinceVersion=1.11
(cherry picked from commit 035c7e29193bdefc16a569047661471a2249ad82)

Our workaround involves creating workspace paths based on the parent job and simplified branch name, with a limit on length, and using a small hash if the length is too long.

For example, in the job example-core-pipeline, we might have a branch named feature/adding-something-important ends up going into the path D:\jenkins\workspace\example-core-pipeline\adding-somB5, and a branch named feature/goodbye-bugs goes to D:\jenkins\workspace\example-core-pipeline\goodbye-bugs.

Trimming off the branch prefix (feature/ in these cases) helps to shorten the path somewhat. We chose to limit the rest of the branch name when used as a folder name to 12 characters. If it is longer, it gets trimmed to 10 characters and the rest of the name is hashed to a simple 2-character hash.

We consider it extremely unlikely that we would have two branches with similar enough names that the first 10 characters plus a 2-character hash will create a collision. On our very active 18-member development team we've been using this since just after I reported JENKINS-38706 with high success. Through many dozens of branches there have been no collisions, the paths are human-readable, and they are not too long for our builds.

I'm also a bit disappointed. I have the workspace folder on C:\ and still can't install the nuget packages in the root of my repo since one nuget package has a long name. Using the long path trick on windows 10 isn't helping since the application that writes the files (in my case nuget) does not support it and throws errors. So no workaround possible for my situation.
Is there any perspective on this?

In the past I've mounted subdirectories as virtual drives to get around windows 260 character path limitations. With some experimentation there could be decent workaround in it.

but how would you do that in a multibranch scenario? Each mount must be dynamically created and unique based on the branch beeing built

What is the current status of this issue please? I am still running plugin 'Multi-Branch Project Plugin' v.0.3 (the latest is 0.5.1) so that my msbuild jobs run correctly on Windows. I don't understand where the current fix is applied or whether I should wait for a new fix. Please will someone clarify the situation?

Current plugin releases fix the originally reported problem, that some characters are unsupported in workspace names. It also fixes the problem of workspace names of unbounded length, for example due to long branch names. The new bound is customizable via system property, and the new behavior can be turned off entirely via that same system property, as detailed in the release notes for this plugin. A more comprehensive solution that would satisfy a larger proportion of use cases out of the box needs to wait for an overhauled workspace handling system as described in JENKINS-2111.

1. You mention the "customizable via system property" detailed in the release notes. But I cannot find those release notes anywhere. I think I found what I needed on a different but related issue that somebody commented on. Can you provide a link to the necessary information so I have the full listing of what is available and the release notes, etc.?

jglick As I mentioned in another issue, simple link can help a lot to people in order to find information about changes done or planned to be done. Currently I am just going trough an endless loop between dependent but 'Resolved as duplicate' issues trying to find the origin where the actual work is done.

I agree with Dmitriy Belyaev. It's very hard to find out what is going on with this issue. Also, I think that quite a while back there were comments that the issue shouldn't be fixed in the multibranch project plugin but in some other part of the system. It would be very helpful if this could all be clarified. Jesse mentioned:

Is he referring to the multibranch project plugin?

Yes, PLEASE! Which plugin and what version fixes this and where do you specify the change. I have a bunch of plugins that all refer to Pipeline and I don't see this option anywhere.

The change was introduced in version 1.11 of the Branch API Plugin (branch-api).

I submitted a Pull Request to reduce the minimum length of workspace directories from 54 characters to 12 characters when

-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX

is set to a value between 1 and 54. The pull request is at https://github.com/jenkinsci/branch-api-plugin/pull/62.

Code changed in jenkins
User: Stephen Connolly
Path:
src/main/java/jenkins/branch/WorkspaceLocatorImpl.java
src/test/java/jenkins/branch/WorkspaceLocatorImplTest.java
http://jenkins-ci.org/commit/branch-api-plugin/6df91ed6911e49df85d3af2efd777ab330d5c497
Log:
Merge pull request #62 from creste/jenkins-34564

JENKINS-34564 Allow workspace paths to be less than 54 characters.

Compare: https://github.com/jenkinsci/branch-api-plugin/compare/e856fdf42eda...6df91ed6911e

Hi Stephen

Please will you summarize the state of this issue?

Does the merge of the pull request mean that this change is now in Jenkins?

Does it mean that the concerns about long path names are mitigated? Where would the user set the path length?

Sorry for the many questions but this issue spans lots of Jira issues and is hard to track by the uninformed.

Thanks

David

See release notes for this plugin.

You are incredibly unhelpful jglick. Your comment does not summarize the state of this issue, it does not indicate whether this change is now in Jenkins, it does indicate that this mitigated, it does not tell the user how to set a path length.

The changelog for workflow-api plugin, which I had to dig around for because you seem to be unable to link to, has not been updated since you checked in your changes. https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+API+Plugin – So it doesn't answer any of these questions either.

Please just answer people's questions.

https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin is the wiki for the plugin named in this issue.

Thank you for the link. That is very helpful.

As far as I can see, it still doesn't answer some of davida2009's questions:

Does it mean that the concerns about long path names are mitigated? Where would the user set the path length?

Hi tyrelh, please see https://issues.jenkins-ci.org/browse/JENKINS-34564?focusedCommentId=281910&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-281910. You can try setting the "jenkins.branch.WorkspaceLocatorImpl.PATH_MAX" environment variable to the maximum length Jenkins should use for workspace paths. There are several ways you can set environment variables in Jenkins so I won't explain that here but Google should have instructions if you need assistance.

Note that the release notes at https://wiki.jenkins-ci.org/display/JENKINS/Branch+API+Plugin probably need to be updated. Right now, the release notes for Version 1.11 still say "The default value is 80; values as low as 54 (but no lower) are possible". The release notes for Version 2.0.7 should probably say that the lower limit is now 12.

There is so much info here.  I am hitting the original encoding issue with a freestyle multibranch project using BitBucket project.   We use Visual Studio, Nuget, msbuild which are all dying on the the %2n.

What do I have to do to get around that encoding being used?  Please help.

What plugin versions are you on jsnmcdev if you are on all the latest, you should not have this as a problem

That's what I thought @Stephen Connolly.  

I have the following or (tried the following):
Bitbucket Branch Source Plugin: 2.1.0
Branch API Plugin: 2.0.8
Multi-Branch Project Plugin: 0.6
Jenkins: 2.32.2

Thanks!

Wait. Did you say you are hitting this on a Freestyle multibranch project, jsnmcdev? If that is true, that is an entirely different plugin.  

Multibranch freestyle jobs are implemented in this plugin: 

https://wiki.jenkins-ci.org/display/JENKINS/Multi-Branch+Project+Plugin

I have tried the Bitbucket team/project and freestyle multi-branch project.

This is what I have installed - I need to use Bitbucket and handle building feature branches.  I didn't see a "Freestyle multibranch" plugin as listed as installed in the admin section.

I was able to get the freestyle multi-branch project working with the Bitbucket plugin after upgrading Jenkins and going through upgrading every plugin involved.  The long paths still cause issues on Windows. I had to upgrade my build agents to Windows Server 2016 and enable long paths.  Some tools in the MS stack still have issues ( nuget restore ).

This gets me closer, but I would really love to see a shorter path.

jsnmcdev did you try setting PATH_MAX (which was why this issue was marked as closed) : https://github.com/jenkinsci/branch-api-plugin/blob/master/src/main/java/jenkins/branch/WorkspaceLocatorImpl.java#L59

stephenc How does one set PATH_MAX?

To set it from start-up add a system property:

-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=...

To change it at run-time, in the groovy console run:

jenkins.branch.WorkspaceLocatorImpl.PATH_MAX=...

Try not to change it often as you will litter the old workspaces every time you change it

stephenc Thanks, and please will you clarify how to get the change that you merged? Is it in Jenkins 2.50?

Upgrade branch-api to 2.0.8 (which should work on any Jenkins from 1.642.3 upwards)

Hi stephenconnolly, does 2.0.8 fix handling "/" in jenkins job name or not? I've just created my first multi-branch pipeline and coming across this issue in job generated from feature/development branch for the first time...

sroche this all depends on your definition of fix.

In Jenkins, job names cannot contain a / ever.

In 2.0.8 (and some earlier versions), we ensure that the job names never contain a / by encoding the / as %2F which fixes the job name. There are also fixed any places where we found links to jobs that were not correctly encoding links to the job (as to work correctly the URL needs to be double encoded so you get %2F -> %252F (which the browsers "helpfully" display as %2F so you might not realise that the URL has been double-encoded)

Now on-disk, there are different rules that govern the on-disk filename, so we instead apply an encoding to map job names into 32-char max names with only a safe character set used and when necessary a hash of the full name is used to abbreviate the job name to ensure it fits into 32-char max.

Next there are issues with workspace paths for jobs, so there is another encoding of the job workspace (I'd like to switch that to use the same mapping as for the job name, but that is a different feature) this means that the job workspace on agents will be another name that is controlled by -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=... 

Finally the env.BRANCH_NAME should be the unescaped un-modified name of the branch.

All the above is basically functioning as intended... so the question is "what was your 'issue' and - assuming it is the workspace path length one - did you change the system property and find your issue fixed?"

After setting the PATH_MAX = 0 the issue with long path has been resolved. I am still having issues with / as my branch name is release/1.0.0. Is there a solution for this problem?

stephenconnolly - if you read the actual description of this issue (and large parts of the discussion) you will find that the actual issue has not been resolved:

• Replacing slashes from branch names with %2F causes lots of tools to not work within the jenkins workspace (see discussion above for examples
• Using a custom workspace in the pipeline script can be a workaround, but it has its limitations. For example, for us it breaks builds for pull requests with Bitbucket server.
• Slashes in branch names are very common since they are used in git flow, the most widespread branching concept in the git world, so this is not an obscure problem.
• Somehow, this issue's discussion at some point shifted the focus towards maximum workspace path length - which was indeed an issue as well, but a separate one. (Still, I am very thankful that you fixed that one!)

I think this is what varun7447 is referring to and as you can see from many, many comments above, it is affecting the work of lots of people. Therefore I think this issue should be reopened.

Thanks stephenconnolly yes that is exactly what I am trying to communicate. As there are two issues and only one is fixed(MAX PATH). This is a breaking change for users of git-flow and also who name the branch name contains /

I am pretty sure long ago the % was dropped from the file name on disk and another scheme was used (and then the other length problems result). I use git flow all the time and don’t see % on disk since over a year.

The file system hasn't used % for some time (since Jesse's fixes in 2016) - but it also can't be mapped back to the branch name.

The way I've been using to get the branch name (e.g. to publish artefacts to S3 in a branch-specific folder) is to use `env.BRANCH_NAME`

This works for me - I don't really care where on disk my build is located, but I do care where I'm going to put it.

The env.JOB_NAME still has escaped characters. Example usage in my pipelines:

def notifySuccessful() {
  hipchatSend color: 'GREEN', credentialId: 'HipChat', message: "<a href='${env.RUN_DISPLAY_URL}'>${env.JOB_NAME.replace('%2F', '/')} ${env.BRANCH_NAME} ${env.BUILD_NUMBER}</a> Successful", notify: true, room: 'Build Server Messages', sendAs: 'Jenkins', server: '', v2enabled: false
}

no idea what this ticket is now about

The original "solution" proposed was to tweak how the generated "jobs" are named, but was it really about what is on disk? 

michaelneale - sorry if I am totally off track here, but yeah, I though it was all about what is on disk. The very first comment to this issue, made by you , actually states this, if I don't get it all wrong:

This also breaks tools like "npm run" as well (even on filesystems where it is ok).

"%" is not recommended as a path name on any platform, even if it is tolerated in some cases.
I think the %2F causes more problems that people realise, so picking something else would make more sense vs making it an "advanced" option.

However, after reading that you and twasink state that you have no % character in your workspace paths, I started digging deeper. And it turns out that the problem was, that I (long time ago) had installed the "Short Workspace Path" plugin. I was this that controlled my path names. While keeping the paths very short indeed, it still kept the %2F in the name.

I have now removed the plugin, and resorted to limiting path length through MAX_PATH=54. This works, and also eliminates the %2F, since it only includes the very last character of the branch name in the folder name.

So, thank you for putting me on the right path to solve this long-term problem on our server! And apologies for incorrectly reopening this issue, I will close it now again.

Thanks for the update. Yes makes sense. Early on the names of job and disk were the same (but that changed about 18 months ago, hence confusion!)

The issue I am talking about is below/attached. Because of the %2F the msbuild does not work. The actual branch name is release/18.22.

varun7447 the path should not have %2F on disk if it is runnign the latest etc - hasn't for a long time, so not sure what is causing it to name things that way (perhaps some path limit?)

michaelneale I have Jenkins (If that is what you are referring) 2.107.2. I think the latest is 2.107.3. Is the issue fixed in the latest 2.107.3 version? There should not be an issue with the path. I have added -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 to Jenkins Master (Linux). I am using Windows Jenkins slave, do I need to add -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 to the slave machine too?

varun7447 I think PATH_MAX=0 is probably what is causing it: it falls back to using URI encoding in the path names (which msbuild and other tools like npm don't like). If you made that a larger value or not set - it could use the non encoded name (at the risk of it being longer though)

michaelneale is it ok to set this like

-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=10

I am not sure how that works, never used it, but I know the new "default" is to not use URI encoding and I think that PATH_MAX=0 forces it to not use the default - if you can try it would be good. 

I have completely removed and now the workspace folders names are large.

I think that was the main reason I have added PATH_MAX=0. Now Is there a way I can avoid long path

varun7447 if you set it to a value that is non zero - does it make it shorter without resorting to the URI encoding? 

like set it to say 20 (it needs enough to make it not collide, as it will be a hash basically)

michaelneale Yeah I tried different combinations. The encoding seems to be fixed. But the problem is the workspace names are messed up. If I have to troubleshoot anything it is hard by looking at the workspace name, if the workspace belongs to which job.

Anyone can tell me if this issue fixed? I use parameter " -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=20 " but did not work, and my jenkins version is 2.89.4.