Status: Resolved (View Workflow)
Jenkins 1.576, Matrix Plugin: 1.4, windows, Unix.
An issue related to 6048, but not just about special characters, but too long path names.
The Matrix Project plugin is picking the multi-configuration directory names, based on both the configuration axis names and their values, and this leads to very long sub-directory paths, not to mention names with inconvenient characters in them, as my options include hyphens as part of their values. (Both Windows and Unix allow hyphens in their directory names, but then to traverse these directories interactively is inconvenient as it requires escaping.)
I have three axis, one for the slave name, (whose parameter name defaults to "label"), a configuration build option for either 32 bit or 64 bit, and a debug or release build type option.
slaves names: company_server_1, company_server_2
values" --enable32, --enable64
values: debug, release
This leads to directory names such as:
In other words, 2 sub-directories as added for each axis consisting of a parameter name, and one of its possible values. Having 3 axis means 6 sub-directories are presently used.
In some cases, when added to the root of the file system used for the slave connection, I can end up a base build path of 100 characters, leading me to build problems when the build themselves contains other very long path names.
Trying to cd into a directory starting with --enable doesn't work, these have to be quoted.
I can do various things to shorten the axis parameter names, but not too much about the values, nor the root of the file system for the slave connection.
For example, I can make the label just "l", and the "bits_options" just "b", and the type option just t, but cannot change the options values themselves. This still leads to 6 sub-directories as follows:
Existing suggestion not workable:
This is a enhancement that is not needed for all projects. Invoking the Jenkins at startup with the MatrixConfiguration.useShortWorkspaceName as described in the 6048 issue would mean all projects get shortened hashname directories, which is not a desirable fix. Other users do not want or need this. The enhancement needs to be able to be applied on a per-project basis.
1) It is NOT necessary to include the parameter name as part of the directory path, as only the parameter values need to be used. In my example earlier, this means we do not need to include the "label" (or "l"), or "bit_option" (or 'b"), or "type" (or "t") in the directory path. Only the parameter values (and not also their names) are required. This would shorten the above example by 22 characters as coded originally, and a a further 6 characters (and 3 sub-directories) less after doing everything I can presently due to shorten the path. Only the following 3 sub-directory paths would be needed, and not 6 as are presently used.
So,the first suggestion is to either drop using the parameter name as part of the directory entirely, or at least, have a tick box which defaults to "on" for compatibility with existing behaviour, but which could be turned off.
2) the second suggestion is to allow the specifying of a directory name for each value. For example, in my situation, I would like to map the following values to shorter directory names:
parameter : company_server_1 ---> s1
parameter : company_server_2 ---> s2
parameter : debug ---> d
parameter : release ---> r
parameter : --enable32 ---> 32
parameter : --enable64 ---> 64
if I could specify the directory name to be used with each value, then with the above changes included, my 3 axis of parameters can be specified as 3 very short directory names, without special characters, as in:
which is considerably shorter then then the present method using 6 sub-directories which makes me skimp on parameter names.
- is related to
JENKINS-30148 Allocate shorter workspace if it will be too long for reasonable use inside build
Label expression axis can also create bad paths. If I have "label1&&label2" expression, the path then include /job/label1&&label2/...
This makes for some nasty workarounds in shell scripts.
spikerjenk2: Wouldn't it suffice to just consistently double-quote references to the folder? What specifically breaks?
You are right if the path was quoted correctly everywhere there would be no issue. However if you build a whole linux distribution with hundreds of third party packages it is very likely that some components get it wrong. To fix this is a huge effort and not possible to fix in the short term.
So I also would appreciate an option that would allow to get more "standard" paths.
The worst offender is python virtualenv. Wrapper scripts insert the shebang of the interpreter, which is in the workspace. So "myProg.py" would have
I unfortunately have to wrap everything in a shell script to re-write the shebang.
I'd also like to add (most) linux kernels limit the shebang line to 127 characters, which is easily outdone with big matrix jobs.
FWIW there's a global (Jenkins-wide) option to use short sub-workspace paths for matrix projects: hudson.matrix.MatrixConfiguration.useShortWorkspaceName
More information on:
Daniel, very useful hint. Thankyou. The applicability of that property sure isn't limited to Cygwin.
Some Windows kernels have bugs relating to deep paths (well short of the official path limit). I've had to abbreviate all my labels to get things to work, and even then it's troublesome.
On *nix, label expressions containing shell metacharacters are "fun". Yes, everything should be carefully quoted. It's a pain when you're working with things like third party shell script snippets in configure scripts though.
Some form of mapping would be very helpful. Or at least exposing this as an advanced matrix build option instead of a system property.
Better solved more generically, as in JENKINS-30148.
(note to self: read first, then comment )