-
Bug
-
Resolution: Fixed
-
Major
-
Powered by SuggestiMate
Our repository contains a variant of the git-flow branch naming convention, where you will see
branches such as dev/main or release/3.2
Using multibranch, strange things happen:
- dev/main gets displayed as dev%2Fmain, then on restart dev%252Fmain, then dev%25252Fmain and all history seems to be lost.
This is likely down to / being a dodgy delimiter for a project name. I tried to fix this by adjusting WorkflowBranchProjectFactory thusly:
- WorkflowJob job = new WorkflowJob((WorkflowMultiBranchProject) getOwner(), branch.getName()); + String branchName = branch.getName().replace("/", "-"); + WorkflowJob job = new WorkflowJob((WorkflowMultiBranchProject) getOwner(), branchName);
but it didn't seem to help.
Happy to try to fix if I can get a pointer as to what to look at.
- depends on
-
JENKINS-30252 Provide easy access to git branch name in multibranch workflow scripts
-
- Resolved
-
-
JENKINS-30595 Obtain SCMHead from Item
-
- Resolved
-
- is blocking
-
JENKINS-32902 Multibranch pipeline has issues with / in jobName
-
- Resolved
-
- is duplicated by
-
JENKINS-34564 Give the ability to choose how the multibranch subprojects will be named.
-
- Resolved
-
- is related to
-
JENKINS-30798 Workflow multibranch failed after rebase: IllegalStateException: could not find branch tip
-
- Resolved
-
-
JENKINS-32701 Slashes in branch names cause 'bat' workflow task to hang
-
- Resolved
-
-
JENKINS-56956 Open Blue Ocean link with / character in branch name redirects to 404
-
- Closed
-
- links to
[JENKINS-30744] multibranch issues if branch contains /
maven fails in really weird places
Probably failing to use the correct APIs to encode filenames as file-protocol URLs.
I was unsure of the implications of the project name != branch name
Will break the current workarounds for JENKINS-30252 and JENKINS-30595.
FTR I talked with stephenconnolly who said that he was not positive this was ever properly fixed in branch-api + literate. MultiBranchProject.getItem path-decodes the Item.name, allowing Item.fullName etc. to be meaningful, though it has a hack to deal with browsers which double-decode paths or something like that. (Meaning a functional test may not suffice: also need an acceptance test verifying that a real browser + servlet container can deal with the resulting mess!) Given that the Item.name is encoded, I am not sure why getRootDirFor redundantly encodes it; that may have been the source of your problem after restart. Note also calls to Util.rawEncode from LiterateBranchProject.getRootDir and LiterateEnvironmentBuild.getShortUrl. Also CC vlatombe.
I don't remember such issues when using literate. so I think it handles it properly.
Code changed in jenkins
User: Jesse Glick
Path:
src/main/java/jenkins/branch/Branch.java
src/main/java/jenkins/branch/BranchProjectFactory.java
src/main/java/jenkins/branch/MultiBranchProject.java
http://jenkins-ci.org/commit/branch-api-plugin/02dc202f355e2a6b28d0069882614bee6c620bae
Log:
JENKINS-30744 Introduced Branch.encodedName, which factories are expected to use for Item.name.
Code changed in jenkins
User: Jesse Glick
Path:
src/main/java/jenkins/branch/Branch.java
src/main/java/jenkins/branch/BranchProjectFactory.java
src/main/java/jenkins/branch/MultiBranchProject.java
http://jenkins-ci.org/commit/branch-api-plugin/9c4bd01e48bcb749f65bcde1f8bc2a9c12c23066
Log:
Merge pull request #18 from jglick/slashy-branches-JENKINS-30744
JENKINS-30744 Introduced Branch.encodedName
Compare: https://github.com/jenkinsci/branch-api-plugin/compare/4d7ce28aedda...9c4bd01e48bc
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/org/cloudbees/literate/jenkins/LiterateBranchProject.java
http://jenkins-ci.org/commit/literate-plugin/a9499bc34ee4201b0ed21c09dbfe008c2ef52942
Log:
JENKINS-30744 Updated to use Branch.encodedName.
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/org/cloudbees/literate/jenkins/LiterateBranchProject.java
http://jenkins-ci.org/commit/literate-plugin/d72b8c94a1d99ae88e6b83641c0cd0b97061b246
Log:
Merge pull request #8 from jglick/slashy-branches-JENKINS-30744
JENKINS-30744 Updated to use Branch.encodedName
Compare: https://github.com/jenkinsci/literate-plugin/compare/677e289761ce...d72b8c94a1d9
Code changed in jenkins
User: Jesse Glick
Path:
multibranch/pom.xml
multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java
multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-plugin/a0b21acecfabf1b0f19cc0193f6f49796b42910a
Log:
JENKINS-30744 Updated to use Branch.encodedName, and added functional test.
Code changed in jenkins
User: Jesse Glick
Path:
multibranch/pom.xml
multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java
multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-plugin/a8ff4ecf71b5d4d9895cb5b3cd53f7916000e11d
Log:
Merge pull request #243 from jglick/slashy-branches-JENKINS-30744
JENKINS-30744 Updated to use Branch.encodedName
Compare: https://github.com/jenkinsci/workflow-plugin/compare/b5082ba5b09c...a8ff4ecf71b5
Code changed in jenkins
User: Jesse Glick
Path:
multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-plugin/fca115faca36a62968c1500ffd89e06b982a56f4
Log:
Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603.
(Multibranch Git webhooks rely only on the url parameter to detect changes.)
Code changed in jenkins
User: Jesse Glick
Path:
multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-plugin/521099bd8a31963ea6149de0a4f4ab9fe5617e97
Log:
Merge pull request #276 from jglick/test-JENKINS-29603
Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603
Compare: https://github.com/jenkinsci/workflow-plugin/compare/3840d8ca847e...521099bd8a31
The encoded path seems to create problems on windows when using 'bat'.
Any branch that has a slash in it for example ('feature/myfeature') will get an encoded path on windows like <workspace_path>/feature%2Fmyfeature@script and <workspace_path>/feature%2Fmyfeature.
When I apply the following workflow in a multibranch job in a root file 'Jenkinsfile', Jenkins hangs forever on the bat command
node { stage 'Checkout' checkout scm stage 'Build' bat "echo 'test'" }
When I add this same script on the 'master' branch everything is fine.
Environment:
Windows 8.1
Jenkins 1.643
Workflow Plugin 1.12
Confirmed that it is related to the encoding of the filepath by dirty-hacking the Branch API Plugin
public String getEncodedName() { return Util.rawEncode(getName()).replace("%2F","_"); }
After updating the plugin to this hacked one on my Windows Jenkins installation, the branches with / in the name work in the MultiBranch Workflow Project
I have a problem which might be related to this one. I use SVN with trunk and branches named like featureBranch1, featureBranch1, etc. (i.e. without / character in branch names).
I try simple Jenkinsfile:
env.MY_VAR1 = 'ABC' node('NODE_1') { bat "echo MY_VAR1 is set to %MY_VAR1% on NODE_1" }
When I create a multibranch job, this script executes correctly for the trunk but always hangs for branches. Last what I see in console is:
[Workflow] bat [branches%2FfeatureBranch1] Running batch script
More details:
In workspace folder (on Windows slave) folders named like branches%2FfeatureBranch1\.jenkins-05123d82 are created for each job. They have only 2 files
- jenkins-main.bat:
echo MY_VAR1 is set to %MY_VAR1% on NODE_1
- and jenkins-wrap.bat:
cmd /c "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-main.bat" > "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-log.txt" 2>&1 echo %ERRORLEVEL% > "c:\Dev\Jenkins\workspace\workflow-sample-multibranch\branches%2FfeatureBranch1\.jenkins-b9eefe38\jenkins-result.txt"
I use Jenkins 1.625.3, Workflow plug-in 1.12. Can %2F in folder name cause the batch command hanging?
@gitt I see the same thing.
Running on master in C:\Jenkins\Home\workspace\dgx_mcpu\topic%2Fpipelinetest [Pipeline] node { [Pipeline] echo Posting GitLab commit status: pending [Pipeline] Enforce time limit : Start [Pipeline] timeout { [Pipeline] isUnix [Pipeline] bat [topic%2Fpipelinetest] Running batch script Sending interrupt signal to process Sending interrupt signal to process Sending interrupt signal to process Click here to forcibly terminate running steps Aborted by Colin D Bennett Sending interrupt signal to process Click here to forcibly terminate running steps Click here to forcibly terminate running steps
This issue is fixed. If there are problems with Windows bat steps that are related to particular characters, file them in the durable-task plugin with workflow label and link to this issue.
So my understanding from the above is that for multi branch this is considered resolved but if we want to make use of multi branch with gitflow on Windows jenkins slaves where our git repo has branches like feature/XXX hotfix/XXX etc we have to either fork and hack this plugin or hope someone resolves it in the durable task ?
I am just trying to understand the possible ways forward as we really like the workflow plugin but can't use it basically due to the above.
Alex
alex_ouzounis
We've run into the same issue with our Windows slaves.
I've been able to make a simple workaround, although still a workaround, that allows us to continue to use the multibranch feature.
def getWorkspace() { pwd().replace("%2F", "_") } node { ws(getWorkspace()) { .... } }
now this is pure genius! We will try this ASAP Also shows how flexible workflow is with the JenkinsFile definitions though.
Many thanks joostvdg !! We owe you a beer !
There is another variant to this problem that i am experiencing on a Linux box.
I have a maven project that uses Jaxb to load a schema document and generate something from it. This project fails with:
[ERROR] [ERROR] schema_reference.4: Failed to read schema document 'file:/apps/.jenkins/workspace/project/feature%2FmyFeature/somePath/src/main/resources/MySchema.xsd', because 1) could not find the document; 2) the document could not be read; 3) the root element of the document is not <xsd:schema>.
[ERROR] unknown location
[ERROR]
Likewise, I have a project with unit tests that read in a file from the filesystem, make modifications, and write it back out. However, these test fail seemingly because the file can't be written out.
When I change my feature branch name from "feature/myFeature" to just "myFeature" the projects succeed in building.
I will create a separate issue since this one has been resolved.
joostvdg works as expected, the problem has now gone away. many thanks for sharing your workaround
Code changed in jenkins
User: Jesse Glick
Path:
multibranch/pom.xml
multibranch/src/main/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactory.java
multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-multibranch-plugin/b11fd4d0b9c61682df0f4eece40acd8e0029148c
Log:
JENKINS-30744 Updated to use Branch.encodedName, and added functional test.
Originally-Committed-As: a0b21acecfabf1b0f19cc0193f6f49796b42910a
Code changed in jenkins
User: Jesse Glick
Path:
multibranch/src/test/java/org/jenkinsci/plugins/workflow/multibranch/WorkflowBranchProjectFactoryTest.java
http://jenkins-ci.org/commit/workflow-multibranch-plugin/c9738eb45a25c1ae6a6e55df0d9688654d328725
Log:
Extend JENKINS-30744 test to confirm that it is unaffected by JENKINS-29603.
(Multibranch Git webhooks rely only on the url parameter to detect changes.)
Originally-Committed-As: fca115faca36a62968c1500ffd89e06b982a56f4
I'm still encountering this issue (using Jenkins 2.6 and all plugins up-to-date). The workaround of Joost van der Griendt works but this issues is quite frustrating when starting with pipelines. Especially when having a build that works perfectly as a "standalone" pipeline and fails in the multibranch environment (with Maven plugins that claim that the path cannot be found as they use java.net.URL).
Edit: Just read your comment on JENKINS-30744. I agree that the bug is in the Maven plugin (so far I encountered 2 such plugins). Nevertheless this is frustrating when migrating to multibranch pipeline builds.
We are also having this issue and are not using Maven at all. We are invoking NDK-build directly from shell and apparently ndk-build cannot work with paths containing %. Also happens when invoking MSBuild. We are currently using trick to allocate a new workspace with %2F replaced with _, however when job is finally removed (after pull request has been merged and branch deleted), the workspace remains, wasting valuable disk space.
Still encountering this in Jenkins 2.10 with all plugins up to date. The workaround works, but it's only a workaround and we have to have separate housekeeping tasks for removing old left-over stuff in the workspace.
Got an issue with branches containing slashes calling
ant.axisWsdl2java
from within an gradle script.
OS: ubuntu 14.04
the org.apache.axis.tools.ant.wsdl.Wsdl2javaAntTask somehow decodes the path and throws an exception in the next step of not being able to find feature/jenkins .. because it is feature%2Fjenkins
I tried the workaround above from Joost van der Griendt which worked for CMake (which does not like % in paths) but then my tests, which use Matlab, fail as now there is a '@' in the path instead (Matlab will also fail with % in the path) A real fix for this would be really useful, as now I'm having to copy everything to a temp folder to run everything (and clean-up is a pain).
Please refer to JENKINS-34564 for the request for alternate workspace naming.
This is not really resolved "fixed", the issue will be resolved when JENKINS-34564 is resolved. Can it be resolved as a duplicate?
Will this codefix also be applied to the old "multi-branch-project-plugin" (and not only to "pipeline multi branch" plugin) ??
A lot of issues (etc. JENKINS-37677 and JENKINS-32239 etc) are regarding the same error - but just for the old "multi-branch-project-plugin" plugin...
All of these issues are marked as duplicated of this fix (which is for the new "pipeline multi branch" plugin).
Changing my workspace name from workspace/foo/bar%2Fstuff to workspace/foo_bar-stuff-ISUDFHISDHFG345934SDHFSDYG82734SDFH has created a lot of problems because it's much longer. I don't find this to be an acceptable solution.
%2F wasn't causing any problems for our org. Why not actually use a / so we can have the workspaces organized with subdirectories? workspace/foo/bar/stuff would be great. Or, alternately, workspace/foo/bar_stuff would be fine. But workspace/foo_bar-stuff-ISUDFHISDHFG345934SDHFSDYG82734SDFH is not.
As noted in JENKINS-38706, this is causing files within them to have extremely long paths such that Windows won't actually let us delete them (Jenkins, CMD, and Windows Explorer all fail to delete them. We have to rename them to shorter names first), which ends up causing us to run out of disk space because our branch builds never get deleted.
Same for me. Even the Windows 10 long path trick is not working since applications like NuGet are not prepared for long paths. So I can't use the multi branch workflow anymore.
The provided fix does not work on Windows using Visual Studio. The path gets too long, so that VS cannot read/write files in deep directory hierarchies. What is the extra suffix good for? Is it a "path friendly" version of the whole branch name? If so, why not just translate the "/" or other forbidden characters into to a short sequence of valid characters? E.g.
/
becomes
_2f
,
%
becomes
_25
and
_
becomes
_5f
. This is almost the same as the previous solution "%2f". But when using the underline as escape character no "smart" software like VS or cmd.exe will try to interpret the % character with all these seen funny results.
For anyone that needs escaped file paths and doesn't want them to be too long, I submitted a Pull Request to reduce the minimum length of workspace directories from 54 characters to 12 characters when
-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX
is set to a value between 1 and 54. The pull request is at https://github.com/jenkinsci/branch-api-plugin/pull/62.
branch-api-plugin 2.0.7 has https://github.com/jenkinsci/branch-api-plugin/pull/62 merged
stephenconnolly I've updates all my plugins and jenkins to the latest. As far as I can tell by build job with the "/" in the branch name completed, but I cannot browse to the job:
I get the following error:
404 - File or directory not found.
The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.
The url is in the following format:
https://myserver.ca/job/myjobname/job/hotfix%252Fbranchname/
This only happens when with branches that contain "/". I've tried
-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 and
-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=1
What am I missing here?
Thanks,
pogorman I ran into that issue too, are you running IIS as a reverse proxy? Try setting the allow double escaping (Go to Request Filtering, go to Edit feature settings and tick the checkbox). That fixed it for me.
What is still a problem though is building branches with a slash using msbuild.
I've been searching and searching, but can't find a solution. I've got a pipeline script running my build. The develop branch builds fine, but now I have a Release/1.5.0.4-hotfix branch. This is what happens (among others):
C:\Jenkins\workspace****\Releases%2F1.5.0.4-hotfix*******.csproj" (default target) (3:3) ->
(CoreCompile target) ->
CSC : error CS0006: Metadata file 'C:\Jenkins\workspace*****\Releases/1.5.0.4-hotfix\packages\Autofac.4.0.1\lib\net45\Autofac.dll' could not be found [C:\Jenkins\workspace\*****\Releases%2F1.5.0.4-hotfix\*****\*****.csproj]
CSC : error CS0006: Metadata file 'C:\Jenkins\workspace*****\Releases/1.5.0.4-hotfix_references\CookComputing.XmlRpcV2.dll' could not be found [C:\Jenkins\workspace\*****\Releases%2F1.5.0.4-hotfix\*****\*****.csproj]
Some of the build goes fine... it checks out the code, it does the nuget restore, but during the build inside msbuild it changes the encoding back and voila there is the slash again... So when is this encoding stuff finally being fixed? Just replace the special characters on the disk. Somewhere it says that you do, but it really doesn't happen...
I need that encoding gone in my folders it suits no purpose and doesn't make it more readable either.
thois is this issue resolved for you? I am facing the same exact same issue. The / in the branch name(Releases/test-db) for branches is not working. Any help or workaround that you are aware of ?
varun7447 No, we've adjusted the way we name our branches, which is a pain.
Is this issue being addressed in the coming release? Or is this has to be fixed by the build tools? In this msbuild and nuget.exe?
I couldn't figure this out either. If we use gitflow - we have to make sure that we don't use the default branch names:
feature/myfeat
bug/themess
I don't know why this issue is marked as closed? What are we missing here?
Yes we are using git-flow and we are having same above issue as pogorman . Is there another defect that we need to track this issue?
Why is this issue closed? I mean any MSBuild run will fail with this conversion of "/". Many branching strategies include branch names that have a "/" in them. Pretty standard for Bitbucket installations as it defaults to.
feature/
hotfix/
bugfix/
release/
Please reopen and convert to something less painful!
nemligtim please refer to the duplicate bug JENKINS-34564 which describes more details. This is a duplicate bug and the project prefers to close duplicates so that they don't clutter searches.
The bug JENKINS-34564 also does not work for MSBuild. It fixes the longpath but not the issues with /
I got around this by giving it a specific workspace.
agent {
node {
label 'windows'
// Todo: This s really bad but Jenkins Multibranch is being very bad with very long paths being generated!
customWorkspace "${JOB_NAME}"
}
}
I got around the conversion of "/" by changing the branching strategy in Bitbucket to feature-, hotfix-, release- instead of feature/, hotfix/, release/.
Both of these are pretty frustrating. Especially the "/".
I am currently using the Branching strategy to include - instead of / . But the default behavior in setting up git flow branches are / .
jajansen, where did you add below code?
public String getEncodedName() { return Util.rawEncode(getName()).replace("%2F","_"); }
Correction - fix did work, but I had to re-create the project. I'll see if it works (I get other oddities with projects with %2F in them, like maven fails in really weird places!)
It occurred to me that maybe the correct place to fix was in Branch (something like branch.getSafeName() or similar) but I was unsure of the implications of the project name != branch name