-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
Powered by SuggestiMate
A '%' character in a branch name breaks the GitHub API calls on a git repository if the job performing the clone is generated by GitHub Organization Folder plugin (windows or Linux) or the GitHub branch source (windows or Linux).
Steps to duplicate the problem:
- Define a GitHub multibranch PIpeline job referencing the MarkEWaite/jenkins-bugs repository (or a GitHub Organization Folders job which references the GitHub organization MarkEWaite, looking only at the repository jenkins-bugs)
- Scan the repository, watch the jobs run
- Open the has-percent-%-JENKINS-44360 job and confirm it failed to clone
Steps to show the same branch working with a multibranch pipeline:
- Define a multi-branch pipeline job using the MarkEWaite/jenkins-bugs github repo using Git as the branch source rather than GitHub
- Scan the repository, watch the jobs run
- Open the has-percent-%-JENKINS-44360 job and confirm it cloned successfully
The issue seems to require:
- GitHub Organization Folders or GitHub multibranch
- '%' in the branch name on the repository
- JENKINS-44360.log
- 159 kB
- blocks
-
JENKINS-34564 Give the ability to choose how the multibranch subprojects will be named.
-
- Resolved
-
- duplicates
-
JENKINS-2111 removing a job (including multibranch/org folder branches/repos) does not remove the workspace
-
- Resolved
-
- relates to
-
JENKINS-54654 A recent update breaks builds by escaping slashes to percent signs in workspace paths
-
- Resolved
-
[JENKINS-44360] '%' in branch name causes GitHub multi-branch job failures
medianick any chance that you've modified the PATH_MAX value (as described in the branch api pliugin)? That changes the workspace naming rules, and may be why you're seeing different results than I'm seeing.
I modified the Jenkinsfile in my validation job to only run on Windows agents, and still only see the problem in the context of a GitHub Organization Folder.
I'm sure you're seeing a problem, but I don't understand what is different between your environment and mine which makes it so that I can't see the problem you're seeing.
Another possible difference: Is your master node running on WIndows? Mine is a docker instance and I haven't yet expended the effort to do docker on Windows
markewaite, I haven't modified PATH_MAX; it's still the default (80). The only thing I've done to accommodate the longer paths generated by Multibranch Pipelines is to set the core.longpaths setting for Git to true. I'm running Windows Server 2012 with everything Jenkins-related under a C:\Jenkins folder.
Perhaps the difference is GitHub.com vs. GitHub Enterprise (which I'm using)?
medianick I interpret your statement:
I'm running Windows Server 2012 with everything Jenkins-related under a C:\Jenkins folder.
to mean that your master is on Windows. If so, then that is the most likely cause of the difference between your case and mine.
The fix of JENKINS-34564 should be taking care of this. Are you sure you are not disabling it?
As far as I can tell from all my checks on my Docker image definition, I am not defining PATH_MAX to the java virtual machine. I don't think that I'm disabling the fix of JENKINS-34564.
Jenkins properties for the Windows master (2.63) that I just installed show no setting for PATH_MAX. I started that master with the command java -jar jenkins.war. There were no additional command line arguments.
See Jenkins-System-Information.pdf for detailed system info output from the Windows master where I confirmed that I can still see the problem.
I've also confirmed on my Windows machine that multi-branch shows the same behavior (exactly as described by medianick). Thanks Nick. for your patience showing me that failure case!
The following output appears in the command processor window where I started Jenkins:
WARNING: Failed to parse URL for GitHubLink{iconClassName='icon-github-branch', url='https://github.com/MarkEWaite/jenkins-bugs/tree/has-percent-%-JENKINS-44360'}: java.net.URISyntaxException: Malformed escape pair at index 60: https://github.com/MarkEWaite/jenkins-bugs/tree/has-percent-%-JENKINS-44360
In my case the console log shows this:
java.io.FileNotFoundException: https://redacted/api/v3/repos/njones/TestRepo/git/refs/heads/test-%-builds at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:596) at org.kohsuke.github.Requester._to(Requester.java:262) Caused: java.io.FileNotFoundException:
with the remaining content being the HTML source of GitHub's standard "Looks like something went wrong!" page. And yes, in the Jenkins log itself, it has:
Failed to parse URL for GitHubLink{iconClassName='icon-github-branch', url='https://redacted/njones/TestRepo/tree/test-%-builds'}: java.net.URISyntaxException: Malformed escape pair at index 55: https://redacted/njones/TestRepo/tree/test-%-builds
It's exactly what you're seeing, markewaite.
You are using the GitHub branch source? If so, perhaps a bug in github-branch-source unrelated to the workspace path.
Indeed we face also this issue and i dare changing the priority because MSBuild is not handling properly this '%2f' string in workspace full path.
This blocks our builds even though our branches just respect the standard GitFlow automatically naming branches 'release/' or 'hotfix/' for example.
Workaround with using ws() command to change folder name before running any build step could be applied but is really not a right solution.
Thanks for your help on that.
Regards,
You must be running a really old build, or disabling the fix, because workspace paths have long since avoided %.
splashnenen you are describing something different than this bug is describing. This bug describes the case where a literal '%' character is embedded in the name of the branch. This bug is not describing a case where a literal '/' is embedded in the name of the branch.
As jglick noted, your case (literal '/' in the branch name) should work with standard settings, unless you have disabled the change which avoids URL escaping the branch name. I've changed the priority back to minor.
markewaite I indeed applied this configuration -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 , is it the one you are mentionning ? Should i remove that to properly handle '/' in branch names ?
Thanks for your help.
I have the same property set (...PATH_MAX=0) as a work around for JENKINS-34564 since the multi-branch pipeline plugin will create long string folder names.
Is there a way to have both? Or is the choice only between paths with percent escape sequences or long random folder paths?
Right now I'm getting around the long folder name issue by pointing my multi-branch job to the SVN /branches/* folder directly rather than /* and filtering on branches. This causes other issues that I'd like to change, but it is working for now.
Please read the https://plugins.jenkins.io/branch-api 1.11 changelog. Any discussion should be on the user’s list. It is off topic for this issue, which again relates to failure to check out branches containing %, and has nothing to do with workspace paths.
Can we just finally add some directory name filtering globally for workspaces? Keep only alphanumerics and simple things like minus or underscore. Could be a per-project setting that's enabled for new jobs by default.
Visual Studio builds are failing due to this and you can't really blame the build tool, because there's no telling nor proper handling for cases when paths get passed around, executed in shell/batch somewhere deep. Simple things stop working, such as plain simple scripts, not just build tools.
is there a fix for this issue? Even after adding below we are still seeing issues with the workspace name. I am having this issue on Windows slave machines. Running .net project with msbuild commandline.
-Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0
Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA
I continue to see job failures when using a branch name containing '%' with either a GitHub organization folder or a GitHub branch source. I don't see those job failures when I use Git as a branch source, only if I use GitHub as the branch source or a GitHub organization folder.
I haven't checked Bitbucket branch source or Gitea to see if the issue is specific to GitHub or is connected to something unrelated to the branch API plugin.
The failing job writes the following surprising log file into build/1/log:
Connecting to https://api.github.com using MarkEWaite/****** (MarkEWaite github username/password) java.io.FileNotFoundException: https://api.github.com/repos/MarkEWaite/jenkins-bugs/git/refs/heads/has-percent-%-JENKINS-44360 at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:625) at org.kohsuke.github.Requester.parse(Requester.java:607) at org.kohsuke.github.Requester._to(Requester.java:285) Caused: org.kohsuke.github.GHFileNotFoundException: ... Long content that looks like an HTML file ... at org.kohsuke.github.Requester.handleApiError(Requester.java:699) at org.kohsuke.github.Requester._to(Requester.java:306) at org.kohsuke.github.Requester.to(Requester.java:247) at org.kohsuke.github.GHRepository.getRef(GHRepository.java:891) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1531) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:98) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILURE
I've uploaded build/1/log as JENKINS-44360.log
The specific branch that shows the failure is in my jenkins-bugs GitHub repository.
markewaite you changed the Status to be IN REVIEW but there is no Assignee and I see no linked PR purporting to fix the issue.
Perhaps you just meant to set this to Open, and change the component to github-branch-source-plugin since that seems to be the source of the problem?
Thanks jglick! I took too simple approach and chose the bug status as one of the proposed in the flow for next status, rather than setting it to the correct state of Open. Status has been corrected and reassigned to github-branch-source-plugin.
In my experience earlier today, this happens with a Multibranch Pipeline pointed at a GitHub repository; it does not require a GitHub Organization Folders job. I had an established Multibranch Pipeline job (which had built master) and when I added a branch called "percents are 100% good", the new branch build in Jenkins failed with a java.io.FileNotFoundException (and dumped a large chunk of HTML to the console log). It looks like GitHub expects the "%" to be escaped as %25 when cloning the remote repository, and it wasn't.
I should add that this was on a Windows Jenkins node, and was pointed at GitHub Enterprise, in case either is a relevant detail.