Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44360

'%' in branch name causes GitHub multi-branch job failures

      A '%' character in a branch name breaks the GitHub API calls on a git repository if the job performing the clone is generated by GitHub Organization Folder plugin (windows or Linux) or the GitHub branch source (windows or Linux).

      Steps to duplicate the problem:

      1. Define a GitHub multibranch PIpeline job referencing the MarkEWaite/jenkins-bugs repository (or a GitHub Organization Folders job which references the GitHub organization MarkEWaite, looking only at the repository jenkins-bugs)
      2. Scan the repository, watch the jobs run
      3. Open the has-percent-%-JENKINS-44360 job and confirm it failed to clone

      Steps to show the same branch working with a multibranch pipeline:

      1. Define a multi-branch pipeline job using the MarkEWaite/jenkins-bugs github repo using Git as the branch source rather than GitHub
      2. Scan the repository, watch the jobs run
      3. Open the has-percent-%-JENKINS-44360 job and confirm it cloned successfully

      The issue seems to require:

      • GitHub Organization Folders or GitHub multibranch
      • '%' in the branch name on the repository

          [JENKINS-44360] '%' in branch name causes GitHub multi-branch job failures

          Nick Jones added a comment - - edited

          In my experience earlier today, this happens with a Multibranch Pipeline pointed at a GitHub repository; it does not require a GitHub Organization Folders job. I had an established Multibranch Pipeline job (which had built master) and when I added a branch called "percents are 100% good", the new branch build in Jenkins failed with a java.io.FileNotFoundException (and dumped a large chunk of HTML to the console log). It looks like GitHub expects the "%" to be escaped as %25 when cloning the remote repository, and it wasn't.

          I should add that this was on a Windows Jenkins node, and was pointed at GitHub Enterprise, in case either is a relevant detail.

          Nick Jones added a comment - - edited In my experience earlier today, this happens with a Multibranch Pipeline pointed at a GitHub repository; it does not require a GitHub Organization Folders job. I had an established Multibranch Pipeline job (which had built master) and when I added a branch called "percents are 100% good", the new branch build in Jenkins failed with a java.io.FileNotFoundException (and dumped a large chunk of HTML to the console log). It looks like GitHub expects the "%" to be escaped as %25 when cloning the remote repository, and it wasn't. I should add that this was on a Windows Jenkins node, and was pointed at GitHub Enterprise, in case either is a relevant detail.

          Mark Waite added a comment - - edited

          medianick any chance that you've modified the PATH_MAX value (as described in the branch api pliugin)? That changes the workspace naming rules, and may be why you're seeing different results than I'm seeing.

          I modified the Jenkinsfile in my validation job to only run on Windows agents, and still only see the problem in the context of a GitHub Organization Folder.

          I'm sure you're seeing a problem, but I don't understand what is different between your environment and mine which makes it so that I can't see the problem you're seeing.

          Another possible difference: Is your master node running on WIndows? Mine is a docker instance and I haven't yet expended the effort to do docker on Windows

          Mark Waite added a comment - - edited medianick any chance that you've modified the PATH_MAX value (as described in the branch api pliugin )? That changes the workspace naming rules, and may be why you're seeing different results than I'm seeing. I modified the Jenkinsfile in my validation job to only run on Windows agents, and still only see the problem in the context of a GitHub Organization Folder. I'm sure you're seeing a problem, but I don't understand what is different between your environment and mine which makes it so that I can't see the problem you're seeing. Another possible difference: Is your master node running on WIndows? Mine is a docker instance and I haven't yet expended the effort to do docker on Windows

          Nick Jones added a comment -

          markewaite, I haven't modified PATH_MAX; it's still the default (80). The only thing I've done to accommodate the longer paths generated by Multibranch Pipelines is to set the core.longpaths setting for Git to true. I'm running Windows Server 2012 with everything Jenkins-related under a C:\Jenkins folder.

          Perhaps the difference is GitHub.com vs. GitHub Enterprise (which I'm using)?

          Nick Jones added a comment - markewaite , I haven't modified PATH_MAX; it's still the default (80). The only thing I've done to accommodate the longer paths generated by Multibranch Pipelines is to set the core.longpaths setting for Git to true. I'm running Windows Server 2012 with everything Jenkins-related under a C:\Jenkins folder. Perhaps the difference is GitHub.com vs. GitHub Enterprise (which I'm using)?

          Mark Waite added a comment -

          medianick I interpret your statement:

          I'm running Windows Server 2012 with everything Jenkins-related under a C:\Jenkins folder.

          to mean that your master is on Windows. If so, then that is the most likely cause of the difference between your case and mine.

          Mark Waite added a comment - medianick I interpret your statement: I'm running Windows Server 2012 with everything Jenkins-related under a C:\Jenkins folder. to mean that your master is on Windows. If so, then that is the most likely cause of the difference between your case and mine.

          Nick Jones added a comment -

          markwaite, yes, master and all build agents are on Windows.

          Nick Jones added a comment - markwaite , yes, master and all build agents are on Windows.

          Jesse Glick added a comment -

          The fix of JENKINS-34564 should be taking care of this. Are you sure you are not disabling it?

          Jesse Glick added a comment - The fix of  JENKINS-34564 should be taking care of this. Are you sure you are not disabling it?

          Mark Waite added a comment - - edited

          As far as I can tell from all my checks on my Docker image definition, I am not defining PATH_MAX to the java virtual machine. I don't think that I'm disabling the fix of JENKINS-34564.

          Jenkins properties for the Windows master (2.63) that I just installed show no setting for PATH_MAX. I started that master with the command java -jar jenkins.war. There were no additional command line arguments.

          See Jenkins-System-Information.pdf for detailed system info output from the Windows master where I confirmed that I can still see the problem.

          I've also confirmed on my Windows machine that multi-branch shows the same behavior (exactly as described by medianick). Thanks Nick. for your patience showing me that failure case!

          The following output appears in the command processor window where I started Jenkins:

          WARNING: Failed to parse URL for GitHubLink{iconClassName='icon-github-branch', url='https://github.com/MarkEWaite/jenkins-bugs/tree/has-percent-%-JENKINS-44360'}: java.net.URISyntaxException: Malformed escape pair at index 60: https://github.com/MarkEWaite/jenkins-bugs/tree/has-percent-%-JENKINS-44360
          

          Mark Waite added a comment - - edited As far as I can tell from all my checks on my Docker image definition, I am not defining PATH_MAX to the java virtual machine. I don't think that I'm disabling the fix of JENKINS-34564 . Jenkins properties for the Windows master (2.63) that I just installed show no setting for PATH_MAX. I started that master with the command java -jar jenkins.war . There were no additional command line arguments. See Jenkins-System-Information.pdf for detailed system info output from the Windows master where I confirmed that I can still see the problem. I've also confirmed on my Windows machine that multi-branch shows the same behavior (exactly as described by medianick ). Thanks Nick. for your patience showing me that failure case! The following output appears in the command processor window where I started Jenkins: WARNING: Failed to parse URL for GitHubLink{iconClassName='icon-github-branch', url='https://github.com/MarkEWaite/jenkins-bugs/tree/has-percent-%-JENKINS-44360'}: java.net.URISyntaxException: Malformed escape pair at index 60: https://github.com/MarkEWaite/jenkins-bugs/tree/has-percent-%-JENKINS-44360

          Nick Jones added a comment -

          In my case the console log shows this:

          java.io.FileNotFoundException: https://redacted/api/v3/repos/njones/TestRepo/git/refs/heads/test-%-builds
          	at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
          	at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
          	at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
          	at org.kohsuke.github.Requester.parse(Requester.java:596)
          	at org.kohsuke.github.Requester._to(Requester.java:262)
          Caused: java.io.FileNotFoundException: 

          with the remaining content being the HTML source of GitHub's standard "Looks like something went wrong!" page. And yes, in the Jenkins log itself, it has:

          Failed to parse URL for GitHubLink{iconClassName='icon-github-branch', url='https://redacted/njones/TestRepo/tree/test-%-builds'}: java.net.URISyntaxException: Malformed escape pair at index 55: https://redacted/njones/TestRepo/tree/test-%-builds

          It's exactly what you're seeing, markewaite.

          Nick Jones added a comment - In my case the console log shows this: java.io.FileNotFoundException: https://redacted/api/v3/repos/njones/TestRepo/git/refs/heads/test-%-builds at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:596) at org.kohsuke.github.Requester._to(Requester.java:262) Caused: java.io.FileNotFoundException: with the remaining content being the HTML source of GitHub's standard "Looks like something went wrong!" page. And yes, in the Jenkins log itself, it has: Failed to parse URL for GitHubLink{iconClassName='icon-github-branch', url='https://redacted/njones/TestRepo/tree/test-%-builds'}: java.net.URISyntaxException: Malformed escape pair at index 55: https://redacted/njones/TestRepo/tree/test-%-builds It's exactly what you're seeing, markewaite .

          Jesse Glick added a comment -

          You are using the GitHub branch source? If so, perhaps a bug in github-branch-source unrelated to the workspace path.

          Jesse Glick added a comment - You are using the GitHub branch source? If so, perhaps a bug in github-branch-source unrelated to the workspace path.

          I am suspecting it is an URL path encoding bug in github-api

          Stephen Connolly added a comment - I am suspecting it is an URL path encoding bug in github-api

          Alexandre Aubert added a comment - - edited

          Indeed we face also this issue and i dare changing the priority because MSBuild is not handling properly this '%2f' string in workspace full path.
          This blocks our builds even though our branches just respect the standard GitFlow automatically naming branches 'release/' or 'hotfix/' for example.

          Workaround with using ws() command to change folder name before running any build step could be applied but is really not a right solution.

          Thanks for your help on that.
          Regards,

          Alexandre Aubert added a comment - - edited Indeed we face also this issue and i dare changing the priority because MSBuild is not handling properly this '%2f' string in workspace full path. This blocks our builds even though our branches just respect the standard GitFlow automatically naming branches 'release/' or 'hotfix/' for example. Workaround with using ws() command to change folder name before running any build step could be applied but is really not a right solution. Thanks for your help on that. Regards,

          Jesse Glick added a comment -

          You must be running a really old build, or disabling the fix, because workspace paths have long since avoided %.

          Jesse Glick added a comment - You must be running a really old build, or disabling the fix, because workspace paths have long since avoided % .

          Mark Waite added a comment -

          splashnenen you are describing something different than this bug is describing. This bug describes the case where a literal '%' character is embedded in the name of the branch. This bug is not describing a case where a literal '/' is embedded in the name of the branch.

          As jglick noted, your case (literal '/' in the branch name) should work with standard settings, unless you have disabled the change which avoids URL escaping the branch name. I've changed the priority back to minor.

          Mark Waite added a comment - splashnenen you are describing something different than this bug is describing. This bug describes the case where a literal '%' character is embedded in the name of the branch. This bug is not describing a case where a literal '/' is embedded in the name of the branch. As jglick noted, your case (literal '/' in the branch name) should work with standard settings, unless you have disabled the change which avoids URL escaping the branch name. I've changed the priority back to minor.

          markewaite I indeed applied this configuration -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 , is it the one you are mentionning ? Should i remove that to properly handle '/' in branch names ?

          Thanks for your help.

          Alexandre Aubert added a comment - markewaite I indeed applied this configuration -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0 , is it the one you are mentionning ? Should i remove that to properly handle '/' in branch names ? Thanks for your help.

          Jesse Glick added a comment -

          Yes remove that property and your issue should go away.

          Jesse Glick added a comment - Yes remove that property and your issue should go away.

          Tyler Smith added a comment -

          I have the same property set (...PATH_MAX=0) as a work around for JENKINS-34564 since the multi-branch pipeline plugin will create long string folder names.

          Is there a way to have both? Or is the choice only between paths with percent escape sequences or long random folder paths?

          Right now I'm getting around the long folder name issue by pointing my multi-branch job to the SVN /branches/* folder directly rather than /* and filtering on branches. This causes other issues that I'd like to change, but it is working for now.

          Tyler Smith added a comment - I have the same property set (...PATH_MAX=0) as a work around for JENKINS-34564 since the multi-branch pipeline plugin will create long string folder names. Is there a way to have both? Or is the choice only between paths with percent escape sequences or long random folder paths? Right now I'm getting around the long folder name issue by pointing my multi-branch job to the SVN /branches/* folder directly rather than /* and filtering on branches. This causes other issues that I'd like to change, but it is working for now.

          Jesse Glick added a comment -

          Please read the https://plugins.jenkins.io/branch-api 1.11 changelog. Any discussion should be on the user’s list. It is off topic for this issue, which again relates to failure to check out branches containing %, and has nothing to do with workspace paths.

          Jesse Glick added a comment - Please read the https://plugins.jenkins.io/branch-api  1.11 changelog. Any discussion should be on the user’s list. It is off topic for this issue, which again relates to failure to check out branches containing % , and has nothing to do with workspace paths.

          Edgars Batna added a comment - - edited

          Can we just finally add some directory name filtering globally for workspaces? Keep only  alphanumerics and simple things like minus or underscore. Could be a per-project setting that's enabled for new jobs by default.

          Visual Studio builds are failing due to this and you can't really blame the build tool, because there's no telling nor proper handling for cases when paths get passed around, executed in shell/batch somewhere deep. Simple things stop working, such as plain simple scripts, not just build tools.

          Edgars Batna added a comment - - edited Can we just finally add some directory name filtering globally for workspaces? Keep only  alphanumerics and simple things like minus or underscore. Could be a per-project setting that's enabled for new jobs by default. Visual Studio builds are failing due to this and you can't really blame the build tool, because there's no telling nor proper handling for cases when paths get passed around, executed in shell/batch somewhere deep. Simple things stop working, such as plain simple scripts, not just build tools.

          Varun Reddy added a comment -

          is there a fix for this issue? Even after adding below we are still seeing issues with the workspace name. I am having this issue on Windows slave machines. Running .net project with msbuild commandline.

          -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0

          Varun Reddy added a comment - is there a fix for this issue? Even after adding below we are still seeing issues with the workspace name. I am having this issue on Windows slave machines. Running .net project with msbuild commandline. -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0

          Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Stephen Connolly added a comment - Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Jesse Glick added a comment -

          Likely obsolete as of JENKINS-2111.

          Jesse Glick added a comment - Likely obsolete as of JENKINS-2111 .

          Mark Waite added a comment - - edited

          I continue to see job failures when using a branch name containing '%' with either a GitHub organization folder or a GitHub branch source. I don't see those job failures when I use Git as a branch source, only if I use GitHub as the branch source or a GitHub organization folder.

          I haven't checked Bitbucket branch source or Gitea to see if the issue is specific to GitHub or is connected to something unrelated to the branch API plugin.

          The failing job writes the following surprising log file into build/1/log:

          Connecting to https://api.github.com using MarkEWaite/****** (MarkEWaite github username/password)
          java.io.FileNotFoundException: https://api.github.com/repos/MarkEWaite/jenkins-bugs/git/refs/heads/has-percent-%-JENKINS-44360
                  at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
                  at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
                  at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
                  at org.kohsuke.github.Requester.parse(Requester.java:625)
                  at org.kohsuke.github.Requester.parse(Requester.java:607)
                  at org.kohsuke.github.Requester._to(Requester.java:285)
          Caused: org.kohsuke.github.GHFileNotFoundException:
          ...  Long content that looks like an HTML file ...
                  at org.kohsuke.github.Requester.handleApiError(Requester.java:699)
                  at org.kohsuke.github.Requester._to(Requester.java:306)
                  at org.kohsuke.github.Requester.to(Requester.java:247)
                  at org.kohsuke.github.GHRepository.getRef(GHRepository.java:891)
                  at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1531)
                  at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)
                  at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:98)
                  at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293)
                  at hudson.model.ResourceController.execute(ResourceController.java:97)
                  at hudson.model.Executor.run(Executor.java:429)
          Finished: FAILURE
          

          I've uploaded build/1/log as JENKINS-44360.log

          The specific branch that shows the failure is in my jenkins-bugs GitHub repository.

          Mark Waite added a comment - - edited I continue to see job failures when using a branch name containing '%' with either a GitHub organization folder or a GitHub branch source. I don't see those job failures when I use Git as a branch source, only if I use GitHub as the branch source or a GitHub organization folder. I haven't checked Bitbucket branch source or Gitea to see if the issue is specific to GitHub or is connected to something unrelated to the branch API plugin. The failing job writes the following surprising log file into build/1/log : Connecting to https://api.github.com using MarkEWaite/****** (MarkEWaite github username/password) java.io.FileNotFoundException: https://api.github.com/repos/MarkEWaite/jenkins-bugs/git/refs/heads/has-percent-%-JENKINS-44360 at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:625) at org.kohsuke.github.Requester.parse(Requester.java:607) at org.kohsuke.github.Requester._to(Requester.java:285) Caused: org.kohsuke.github.GHFileNotFoundException: ... Long content that looks like an HTML file ... at org.kohsuke.github.Requester.handleApiError(Requester.java:699) at org.kohsuke.github.Requester._to(Requester.java:306) at org.kohsuke.github.Requester.to(Requester.java:247) at org.kohsuke.github.GHRepository.getRef(GHRepository.java:891) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1531) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:98) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILURE I've uploaded build/1/log as JENKINS-44360.log The specific branch that shows the failure is in my jenkins-bugs GitHub repository .

          Jesse Glick added a comment -

          markewaite you changed the Status to be IN REVIEW but there is no Assignee and I see no linked PR purporting to fix the issue.

          Perhaps you just meant to set this to Open, and change the component to github-branch-source-plugin since that seems to be the source of the problem?

          Jesse Glick added a comment - markewaite you changed the Status to be IN REVIEW but there is no Assignee and I see no linked PR purporting to fix the issue. Perhaps you just meant to set this to Open , and change the component to github-branch-source-plugin since that seems to be the source of the problem?

          Mark Waite added a comment - - edited

          Thanks jglick! I took too simple approach and chose the bug status as one of the proposed in the flow for next status, rather than setting it to the correct state of Open. Status has been corrected and reassigned to github-branch-source-plugin.

          Mark Waite added a comment - - edited Thanks jglick ! I took too simple approach and chose the bug status as one of the proposed in the flow for next status, rather than setting it to the correct state of Open . Status has been corrected and reassigned to github-branch-source-plugin.

            Unassigned Unassigned
            markewaite Mark Waite
            Votes:
            5 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: