Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-44360

'%' in branch name causes GitHub multi-branch job failures

      A '%' character in a branch name breaks the GitHub API calls on a git repository if the job performing the clone is generated by GitHub Organization Folder plugin (windows or Linux) or the GitHub branch source (windows or Linux).

      Steps to duplicate the problem:

      1. Define a GitHub multibranch PIpeline job referencing the MarkEWaite/jenkins-bugs repository (or a GitHub Organization Folders job which references the GitHub organization MarkEWaite, looking only at the repository jenkins-bugs)
      2. Scan the repository, watch the jobs run
      3. Open the has-percent-%-JENKINS-44360 job and confirm it failed to clone

      Steps to show the same branch working with a multibranch pipeline:

      1. Define a multi-branch pipeline job using the MarkEWaite/jenkins-bugs github repo using Git as the branch source rather than GitHub
      2. Scan the repository, watch the jobs run
      3. Open the has-percent-%-JENKINS-44360 job and confirm it cloned successfully

      The issue seems to require:

      • GitHub Organization Folders or GitHub multibranch
      • '%' in the branch name on the repository

          [JENKINS-44360] '%' in branch name causes GitHub multi-branch job failures

          Jesse Glick added a comment -

          Yes remove that property and your issue should go away.

          Jesse Glick added a comment - Yes remove that property and your issue should go away.

          Tyler Smith added a comment -

          I have the same property set (...PATH_MAX=0) as a work around for JENKINS-34564 since the multi-branch pipeline plugin will create long string folder names.

          Is there a way to have both? Or is the choice only between paths with percent escape sequences or long random folder paths?

          Right now I'm getting around the long folder name issue by pointing my multi-branch job to the SVN /branches/* folder directly rather than /* and filtering on branches. This causes other issues that I'd like to change, but it is working for now.

          Tyler Smith added a comment - I have the same property set (...PATH_MAX=0) as a work around for JENKINS-34564 since the multi-branch pipeline plugin will create long string folder names. Is there a way to have both? Or is the choice only between paths with percent escape sequences or long random folder paths? Right now I'm getting around the long folder name issue by pointing my multi-branch job to the SVN /branches/* folder directly rather than /* and filtering on branches. This causes other issues that I'd like to change, but it is working for now.

          Jesse Glick added a comment -

          Please read the https://plugins.jenkins.io/branch-api 1.11 changelog. Any discussion should be on the user’s list. It is off topic for this issue, which again relates to failure to check out branches containing %, and has nothing to do with workspace paths.

          Jesse Glick added a comment - Please read the https://plugins.jenkins.io/branch-api  1.11 changelog. Any discussion should be on the user’s list. It is off topic for this issue, which again relates to failure to check out branches containing % , and has nothing to do with workspace paths.

          Edgars Batna added a comment - - edited

          Can we just finally add some directory name filtering globally for workspaces? Keep only  alphanumerics and simple things like minus or underscore. Could be a per-project setting that's enabled for new jobs by default.

          Visual Studio builds are failing due to this and you can't really blame the build tool, because there's no telling nor proper handling for cases when paths get passed around, executed in shell/batch somewhere deep. Simple things stop working, such as plain simple scripts, not just build tools.

          Edgars Batna added a comment - - edited Can we just finally add some directory name filtering globally for workspaces? Keep only  alphanumerics and simple things like minus or underscore. Could be a per-project setting that's enabled for new jobs by default. Visual Studio builds are failing due to this and you can't really blame the build tool, because there's no telling nor proper handling for cases when paths get passed around, executed in shell/batch somewhere deep. Simple things stop working, such as plain simple scripts, not just build tools.

          Varun Reddy added a comment -

          is there a fix for this issue? Even after adding below we are still seeing issues with the workspace name. I am having this issue on Windows slave machines. Running .net project with msbuild commandline.

          -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0

          Varun Reddy added a comment - is there a fix for this issue? Even after adding below we are still seeing issues with the workspace name. I am having this issue on Windows slave machines. Running .net project with msbuild commandline. -Djenkins.branch.WorkspaceLocatorImpl.PATH_MAX=0

          Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Stephen Connolly added a comment - Removing myself as assignee. My current work assignments do not provide sufficient bandwidth to review these issues and in the majority of cases I am only assigned by virtue of being the default assignee. For the credentials-api and scm-api related plugins I have permission to allocate time reviewing changes to these APIs themselves to ensure these APIs remain cohesive, but that can be handled through PR reviews rather than assigning issues in JIRA

          Jesse Glick added a comment -

          Likely obsolete as of JENKINS-2111.

          Jesse Glick added a comment - Likely obsolete as of JENKINS-2111 .

          Mark Waite added a comment - - edited

          I continue to see job failures when using a branch name containing '%' with either a GitHub organization folder or a GitHub branch source. I don't see those job failures when I use Git as a branch source, only if I use GitHub as the branch source or a GitHub organization folder.

          I haven't checked Bitbucket branch source or Gitea to see if the issue is specific to GitHub or is connected to something unrelated to the branch API plugin.

          The failing job writes the following surprising log file into build/1/log:

          Connecting to https://api.github.com using MarkEWaite/****** (MarkEWaite github username/password)
          java.io.FileNotFoundException: https://api.github.com/repos/MarkEWaite/jenkins-bugs/git/refs/heads/has-percent-%-JENKINS-44360
                  at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243)
                  at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210)
                  at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25)
                  at org.kohsuke.github.Requester.parse(Requester.java:625)
                  at org.kohsuke.github.Requester.parse(Requester.java:607)
                  at org.kohsuke.github.Requester._to(Requester.java:285)
          Caused: org.kohsuke.github.GHFileNotFoundException:
          ...  Long content that looks like an HTML file ...
                  at org.kohsuke.github.Requester.handleApiError(Requester.java:699)
                  at org.kohsuke.github.Requester._to(Requester.java:306)
                  at org.kohsuke.github.Requester.to(Requester.java:247)
                  at org.kohsuke.github.GHRepository.getRef(GHRepository.java:891)
                  at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1531)
                  at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582)
                  at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:98)
                  at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293)
                  at hudson.model.ResourceController.execute(ResourceController.java:97)
                  at hudson.model.Executor.run(Executor.java:429)
          Finished: FAILURE
          

          I've uploaded build/1/log as JENKINS-44360.log

          The specific branch that shows the failure is in my jenkins-bugs GitHub repository.

          Mark Waite added a comment - - edited I continue to see job failures when using a branch name containing '%' with either a GitHub organization folder or a GitHub branch source. I don't see those job failures when I use Git as a branch source, only if I use GitHub as the branch source or a GitHub organization folder. I haven't checked Bitbucket branch source or Gitea to see if the issue is specific to GitHub or is connected to something unrelated to the branch API plugin. The failing job writes the following surprising log file into build/1/log : Connecting to https://api.github.com using MarkEWaite/****** (MarkEWaite github username/password) java.io.FileNotFoundException: https://api.github.com/repos/MarkEWaite/jenkins-bugs/git/refs/heads/has-percent-%-JENKINS-44360 at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:625) at org.kohsuke.github.Requester.parse(Requester.java:607) at org.kohsuke.github.Requester._to(Requester.java:285) Caused: org.kohsuke.github.GHFileNotFoundException: ... Long content that looks like an HTML file ... at org.kohsuke.github.Requester.handleApiError(Requester.java:699) at org.kohsuke.github.Requester._to(Requester.java:306) at org.kohsuke.github.Requester.to(Requester.java:247) at org.kohsuke.github.GHRepository.getRef(GHRepository.java:891) at org.jenkinsci.plugins.github_branch_source.GitHubSCMSource.retrieve(GitHubSCMSource.java:1531) at jenkins.scm.api.SCMSource.fetch(SCMSource.java:582) at org.jenkinsci.plugins.workflow.multibranch.SCMBinder.create(SCMBinder.java:98) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:293) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILURE I've uploaded build/1/log as JENKINS-44360.log The specific branch that shows the failure is in my jenkins-bugs GitHub repository .

          Jesse Glick added a comment -

          markewaite you changed the Status to be IN REVIEW but there is no Assignee and I see no linked PR purporting to fix the issue.

          Perhaps you just meant to set this to Open, and change the component to github-branch-source-plugin since that seems to be the source of the problem?

          Jesse Glick added a comment - markewaite you changed the Status to be IN REVIEW but there is no Assignee and I see no linked PR purporting to fix the issue. Perhaps you just meant to set this to Open , and change the component to github-branch-source-plugin since that seems to be the source of the problem?

          Mark Waite added a comment - - edited

          Thanks jglick! I took too simple approach and chose the bug status as one of the proposed in the flow for next status, rather than setting it to the correct state of Open. Status has been corrected and reassigned to github-branch-source-plugin.

          Mark Waite added a comment - - edited Thanks jglick ! I took too simple approach and chose the bug status as one of the proposed in the flow for next status, rather than setting it to the correct state of Open . Status has been corrected and reassigned to github-branch-source-plugin.

            Unassigned Unassigned
            markewaite Mark Waite
            Votes:
            5 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated: