Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41121

GitHub Branch Source upgrade can cause a lot of rebuilds

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • scm-api-plugin
    • None

      The gist of this is that due to some encoding change made in the SCM API 2.0 release, conventional URIs

      This was originally noticed by the Blue Ocean team in this pull request discussing integrating SCM API 2.0 changes. See also JENKINS-40299.

      The blog post discusses some of the data migrations which are supposed to take place, however the URI changing issue results in two bad behaviors:

      * URIs change for Multibranch-based Pipelines which use "special characters" in their names, such as improvement/JENKINS-41047 would previously be located at: https://ci.blueocean.io/job/blueocean/job/improvement%2FJENKINS-41047/. After these changes it would be found at https://ci.blueocean.io/job/blueocean/job/improvement-JENKINS-41047.d61e72/
      * Post-upgrade, rebooting the Jenkins installation results in *every* Multibranch Pipeline being retriggered. Such as this Pipeline despite no new changes being made.


      Because the upgrade to SCM API 2.0 is a one-direction-only change, I propose we blacklist the updated versions of the plugins referenced in this post before too many people upgrade a non-reversible upgrade.

      Incorrect analysis with lots of FUD.

      The renaming of jobs has nothing to do here.

      The issue here is that GitHub Branch Source 1.x passes a null id to the constructor of it's SCMSource: https://github.com/jenkinsci/github-branch-source-plugin/blob/1.10-stable/src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMNavigator.java#L312

      This means that every time you perform a scan of the organization folder, all the SCMSource instances get a new ID.

      When the branch changes SCMSource instance (i.e. the ID is different) then the Branch API is supposed to trigger a rebuild (because the branch is from a different source, never mind that it may be the same revision, we have no way of knowing)

      In 2.x we have a consistent ID: https://github.com/jenkinsci/github-branch-source-plugin/blob/master/src/main/java/org/jenkinsci/plugins/github_branch_source/GitHubSCMNavigator.java#L446

      The issue here is that GitHub Branch Source did not correctly ensure that the SCMSource from a Organization Folder had a consistent ID.

      Thus, every time you do a scan of an organization folder, with the old logic it should have been re-triggering a build... It wasn't because of other bugs... this leads to things like "I cannot re-trigger PRs", etc

      In fixing the problems in GitHub Branch Source, we get to a point where everything is behaving correctly, but the data for the previous branch origins is not current and thus the PRs have not been built from their current source.

          [JENKINS-41121] GitHub Branch Source upgrade can cause a lot of rebuilds

          R. Tyler Croy created issue -

          Code changed in jenkins
          User: R. Tyler Croy
          Path:
          src/main/resources/artifact-ignores.properties
          http://jenkins-ci.org/commit/backend-update-center2/b52d4f68547e2d09d50cc4127d37a153622b6c85
          Log:
          Ignore SCM API 2.0 related changes until the cause of JENKINS-41121

          The basic idea of this is that the upgrades are sufficiently dangerous and *one
          direction* (i.e. no rollback except for restoring from a pre-upgrade backup of
          JENKINS_HOME) that it's best for Jenkins users to hide these versions until this
          is figured out and made more safe.

          See https://jenkins.io/blog/2017/01/17/scm-api-2/ for the list of affected
          plugins.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: src/main/resources/artifact-ignores.properties http://jenkins-ci.org/commit/backend-update-center2/b52d4f68547e2d09d50cc4127d37a153622b6c85 Log: Ignore SCM API 2.0 related changes until the cause of JENKINS-41121 The basic idea of this is that the upgrades are sufficiently dangerous and *one direction* (i.e. no rollback except for restoring from a pre-upgrade backup of JENKINS_HOME) that it's best for Jenkins users to hide these versions until this is figured out and made more safe. See https://jenkins.io/blog/2017/01/17/scm-api-2/ for the list of affected plugins.
          R. Tyler Croy made changes -
          Link New: This issue is related to JENKINS-40299 [ JENKINS-40299 ]
          R. Tyler Croy made changes -
          Summary Original: New scm-api versions changes branch name encoding breaking URIs and causing data loss New: New scm-api versions changes branch name encoding breaking URIs and DoSing

          R. Tyler Croy added a comment -

          Based on michaelneale/'s comment here, webhook triggering is broken for any branch with a "weird" character in it.

          R. Tyler Croy added a comment - Based on michaelneale /'s comment here , webhook triggering is broken for any branch with a "weird" character in it.
          R. Tyler Croy made changes -
          Summary Original: New scm-api versions changes branch name encoding breaking URIs and DoSing New: New scm-api versions changes branch name encoding breaking URIs and can overload the environment

          Code changed in jenkins
          User: R. Tyler Croy
          Path:
          content/blog/2017/2017-01-17-scm-api-2.0-release.adoc
          http://jenkins-ci.org/commit/jenkins.io/cceffabdd93994ec1d9f974e67fb927fb50ae496
          Log:
          Advise against upgrading plugins until JENKINS-41121 is resolved

          This commit should be reverted when the fixes hit the update center

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/blog/2017/2017-01-17-scm-api-2.0-release.adoc http://jenkins-ci.org/commit/jenkins.io/cceffabdd93994ec1d9f974e67fb927fb50ae496 Log: Advise against upgrading plugins until JENKINS-41121 is resolved This commit should be reverted when the fixes hit the update center

          Code changed in jenkins
          User: R. Tyler Croy
          Path:
          content/blog/2017/2017-01-17-scm-api-2.0-release.adoc
          http://jenkins-ci.org/commit/jenkins.io/beac68e9fb301b1b3a6993ae066f44c16d5486a8
          Log:
          Merge pull request #550 from rtyler/41121

          Advise against upgrading plugins until JENKINS-41121 is resolved

          Compare: https://github.com/jenkins-infra/jenkins.io/compare/7dfbc443c845...beac68e9fb30

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: R. Tyler Croy Path: content/blog/2017/2017-01-17-scm-api-2.0-release.adoc http://jenkins-ci.org/commit/jenkins.io/beac68e9fb301b1b3a6993ae066f44c16d5486a8 Log: Merge pull request #550 from rtyler/41121 Advise against upgrading plugins until JENKINS-41121 is resolved Compare: https://github.com/jenkins-infra/jenkins.io/compare/7dfbc443c845...beac68e9fb30
          R. Tyler Croy made changes -
          Assignee Original: R. Tyler Croy [ rtyler ]
          Stephen Connolly made changes -
          Summary Original: New scm-api versions changes branch name encoding breaking URIs and can overload the environment New: GitHub Branch Source upgrade can cause a lot of rebuilds

            stephenconnolly Stephen Connolly
            rtyler R. Tyler Croy
            Votes:
            4 Vote for this issue
            Watchers:
            31 Start watching this issue

              Created:
              Updated:
              Resolved: