Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38046

Gerrit Trigger Plugin Should be a Source for Multibranch Pipeline

      I want to use the new Multibranch Pipeline features of Jenkins with Gerrit source control.

      I should be able to choose Gerrit as the source for a multibranch pipeline. I should be able to select one or more repositories from Gerrit that each have their own Jenkinsfile. The pipeline would run the Jenkinsfile from the patch set that was pushed to Gerrit.

      There is no known workaround for this issue because one core requirement is to see each gerrit branch and change request as a different sub-job with its own history. This is essential because it allow you to know the build status of the branch instead of seeing a long queue of builds made on various branches and CRs, most likely full of failures.

      The behaviour described in this ticket is almost identical with the GitHub Multibranch implementation, the difference is that instead of having GitHub branches and PRs as the data source, we want to have Gerrit.

          [JENKINS-38046] Gerrit Trigger Plugin Should be a Source for Multibranch Pipeline

          Mike Kasberg created issue -
          Mike Kasberg made changes -
          Epic Link New: JENKINS-31152 [ 165808 ]

          Mike Kasberg added a comment - - edited

          Here is an (ugly) workaround:

          Use a normal pipeline job with the gerrit trigger. Have the pipeline execute a simple bootstrapping script, which loads the correct Jenkinsfile. The script might look like this:

          #!/usr/bin/env groovy
          
          node('label') {
              stage 'Checkout' {
                  // Checkout the patch set we're testing.
                  // See https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger#GerritTrigger-PipelineJobs
                  // Using Gerrit Trigger with Pipeline jobs.
          
                  git url: 'ssh://foo@gerrit/repository'
                      def changeBranch = "change-${GERRIT_CHANGE_NUMBER}-${GERRIT_PATCHSET_NUMBER}"
                      sh "git fetch origin ${GERRIT_REFSPEC}:${changeBranch}"
                      sh "git checkout ${changeBranch}"
                      sh "git clean -d -f -q -x"
              }
          
              // The rest comes from the repository itself!
              load 'Jenkinsfile'
          }
          

          Mike Kasberg added a comment - - edited Here is an (ugly) workaround: Use a normal pipeline job with the gerrit trigger. Have the pipeline execute a simple bootstrapping script, which loads the correct Jenkinsfile. The script might look like this: #!/usr/bin/env groovy node( 'label' ) { stage 'Checkout' { // Checkout the patch set we're testing. // See https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Trigger#GerritTrigger-PipelineJobs // Using Gerrit Trigger with Pipeline jobs. git url: 'ssh: //foo@gerrit/repository' def changeBranch = "change-${GERRIT_CHANGE_NUMBER}-${GERRIT_PATCHSET_NUMBER}" sh "git fetch origin ${GERRIT_REFSPEC}:${changeBranch}" sh "git checkout ${changeBranch}" sh "git clean -d -f -q -x" } // The rest comes from the repository itself! load 'Jenkinsfile' }
          Mike Kasberg made changes -
          Link New: This issue is duplicated by JENKINS-40389 [ JENKINS-40389 ]
          Stephen Connolly made changes -
          Labels Original: pipeline New: multibranch pipeline

          Austin Phillips added a comment - - edited

          Here's another workaround using a normal pipeline job with the gerrit trigger plugin.

          • Set up Gerrit triggers as per normal
          • Set up Pipeline with 'Pipeline script from SCM', using the Gerrit environment variables as source information for the Jenkinsfile
          • Use a Jenkinsfile in your source as normal, complete with 'checkout scm' step.

          Configuration of the Pipeline SCM is as follows:

          • Set 'Repository URL' to $GERRIT_SCHEME://$GERRIT_HOST:$GERRIT_PORT/$GERRIT_PROJECT
          • Click the 'Advanced' button and set 'Refspec' to $GERRIT_REFSPEC:$GERRIT_REFSPEC
          • Set 'Branches to build' to $GERRIT_REFSPEC
          • If desired, add 'Advanced clone behaviours' with 'Shallow clone' enabled with a 'Shallow clone depth' set to 1.
          • Untick 'Lightweight checkout'

          Then, in your Gerrit repository Jenkins file:

           

           node {
              stage('Checkout') {
                  checkout scm
             }
             stage('Build') {
                 echo "Building...'
             }
          }

           

          Additional information:

          • The advantage of this method is that there is only a single Jenkinsfile specification which lives in your SCM.
          • The Pipeline configuration can be consistent across multiple projects as all of the SCM information is derived from Gerrit Trigger environment variables.  The Gerrit Trigger parameters become the source for the Jenkinsfile pipeline definition.
          • The Jenkins master will check out a copy of the repository on the master to get access to the Jenkinsfile (ie Not in a workspace)
          • The 'checkout scm' step inside the pipeline will check out the code according to the SCM information entered in the pipeline step.  Because the checkout occurs inside a 'node' block, the source repo ends up being checked out twice.  Once to obtain the Jenkinsfile by the master, once as part of the Jenkinsfile pipeline step.  The shallow clone options may help speed things up here.
          • This has only been tested with the 'Patchset Created' Gerrit Trigger type.
          • The 'Lightweight checkout' options appears not to work due to the way that git is used for this option. Following error is generated: 
            hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/$GERRIT_REFSPEC:refs/remotes/origin/$GERRIT_REFSPEC --prune" returned status code 128:

           

          Austin Phillips added a comment - - edited Here's another workaround using a normal pipeline job with the gerrit trigger plugin. Set up Gerrit triggers as per normal Set up Pipeline with 'Pipeline script from SCM', using the Gerrit environment variables as source information for the Jenkinsfile Use a Jenkinsfile in your source as normal, complete with 'checkout scm' step. Configuration of the Pipeline SCM is as follows: Set 'Repository URL' to $GERRIT_SCHEME://$GERRIT_HOST:$GERRIT_PORT/$GERRIT_PROJECT Click the 'Advanced' button and set 'Refspec' to $GERRIT_REFSPEC:$GERRIT_REFSPEC Set 'Branches to build' to $GERRIT_REFSPEC If desired, add 'Advanced clone behaviours' with 'Shallow clone' enabled with a 'Shallow clone depth' set to 1. Untick 'Lightweight checkout' Then, in your Gerrit repository Jenkins file:    node {     stage('Checkout') {         checkout scm    } stage('Build') { echo "Building...'    } }   Additional information: The advantage of this method is that there is only a single Jenkinsfile specification which lives in your SCM. The Pipeline configuration can be consistent across multiple projects as all of the SCM information is derived from Gerrit Trigger environment variables.  The Gerrit Trigger parameters become the source for the Jenkinsfile pipeline definition. The Jenkins master will check out a copy of the repository on the master to get access to the Jenkinsfile (ie Not in a workspace) The 'checkout scm' step inside the pipeline will check out the code according to the SCM information entered in the pipeline step.  Because the checkout occurs inside a 'node' block, the source repo ends up being checked out twice.  Once to obtain the Jenkinsfile by the master, once as part of the Jenkinsfile pipeline step.  The shallow clone options may help speed things up here. This has only been tested with the 'Patchset Created' Gerrit Trigger type. The 'Lightweight checkout' options appears not to work due to the way that git is used for this option. Following error is generated:  hudson.plugins.git.GitException: Command "git fetch --tags --progress origin +refs/heads/$GERRIT_REFSPEC:refs/remotes/origin/$GERRIT_REFSPEC --prune" returned status code 128:  

          Sorin Sbarnea added a comment -

          Does anyone have a Jenkins instance where we can see this working? If that's not public, can we see some screenshots?

          I am really interested about this because currently we do build all gerrit patches for multiple branches using a single job and this makes impossible to get a reliable build status because builds from different branches are one after another so the build history is polluted.

          Ideally I would like to see something similar to the github multi-branch implementation where we can see the status of each branch and also the status of each pull-request.

          Can we achieve something similar? ... or at least something that would allow us to see status of each branch individually.

          Sorin Sbarnea added a comment - Does anyone have a Jenkins instance where we can see this working? If that's not public, can we see some screenshots? I am really interested about this because currently we do build all gerrit patches for multiple branches using a single job and this makes impossible to get a reliable build status because builds from different branches are one after another so the build history is polluted. Ideally I would like to see something similar to the github multi-branch implementation where we can see the status of each branch and also the status of each pull-request. Can we achieve something similar? ... or at least something that would allow us to see status of each branch individually.

          Mike Kasberg added a comment -

          The workarounds that have been posted will still use a single Jenkins Pipeline job on multiple branches, which will result in a polluted build history. I don't think what you're asking for will be possible until this feature is implemented correctly, unless someone else has figured out a better workaround that I don't know about.

          I do agree with you about the pains of history pollution - it would be great to see some progress on this feature.

          Mike Kasberg added a comment - The workarounds that have been posted will still use a single Jenkins Pipeline job on multiple branches, which will result in a polluted build history. I don't think what you're asking for will be possible until this feature is implemented correctly, unless someone else has figured out a better workaround that I don't know about. I do agree with you about the pains of history pollution - it would be great to see some progress on this feature.
          Sorin Sbarnea made changes -
          Description Original: I want to use the new Multibranch Pipeline features of Jenkins with Gerrit source control.

          I should be able to choose Gerrit as the source for a multibranch pipeline. I should be able to select one or more repositories from Gerrit that each have their own Jenkinsfile. The pipeline would run the Jenkinsfile from the patch set that was pushed to Gerrit.
          New: I want to use the new Multibranch Pipeline features of Jenkins with Gerrit source control.

          I should be able to choose Gerrit as the source for a multibranch pipeline. I should be able to select one or more repositories from Gerrit that each have their own Jenkinsfile. The pipeline would run the Jenkinsfile from the patch set that was pushed to Gerrit.

          There is *no known workaround* for this issue because one core requirement is to see *{color:#f79232}each gerrit branch and change request as a different build{color}* with its own history. This is essential because it allow you to know the build status of the branch instead of seeing a long queue of builds made on various branches and CRs, most likely full of failures.
          Sorin Sbarnea made changes -
          Description Original: I want to use the new Multibranch Pipeline features of Jenkins with Gerrit source control.

          I should be able to choose Gerrit as the source for a multibranch pipeline. I should be able to select one or more repositories from Gerrit that each have their own Jenkinsfile. The pipeline would run the Jenkinsfile from the patch set that was pushed to Gerrit.

          There is *no known workaround* for this issue because one core requirement is to see *{color:#f79232}each gerrit branch and change request as a different build{color}* with its own history. This is essential because it allow you to know the build status of the branch instead of seeing a long queue of builds made on various branches and CRs, most likely full of failures.
          New: I want to use the new Multibranch Pipeline features of Jenkins with Gerrit source control.

          I should be able to choose Gerrit as the source for a multibranch pipeline. I should be able to select one or more repositories from Gerrit that each have their own Jenkinsfile. The pipeline would run the Jenkinsfile from the patch set that was pushed to Gerrit.

          There is *no known workaround* for this issue because one core requirement is to see *{color:#f79232}each gerrit branch and change request as a different sub-job{color}* with its own history. This is essential because it allow you to know the build status of the branch instead of seeing a long queue of builds made on various branches and CRs, most likely full of failures.

          The behaviour described in this ticket is almost identical with the GitHub Multibranch implementation, the difference is that instead of having GitHub branches and PRs as the data source, we want to have Gerrit.

            lucamilanesio Luca Domenico Milanesio
            mkasberg Mike Kasberg
            Votes:
            48 Vote for this issue
            Watchers:
            57 Start watching this issue

              Created:
              Updated: