Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45688

Unable to set extensions for Git SCM Source via DSL

      In the past I used `gitSCMSource` to specify my git configuration with extensions in my Job DSL.
       
      Template I used:

      multibranchPipelineJob('${JOB_NAME}') {
          displayName('${JOB_NAME}')
          description('${JOB_DESCRIPTION}')
          orphanedItemStrategy {
              discardOldItems {
                  numToKeep(7)
              }
          }
          branchSources {
              branchSource {
                  source {
                      gitSCMSource {
                          id('')
                          remoteName('origin')
                          rawRefSpecs('')
                          ignoreOnPushNotifications(false)
                          remote('${JOB_GIT_URL}')
                          credentialsId('${GIT_CREDENTIALS_ID}')
                          includes('*')
                          excludes('')
                          extensions {
                              wipeWorkspace()
                              preBuildMerge {
                                  options {
                                      mergeRemote('origin')
                                      mergeTarget('${GIT_BRANCH}')
                                      mergeStrategy('default')
                                      fastForwardMode('FF')
                                  }
                              }
                          }
                      }
                  }
              }
          }
      }

      Now after updating to latest Jenkins/Plugins this no longer works. I looked at using the `git` source instead of `gitSCMSource` but it does not provide a way to set the extensions.

      In the new git-plugin 3.4.0 extensions are deprecated and are now traits. Not sure if this has anything to do with the issue.

          [JENKINS-45688] Unable to set extensions for Git SCM Source via DSL

          Mark Waite added a comment - - edited

          I suspect the fact that it worked before was not an intended use of the GitSCMSource API. stephenc will have more insights on the expectation of the use of the extensions from GitSCMSource.

          Does the technique in the official documentation work? It seems to be setting extensions using a git node in the DSL, without using GitSCMSource.

          Does the technique described in "use unsupported plugins with job DSL" still work? That technique does not seem to require GitSCMSource.

          A similar technique is described in "How to use Jenkins JobDSL to set 'Check out to specific local branch' in Git Plugin" , in "sparse checkout with jenkins dsl plugin" and in "use Jenkins DSL to specify a git executable in github SCM".

          I'm prone to close this as "Won't fix", but will wait for verification from stephenconnolly

          Mark Waite added a comment - - edited I suspect the fact that it worked before was not an intended use of the GitSCMSource API. stephenc will have more insights on the expectation of the use of the extensions from GitSCMSource. Does the technique in the official documentation work? It seems to be setting extensions using a git node in the DSL, without using GitSCMSource. Does the technique described in " use unsupported plugins with job DSL " still work? That technique does not seem to require GitSCMSource. A similar technique is described in " How to use Jenkins JobDSL to set 'Check out to specific local branch' in Git Plugin " , in " sparse checkout with jenkins dsl plugin " and in " use Jenkins DSL to specify a git executable in github SCM ". I'm prone to close this as "Won't fix", but will wait for verification from stephenconnolly

          So my understanding is that you need to use the techniques for using unsupported plugins with job DSL to configure traits... at least until we get around to adding the @Symbol annotations (JENKINS-45504)... of course that is currently blocked on JENKINS-45503 because for reasons documented in those issues.

          Having said all that, we did retain the setter for extensions (just marked deprecated is all) and that setter will create the matching traits, so the only thing I can think is that you may be hitting the ref trimming... in which case all you need to do is add the trait that allows specifying an additional refspec to fetch and ensure that the master branch refspec is always fetched from the remote so that the merge extension / trait can work...

          markewaite I think we need to hear back on how this "no longer works" to determine if:

          • The jobs are not getting configured correctly; or
          • The configured jobs are failing because of the missing refspec for origin; or
          • Something else

          Stephen Connolly added a comment - So my understanding is that you need to use the techniques for using unsupported plugins with job DSL to configure traits... at least until we get around to adding the @Symbol annotations ( JENKINS-45504 )... of course that is currently blocked on JENKINS-45503 because for reasons documented in those issues. Having said all that, we did retain the setter for extensions (just marked deprecated is all) and that setter will create the matching traits, so the only thing I can think is that you may be hitting the ref trimming... in which case all you need to do is add the trait that allows specifying an additional refspec to fetch and ensure that the master branch refspec is always fetched from the remote so that the merge extension / trait can work... markewaite I think we need to hear back on how this "no longer works" to determine if: The jobs are not getting configured correctly; or The configured jobs are failing because of the missing refspec for origin; or Something else

          Art V added a comment -

          Using this example:

          multibranchPipelineJob('AMQ-Docker-Branch') {
            displayName('AMQ-Docker-Branch')
            description('AMQ Docker Container Branch Builds')
            orphanedItemStrategy {
              discardOldItems {
                numToKeep(7)
              }
            }
            branchSources {
              git {
                id('')
                ignoreOnPushNotifications(false)
                remote('git@git.com/amq-docker.git')
                credentialsId('SSHKeyCredentials')
                includes('*')
                excludes('master')
                extensions {
                  wipeWorkspace()
                  preBuildMerge {
                    options {
                      mergeRemote('origin')
                      mergeTarget('master')
                      mergeStrategy('default')
                      fastForwardMode('FF')
                    }
                  }
                }
              }
            }
          }
          

          I get this error:

          ERROR: (script, line 17) No signature of method: javaposse.jobdsl.dsl.helpers.workflow.GitBranchSourceContext.extensions() is applicable for argument types: (script$_run_closure1$_closure3$_closure5$_closure6) values: [script$_run_closure1$_closure3$_closure5$_closure6@5218bc64]
          Finished: FAILURE
          

          If I try to do the unsupported approach - I am using this sample, but it does not create the extensions.

          multibranchPipelineJob('CI-TEST-MB') {
            displayName('CI-TEST-MB')
            description('Test Multibranch Job')
            orphanedItemStrategy {
              discardOldItems {
                numToKeep(7)
              }
            }
          branchSources {
            git {
              id('')
              ignoreOnPushNotifications(false)
              remote('git@git.com/test.git')
              credentialsId('test')
              includes('*')
              excludes('master')
              configure { node -> //the GitSCM node is passed in
          
                  // Add the WipeWorkspace functionality
                  node / 'extensions' << 'hudson.plugins.git.extensions.impl.WipeWorkspace'  {
                  }
                }
              }
            }
          }
          

           See attachements for result..

           

          Art V added a comment - Using this example: multibranchPipelineJob( 'AMQ-Docker-Branch' ) { displayName( 'AMQ-Docker-Branch' ) description( 'AMQ Docker Container Branch Builds' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { git { id('') ignoreOnPushNotifications( false ) remote( 'git@git.com/amq-docker.git' ) credentialsId( 'SSHKeyCredentials' ) includes( '*' ) excludes( 'master' ) extensions { wipeWorkspace() preBuildMerge { options { mergeRemote( 'origin' ) mergeTarget( 'master' ) mergeStrategy( ' default ' ) fastForwardMode( 'FF' ) } } } } } } I get this error: ERROR: (script, line 17) No signature of method: javaposse.jobdsl.dsl.helpers.workflow.GitBranchSourceContext.extensions() is applicable for argument types: (script$_run_closure1$_closure3$_closure5$_closure6) values: [script$_run_closure1$_closure3$_closure5$_closure6@5218bc64] Finished: FAILURE If I try to do the unsupported approach - I am using this sample, but it does not create the extensions. multibranchPipelineJob( 'CI-TEST-MB' ) { displayName( 'CI-TEST-MB' ) description( 'Test Multibranch Job' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { git { id('') ignoreOnPushNotifications( false ) remote( 'git@git.com/test.git' ) credentialsId( 'test' ) includes( '*' ) excludes( 'master' ) configure { node -> //the GitSCM node is passed in // Add the WipeWorkspace functionality node / 'extensions' << 'hudson.plugins.git.extensions.impl.WipeWorkspace' { } } } } }  See attachements for result..  

          Art V added a comment -

          With my latest testing using the configure block - I noticed the node reference is always referencing the root xml node, not the current node. I will continue to test/configure using the job dsl config block and will let you know what I find.

          Art V added a comment - With my latest testing using the configure block - I noticed the node reference is always referencing the root xml node, not the current node. I will continue to test/configure using the job dsl config block and will let you know what I find.

          Art V added a comment -

          A hackish workaround using the configure block (bypassing git block entirely)..

          multibranchPipelineJob('CI-TEST-MB') {
            displayName('CI-TEST-MB')
            description('Test Multibranch Job')
            orphanedItemStrategy {
              discardOldItems {
                numToKeep(7)
              }
            }
            branchSources {
              configure { node ->
                node / sources(class: 'jenkins.branch.MultiBranchProject$BranchSourceList') / data / 'jenkins.branch.BranchSource' / source(class: 'jenkins.plugins.git.GitSCMSource') {
                  remote 'git@git.com/test.git'
                  credentialsId 'test'
                  includes '*'
                  excludes 'master'
                  ignoreOnPushNotifications 'false'
                  id ''
                  traits {
                    'jenkins.plugins.git.traits.BranchDiscoveryTrait'()
                    'jenkins.plugins.git.traits.WipeWorkspaceTrait'() {
                      extension(class: 'hudson.plugins.git.extensions.impl.WipeWorkspace')
                    }
                    'jenkins.plugins.git.traits.PreBuildMergeTrait'() {
                      extension(class: 'hudson.plugins.git.extensions.impl.PreBuildMerge') {
                        options {
                          mergeRemote 'origin'
                          mergeTarget 'master'
                          mergeStrategy 'default'
                          fastForwardMode 'FF'
                        }
                      }
                    }
                  }
                }
              }
            }
          }
          

           

          Reason I removed the git {} section entirely was because it would always add a new record of the 'jenkins.branch.BranchSource' section (not updating the existing BranchSource node)

          This might be a separate bug (or misunderstood documentation) If I put the configure block inside the git{} section, the reference 'node' would always point to the root xml element not the current node/git element. 
          Example from documentation..

          job('example') {
              scm {
                  git {
                      remote {
                          url('git@server:account/repo1.git')
                      }
                      configure { node ->
                          // node represents <hudson.plugins.git.GitSCM>
                      }
                  }
              }
          }
          

           

          Art V added a comment - A hackish workaround using the configure block (bypassing git block entirely).. multibranchPipelineJob( 'CI-TEST-MB' ) { displayName( 'CI-TEST-MB' ) description( 'Test Multibranch Job' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { configure { node -> node / sources(class: 'jenkins.branch.MultiBranchProject$BranchSourceList' ) / data / 'jenkins.branch.BranchSource' / source(class: 'jenkins.plugins.git.GitSCMSource' ) { remote 'git@git.com/test.git' credentialsId 'test' includes '*' excludes 'master' ignoreOnPushNotifications ' false ' id '' traits { 'jenkins.plugins.git.traits.BranchDiscoveryTrait' () 'jenkins.plugins.git.traits.WipeWorkspaceTrait' () { extension(class: 'hudson.plugins.git.extensions.impl.WipeWorkspace' ) } 'jenkins.plugins.git.traits.PreBuildMergeTrait' () { extension(class: 'hudson.plugins.git.extensions.impl.PreBuildMerge' ) { options { mergeRemote 'origin' mergeTarget 'master' mergeStrategy ' default ' fastForwardMode 'FF' } } } } } } } }   Reason I removed the git {} section entirely was because it would always add a new record of the 'jenkins.branch.BranchSource' section (not updating the existing BranchSource node) This might be a separate bug (or misunderstood documentation) If I put the configure block inside the git{} section, the reference 'node' would always point to the root xml element not the current node/git element.  Example from documentation.. job( 'example' ) { scm { git { remote { url( 'git@server:account/repo1.git' ) } configure { node -> // node represents <hudson.plugins.git.GitSCM> } } } }  

          I can not reproduce this with all plugins updated to the latest version. I see multibranchPipelineJob > branchSources > branchSource > source > git > extensions as well as multibranchPipelineJob > branchSources > branchSource > source > git > traits in my embedded API viewer. Can you create a working configuration manually and post the config XML? Then we can figure out what's missing in Job DSL.

          Daniel Spilker added a comment - I can not reproduce this with all plugins updated to the latest version. I see multibranchPipelineJob > branchSources > branchSource > source > git > extensions as well as multibranchPipelineJob > branchSources > branchSource > source > git > traits in my embedded API viewer. Can you create a working configuration manually and post the config XML? Then we can figure out what's missing in Job DSL.

          Sam Gleske added a comment - - edited

          daspilker I encountered this issue after installing blue ocean. Blue ocean (for some strange reason) requires bit bucket branch source and github branch source plugins to operate. With those plugins installed one faces the issue when trying to configure the git plugin as a branch source.

          When I manually disable those two plugins (breaking blue ocean) I get a working DSL again.

          Sam Gleske added a comment - - edited daspilker I encountered this issue after installing blue ocean. Blue ocean (for some strange reason) requires bit bucket branch source and github branch source plugins to operate. With those plugins installed one faces the issue when trying to configure the git plugin as a branch source. When I manually disable those two plugins (breaking blue ocean) I get a working DSL again.

          Sam Gleske added a comment -

          Here's a link to the DSL used.

          https://github.com/gimp-ci/jenkins-dsl/blob/292cafd41e8c26e200dbe967aba7ec71e120df84/jobs/gimp_multibranch_pipelines.groovy

          Here's a list of plugins installed that broke the the DSL. https://github.com/gimp-ci/jenkins-os-packages/commit/2e9e0e6b5a2e0287268ca70d836f2a7e71fef72b

          In particular I confirmed the following plugins broke the DSL.

          • org.jenkins-ci.plugins:cloudbees-bitbucket-branch-source:2.2.11
          • org.jenkins-ci.plugins:github-branch-source:2.3.4

          Sam Gleske added a comment - Here's a link to the DSL used. https://github.com/gimp-ci/jenkins-dsl/blob/292cafd41e8c26e200dbe967aba7ec71e120df84/jobs/gimp_multibranch_pipelines.groovy Here's a list of plugins installed that broke the the DSL. https://github.com/gimp-ci/jenkins-os-packages/commit/2e9e0e6b5a2e0287268ca70d836f2a7e71fef72b In particular I confirmed the following plugins broke the DSL. org.jenkins-ci.plugins:cloudbees-bitbucket-branch-source:2.2.11 org.jenkins-ci.plugins:github-branch-source:2.3.4

          Sam Gleske added a comment -

          This nonsense requiring the configure block is ridiculous.  I've spent the past 5 hours trying to figure it out and I'm definitely not a Job DSL newb.

          Sam Gleske added a comment - This nonsense requiring the configure block is ridiculous.  I've spent the past 5 hours trying to figure it out and I'm definitely not a Job DSL newb.

          sag47 I am sure this is not by intention and the maintainers will welcome your pull requests to stop this nonsense.

          Steffen Gebert added a comment - sag47 I am sure this is not by intention and the maintainers will welcome your pull requests to stop this nonsense.

          Sam Gleske added a comment - - edited

          stephenking my comments have added a lot of value to this issue by referencing both source code, examples, and reproduction.  Your comment claims I should stop doing this.  I disagree.

          Edit: I think I initially misread your comment.  In any case, "contribute code" should never be the default retort as a comeback in my opinion.

          Sam Gleske added a comment - - edited stephenking my comments have added a lot of value to this issue by referencing both source code, examples, and reproduction.  Your comment claims I should stop doing this.  I disagree. Edit: I think I initially misread your comment.  In any case, "contribute code" should never be the default retort as a comeback in my opinion.

          No, my point was "please be friendly". Nobody creates "nonsense" by intention.

          Steffen Gebert added a comment - No, my point was "please be friendly". Nobody creates "nonsense" by intention.

          Sam Gleske added a comment -

          Sure, I can put my frustration in check.  My intent wasn't to be hostile nor to imply it was done on purpose (it would be odd to assume that it's on purpose).

          Sam Gleske added a comment - Sure, I can put my frustration in check.  My intent wasn't to be hostile nor to imply it was done on purpose (it would be odd to assume that it's on purpose).

          Sam Gleske added a comment -

          Added git-plugin since the git-plugin should define "@Symbol" for its classes to help with the DSL generation to be unique.

          Sam Gleske added a comment - Added git-plugin since the git-plugin should define "@Symbol" for its classes to help with the DSL generation to be unique.

          Sam Gleske added a comment -

          Sam Gleske added a comment - https://github.com/jenkinsci/git-plugin/pull/595

          sag47, thanks for bringing this forward!

          Steffen Gebert added a comment - sag47 , thanks for bringing this forward!

          Sam Gleske added a comment -

          You're welcome.

          Sam Gleske added a comment - You're welcome.

            Unassigned Unassigned
            arty13 Art V
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: