Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45688

Unable to set extensions for Git SCM Source via DSL

XMLWordPrintable

      In the past I used `gitSCMSource` to specify my git configuration with extensions in my Job DSL.
       
      Template I used:

      multibranchPipelineJob('${JOB_NAME}') {
    displayName('${JOB_NAME}')
    description('${JOB_DESCRIPTION}')
    orphanedItemStrategy {
        discardOldItems {
            numToKeep(7)
        }
    }
    branchSources {
        branchSource {
            source {
                gitSCMSource {
                    id('')
                    remoteName('origin')
                    rawRefSpecs('')
                    ignoreOnPushNotifications(false)
                    remote('${JOB_GIT_URL}')
                    credentialsId('${GIT_CREDENTIALS_ID}')
                    includes('*')
                    excludes('')
                    extensions {
                        wipeWorkspace()
                        preBuildMerge {
                            options {
                                mergeRemote('origin')
                                mergeTarget('${GIT_BRANCH}')
                                mergeStrategy('default')
                                fastForwardMode('FF')
                            }
                        }
                    }
                }
            }
        }
    }
}

      Now after updating to latest Jenkins/Plugins this no longer works. I looked at using the `git` source instead of `gitSCMSource` but it does not provide a way to set the extensions.

      In the new git-plugin 3.4.0 extensions are deprecated and are now traits. Not sure if this has anything to do with the issue.

        1. Screen Shot 2017-07-21 at 09.36.06.png
          Screen Shot 2017-07-21 at 09.36.06.png
          66 kB
        2. Screen Shot 2017-07-21 at 09.36.26.png
          Screen Shot 2017-07-21 at 09.36.26.png
          87 kB

          [JENKINS-45688] Unable to set extensions for Git SCM Source via DSL

          Art V created issue -

          Mark Waite added a comment - - edited

          I suspect the fact that it worked before was not an intended use of the GitSCMSource API. stephenc will have more insights on the expectation of the use of the extensions from GitSCMSource.

          Does the technique in the official documentation work? It seems to be setting extensions using a git node in the DSL, without using GitSCMSource.

          Does the technique described in "use unsupported plugins with job DSL" still work? That technique does not seem to require GitSCMSource.

          A similar technique is described in "How to use Jenkins JobDSL to set 'Check out to specific local branch' in Git Plugin" , in "sparse checkout with jenkins dsl plugin" and in "use Jenkins DSL to specify a git executable in github SCM".

          I'm prone to close this as "Won't fix", but will wait for verification from stephenconnolly

          Mark Waite added a comment - - edited I suspect the fact that it worked before was not an intended use of the GitSCMSource API. stephenc will have more insights on the expectation of the use of the extensions from GitSCMSource. Does the technique in the official documentation work? It seems to be setting extensions using a git node in the DSL, without using GitSCMSource. Does the technique described in " use unsupported plugins with job DSL " still work? That technique does not seem to require GitSCMSource. A similar technique is described in " How to use Jenkins JobDSL to set 'Check out to specific local branch' in Git Plugin " , in " sparse checkout with jenkins dsl plugin " and in " use Jenkins DSL to specify a git executable in github SCM ". I'm prone to close this as "Won't fix", but will wait for verification from stephenconnolly

          Stephen Connolly added a comment -

          So my understanding is that you need to use the techniques for using unsupported plugins with job DSL to configure traits... at least until we get around to adding the @Symbol annotations (JENKINS-45504)... of course that is currently blocked on JENKINS-45503 because for reasons documented in those issues.

          Having said all that, we did retain the setter for extensions (just marked deprecated is all) and that setter will create the matching traits, so the only thing I can think is that you may be hitting the ref trimming... in which case all you need to do is add the trait that allows specifying an additional refspec to fetch and ensure that the master branch refspec is always fetched from the remote so that the merge extension / trait can work...

          markewaite I think we need to hear back on how this "no longer works" to determine if:

          • The jobs are not getting configured correctly; or
          • The configured jobs are failing because of the missing refspec for origin; or
          • Something else

          Stephen Connolly added a comment - So my understanding is that you need to use the techniques for using unsupported plugins with job DSL to configure traits... at least until we get around to adding the @Symbol annotations ( JENKINS-45504 )... of course that is currently blocked on JENKINS-45503 because for reasons documented in those issues. Having said all that, we did retain the setter for extensions (just marked deprecated is all) and that setter will create the matching traits, so the only thing I can think is that you may be hitting the ref trimming... in which case all you need to do is add the trait that allows specifying an additional refspec to fetch and ensure that the master branch refspec is always fetched from the remote so that the merge extension / trait can work... markewaite I think we need to hear back on how this "no longer works" to determine if: The jobs are not getting configured correctly; or The configured jobs are failing because of the missing refspec for origin; or Something else
          Art V made changes -
          Attachment New: Screen Shot 2017-07-21 at 09.36.26.png [ 38988 ]
          Attachment New: Screen Shot 2017-07-21 at 09.36.06.png [ 38989 ]

          Art V added a comment -

          Using this example:

          multibranchPipelineJob('AMQ-Docker-Branch') {
  displayName('AMQ-Docker-Branch')
  description('AMQ Docker Container Branch Builds')
  orphanedItemStrategy {
    discardOldItems {
      numToKeep(7)
    }
  }
  branchSources {
    git {
      id('')
      ignoreOnPushNotifications(false)
      remote('git@git.com/amq-docker.git')
      credentialsId('SSHKeyCredentials')
      includes('*')
      excludes('master')
      extensions {
        wipeWorkspace()
        preBuildMerge {
          options {
            mergeRemote('origin')
            mergeTarget('master')
            mergeStrategy('default')
            fastForwardMode('FF')
          }
        }
      }
    }
  }
}

          I get this error:

          ERROR: (script, line 17) No signature of method: javaposse.jobdsl.dsl.helpers.workflow.GitBranchSourceContext.extensions() is applicable for argument types: (script$_run_closure1$_closure3$_closure5$_closure6) values: [script$_run_closure1$_closure3$_closure5$_closure6@5218bc64]
Finished: FAILURE

          If I try to do the unsupported approach - I am using this sample, but it does not create the extensions.

          multibranchPipelineJob('CI-TEST-MB') {
  displayName('CI-TEST-MB')
  description('Test Multibranch Job')
  orphanedItemStrategy {
    discardOldItems {
      numToKeep(7)
    }
  }
branchSources {
  git {
    id('')
    ignoreOnPushNotifications(false)
    remote('git@git.com/test.git')
    credentialsId('test')
    includes('*')
    excludes('master')
    configure { node -> //the GitSCM node is passed in

        // Add the WipeWorkspace functionality
        node / 'extensions' << 'hudson.plugins.git.extensions.impl.WipeWorkspace'  {
        }
      }
    }
  }
}

           See attachements for result..

           

          Art V added a comment - Using this example: multibranchPipelineJob( 'AMQ-Docker-Branch' ) { displayName( 'AMQ-Docker-Branch' ) description( 'AMQ Docker Container Branch Builds' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { git { id('') ignoreOnPushNotifications( false ) remote( 'git@git.com/amq-docker.git' ) credentialsId( 'SSHKeyCredentials' ) includes( '*' ) excludes( 'master' ) extensions { wipeWorkspace() preBuildMerge { options { mergeRemote( 'origin' ) mergeTarget( 'master' ) mergeStrategy( ' default ' ) fastForwardMode( 'FF' ) } } } } } } I get this error: ERROR: (script, line 17) No signature of method: javaposse.jobdsl.dsl.helpers.workflow.GitBranchSourceContext.extensions() is applicable for argument types: (script$_run_closure1$_closure3$_closure5$_closure6) values: [script$_run_closure1$_closure3$_closure5$_closure6@5218bc64] Finished: FAILURE If I try to do the unsupported approach - I am using this sample, but it does not create the extensions. multibranchPipelineJob( 'CI-TEST-MB' ) { displayName( 'CI-TEST-MB' ) description( 'Test Multibranch Job' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { git { id('') ignoreOnPushNotifications( false ) remote( 'git@git.com/test.git' ) credentialsId( 'test' ) includes( '*' ) excludes( 'master' ) configure { node -> //the GitSCM node is passed in // Add the WipeWorkspace functionality node / 'extensions' << 'hudson.plugins.git.extensions.impl.WipeWorkspace' { } } } } }  See attachements for result..  

          Art V added a comment -

          With my latest testing using the configure block - I noticed the node reference is always referencing the root xml node, not the current node. I will continue to test/configure using the job dsl config block and will let you know what I find.

          Art V added a comment - With my latest testing using the configure block - I noticed the node reference is always referencing the root xml node, not the current node. I will continue to test/configure using the job dsl config block and will let you know what I find.

          Art V added a comment -

          A hackish workaround using the configure block (bypassing git block entirely)..

          multibranchPipelineJob('CI-TEST-MB') {
  displayName('CI-TEST-MB')
  description('Test Multibranch Job')
  orphanedItemStrategy {
    discardOldItems {
      numToKeep(7)
    }
  }
  branchSources {
    configure { node ->
      node / sources(class: 'jenkins.branch.MultiBranchProject$BranchSourceList') / data / 'jenkins.branch.BranchSource' / source(class: 'jenkins.plugins.git.GitSCMSource') {
        remote 'git@git.com/test.git'
        credentialsId 'test'
        includes '*'
        excludes 'master'
        ignoreOnPushNotifications 'false'
        id ''
        traits {
          'jenkins.plugins.git.traits.BranchDiscoveryTrait'()
          'jenkins.plugins.git.traits.WipeWorkspaceTrait'() {
            extension(class: 'hudson.plugins.git.extensions.impl.WipeWorkspace')
          }
          'jenkins.plugins.git.traits.PreBuildMergeTrait'() {
            extension(class: 'hudson.plugins.git.extensions.impl.PreBuildMerge') {
              options {
                mergeRemote 'origin'
                mergeTarget 'master'
                mergeStrategy 'default'
                fastForwardMode 'FF'
              }
            }
          }
        }
      }
    }
  }
}

           

          Reason I removed the git {} section entirely was because it would always add a new record of the 'jenkins.branch.BranchSource' section (not updating the existing BranchSource node)

          This might be a separate bug (or misunderstood documentation) If I put the configure block inside the git{} section, the reference 'node' would always point to the root xml element not the current node/git element. 
          Example from documentation..

          job('example') {
    scm {
        git {
            remote {
                url('git@server:account/repo1.git')
            }
            configure { node ->
                // node represents <hudson.plugins.git.GitSCM>
            }
        }
    }
}

           

          Art V added a comment - A hackish workaround using the configure block (bypassing git block entirely).. multibranchPipelineJob( 'CI-TEST-MB' ) { displayName( 'CI-TEST-MB' ) description( 'Test Multibranch Job' ) orphanedItemStrategy { discardOldItems { numToKeep(7) } } branchSources { configure { node -> node / sources(class: 'jenkins.branch.MultiBranchProject$BranchSourceList' ) / data / 'jenkins.branch.BranchSource' / source(class: 'jenkins.plugins.git.GitSCMSource' ) { remote 'git@git.com/test.git' credentialsId 'test' includes '*' excludes 'master' ignoreOnPushNotifications ' false ' id '' traits { 'jenkins.plugins.git.traits.BranchDiscoveryTrait' () 'jenkins.plugins.git.traits.WipeWorkspaceTrait' () { extension(class: 'hudson.plugins.git.extensions.impl.WipeWorkspace' ) } 'jenkins.plugins.git.traits.PreBuildMergeTrait' () { extension(class: 'hudson.plugins.git.extensions.impl.PreBuildMerge' ) { options { mergeRemote 'origin' mergeTarget 'master' mergeStrategy ' default ' fastForwardMode 'FF' } } } } } } } }   Reason I removed the git {} section entirely was because it would always add a new record of the 'jenkins.branch.BranchSource' section (not updating the existing BranchSource node) This might be a separate bug (or misunderstood documentation) If I put the configure block inside the git{} section, the reference 'node' would always point to the root xml element not the current node/git element.  Example from documentation.. job( 'example' ) { scm { git { remote { url( 'git@server:account/repo1.git' ) } configure { node -> // node represents <hudson.plugins.git.GitSCM> } } } }  
          Steffen Gebert made changes -
          Link New: This issue is related to JENKINS-46202 [ JENKINS-46202 ]
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]

          Daniel Spilker added a comment -

          I can not reproduce this with all plugins updated to the latest version. I see multibranchPipelineJob > branchSources > branchSource > source > git > extensions as well as multibranchPipelineJob > branchSources > branchSource > source > git > traits in my embedded API viewer. Can you create a working configuration manually and post the config XML? Then we can figure out what's missing in Job DSL.

          Daniel Spilker added a comment - I can not reproduce this with all plugins updated to the latest version. I see multibranchPipelineJob > branchSources > branchSource > source > git > extensions as well as multibranchPipelineJob > branchSources > branchSource > source > git > traits in my embedded API viewer. Can you create a working configuration manually and post the config XML? Then we can figure out what's missing in Job DSL.

            Unassigned Unassigned
            arty13 Art V
            Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: