Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-38508

git scm poll doesn't ignore changes with pipeline

      We are using the job as attached with the groovy test.groovy file. We poll the SCM for changes but want to ignore any change that is in our jenkins-jobs git repo. This is because in a full setup we poll more repositories which do change. In the jenkins pipeline we use .* as exclude region. This does work unless we clone the same repository in our test.groovy. There we say poll=false, but the repository is polled and at the moment we make a commit to jenkins-jobs.git the job is run. With these configuration we would expect this job to never run, even when we schedule it frequrently

        1. config.xml
          2 kB
        2. test.groovy
          0.2 kB

          [JENKINS-38508] git scm poll doesn't ignore changes with pipeline

          Jesse Glick added a comment -

          Would need to study your example more closely since so far I do not understand it. (Generally, steps to reproduce from scratch are more helpful than explanations.) Possibly same as JENKINS-38659.

          Jesse Glick added a comment - Would need to study your example more closely since so far I do not understand it. (Generally, steps to reproduce from scratch are more helpful than explanations.) Possibly same as JENKINS-38659 .

          I also don't 100% understand it. Mostly it seems the polling does work, but in some cases we still see jobs to run even when nothing changed

          Johnny Willemsen added a comment - I also don't 100% understand it. Mostly it seems the polling does work, but in some cases we still see jobs to run even when nothing changed

          Martin Sander added a comment - - edited

          JENKINS-39621 might also be related.

          Martin Sander added a comment - - edited JENKINS-39621 might also be related.

          Jukka Lehtniemi added a comment - - edited

          I can reproduce this (Jenkins 2.43, Git plugin 3.0.1, Pipeline plugin 2.4).

          Steps to reproduce:

          1. Create a test repository:
            git init /tmp/testrepo; cd /tmp/testrepo; echo foo > foo; git add foo; git commit -m foo
            
          2. Create a new pipeline job with script:
            properties([pipelineTriggers([pollSCM('* * * * *')])])
            node("master") {
                git(url: '/tmp/testrepo', poll:false)
            }
            
          3. Trigger the pipeline once manually
          4. Commit more content to the test repository
             
            echo foo2 >> foo; git commit -a -m foo2
            
          5. Observe how the build is being triggered in a minute even if the poll=false is set

          Jukka Lehtniemi added a comment - - edited I can reproduce this (Jenkins 2.43, Git plugin 3.0.1, Pipeline plugin 2.4). Steps to reproduce: Create a test repository: git init /tmp/testrepo; cd /tmp/testrepo; echo foo > foo; git add foo; git commit -m foo Create a new pipeline job with script: properties([pipelineTriggers([pollSCM( '* * * * *' )])]) node( "master" ) { git(url: '/tmp/testrepo' , poll: false ) } Trigger the pipeline once manually Commit more content to the test repository echo foo2 >> foo; git commit -a -m foo2 Observe how the build is being triggered in a minute even if the poll=false is set

          Pavel Georgiev added a comment - - edited

          I'm not sure if this is the correct defect but im my case when i checkout a git repo using the checkout step anc configure poll: false i see in the git pooling log that this repo is being checked for changes....

          node('build') {

          checkout changelog: false, poll: false,
          scm: [$class: 'GitSCM',
          branches: [[name: '*/master']],
          doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [],
          userRemoteConfigs: [[credentialsId: 'git_ci_credentials', url: 'https://hidden']]]
          }

          The pooling log:

          Using strategy: Default
          [poll] Last Built Revision: Revision hidden (refs/remotes/origin/master)
          using GIT_ASKPASS to set credentials git_ci_credentials
          > git ls-remote -h https://hidden # timeout=10
          Found 6 remote heads on https://hidden
          [poll] Latest remote head revision on refs/heads/master is: hidden - already built by 171
          Done. Took 5.8 sec
          No changes

          Pavel Georgiev added a comment - - edited I'm not sure if this is the correct defect but im my case when i checkout a git repo using the checkout step anc configure poll: false i see in the git pooling log that this repo is being checked for changes.... node('build') { checkout changelog: false, poll: false, scm: [$class: 'GitSCM', branches: [ [name: '*/master'] ], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [ [credentialsId: 'git_ci_credentials', url: 'https:// hidden '] ]] } The pooling log: Using strategy: Default [poll] Last Built Revision: Revision hidden (refs/remotes/origin/master) using GIT_ASKPASS to set credentials git_ci_credentials > git ls-remote -h https:// hidden # timeout=10 Found 6 remote heads on https:// hidden [poll] Latest remote head revision on refs/heads/master is: hidden - already built by 171 Done. Took 5.8 sec No changes

          ansib added a comment - - edited

          I have here a small Explanation on StackOverflow with the same/similar case. Maybe it could help you to understand what the problem with the polling is.

          http://stackoverflow.com/questions/40017034/jenkins-pipeline-poll-scm-with-different-repo-for-jenkinsfile-and-application-re

          I try to summarize the problem we have at our company.

          We have two repositories. One for the configuration of Jenkinsfiles (config-repo) and on for our Development (dev-repo), which is included in the Jenkinsfile.
          In Jenkins we have 3 Jobs for deployment of the dev-repo. One for dev branch, one for hotfix branch and one for release branch. --> Jenkinsfiles are located in the config-repo. the git step in the Jenkinsfile check out the dev-repo.

          If i check new code into the config-repo, the builds aren't triggered, but if we check new code into dev-repo after a checkin/change in the config-repo all jobs configured with the config-repo and using the dev-repo starts a build.

          The Jobs should never start a build after checkins into config-repo and only starts a build, if new code is checked into their specific branch at the dev-repo.

          ansib added a comment - - edited I have here a small Explanation on StackOverflow with the same/similar case. Maybe it could help you to understand what the problem with the polling is. http://stackoverflow.com/questions/40017034/jenkins-pipeline-poll-scm-with-different-repo-for-jenkinsfile-and-application-re I try to summarize the problem we have at our company. We have two repositories. One for the configuration of Jenkinsfiles (config-repo) and on for our Development (dev-repo), which is included in the Jenkinsfile. In Jenkins we have 3 Jobs for deployment of the dev-repo. One for dev branch, one for hotfix branch and one for release branch. --> Jenkinsfiles are located in the config-repo. the git step in the Jenkinsfile check out the dev-repo. If i check new code into the config-repo, the builds aren't triggered, but if we check new code into dev-repo after a checkin/change in the config-repo all jobs configured with the config-repo and using the dev-repo starts a build. The Jobs should never start a build after checkins into config-repo and only starts a build, if new code is checked into their specific branch at the dev-repo.

          Edmund Haselwanter added a comment - - edited

          Update:

                    // poll false does not count
                    [$class: 'DisableRemotePoll'],
                    [$class: 'PathRestriction', excludedRegions: '', includedRegions: 'foo/.*']
          

          works as a workaround

          ------ old report -----

          same here.

              checkout(
                poll: true,
                scm: [
                  $class: 'GitSCM',
                  userRemoteConfigs: [[
                    url: 'git@xxxxxx/deployed-service-versions.git',
                    credentialsId: xxxxxx,
                  ]],
                  branches: [[name: '*/master']],
                  extensions: [
                    [$class: 'RelativeTargetDirectory', relativeTargetDir: 'deployed-service-versions'],
                    [$class: 'CleanCheckout'],
                  ],
                  doGenerateSubmoduleConfigurations: false,
                  submoduleCfg: [],
                ],
              )
          
              checkout(
                poll: false,
                scm: [
                  $class: 'GitSCM',
                  userRemoteConfigs: [[
                    url: 'git@8xxxxxxxxr/images-reporting.git',
                    credentialsId: 'xxxxx',
                  ]],
                  branches: [[name: '*/master']],
                  extensions: [
                    [$class: 'RelativeTargetDirectory', relativeTargetDir: 'images-reporting'],
                    // prune everything in workspace
                    [$class: 'CleanCheckout'],
                    // avoid detached head. it calculates a local branch derived from the clone (mater in this case)
                    [$class: 'LocalBranch'],
                  ],
                  doGenerateSubmoduleConfigurations: false,
                  submoduleCfg: [],
                ],
              )
          
          

          does poll and trigger on the `images-reporting.git` repo, where `poll: false` is set

          Started on Feb 17, 2017 11:27:00 AM
          Using strategy: Default
          [poll] Last Built Revision: Revision da056b8e354ad819942980d620f35751b52fad6a (refs/remotes/origin/master)
           > git ls-remote -h git@xxxxxxxx/deployed-service-versions.git # timeout=10
          Found 1 remote heads on git@xxxxxxxxxxxxx/deployed-service-versions.git
          [poll] Latest remote head revision on refs/heads/master is: da056b8e354ad819942980d620f35751b52fad6a - already built by 202
          Using strategy: Default
          [poll] Last Built Revision: Revision 724e92bee2268fe431d86239d97ce04ed673a047 (refs/remotes/origin/master)
          using GIT_SSH to set credentials gitlab
           > git ls-remote -h git@xxxxxxxx/images-reporting.git # timeout=10
          Found 1 remote heads on git@xxxxxx/-images-reporting.git
          [poll] Latest remote head revision on refs/heads/master is: e9cfddb95d27a46e3048d12e0a83fedd9a3dc2ae
          Done. Took 1.6 sec
          Changes found
          

          so, how can I ignore changes in `/images-reporting.git` but build on changes in the the `deployed-service-versions.git' repo?

          Edmund Haselwanter added a comment - - edited Update: // poll false does not count [$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: ' foo/.*'] works as a workaround ------ old report ----- same here. checkout( poll: true , scm: [ $class: 'GitSCM' , userRemoteConfigs: [[ url: 'git@xxxxxx/deployed-service-versions.git' , credentialsId: xxxxxx, ]], branches: [[name: '*/master' ]], extensions: [ [$class: 'RelativeTargetDirectory' , relativeTargetDir: 'deployed-service-versions' ], [$class: 'CleanCheckout' ], ], doGenerateSubmoduleConfigurations: false , submoduleCfg: [], ], ) checkout( poll: false , scm: [ $class: 'GitSCM' , userRemoteConfigs: [[ url: 'git@8xxxxxxxxr/images-reporting.git' , credentialsId: 'xxxxx' , ]], branches: [[name: '*/master' ]], extensions: [ [$class: 'RelativeTargetDirectory' , relativeTargetDir: 'images-reporting' ], // prune everything in workspace [$class: 'CleanCheckout' ], // avoid detached head. it calculates a local branch derived from the clone (mater in this case ) [$class: 'LocalBranch' ], ], doGenerateSubmoduleConfigurations: false , submoduleCfg: [], ], ) does poll and trigger on the `images-reporting.git` repo, where `poll: false` is set Started on Feb 17, 2017 11:27:00 AM Using strategy: Default [poll] Last Built Revision: Revision da056b8e354ad819942980d620f35751b52fad6a (refs/remotes/origin/master) > git ls-remote -h git@xxxxxxxx/deployed-service-versions.git # timeout=10 Found 1 remote heads on git@xxxxxxxxxxxxx/deployed-service-versions.git [poll] Latest remote head revision on refs/heads/master is: da056b8e354ad819942980d620f35751b52fad6a - already built by 202 Using strategy: Default [poll] Last Built Revision: Revision 724e92bee2268fe431d86239d97ce04ed673a047 (refs/remotes/origin/master) using GIT_SSH to set credentials gitlab > git ls-remote -h git@xxxxxxxx/images-reporting.git # timeout=10 Found 1 remote heads on git@xxxxxx/-images-reporting.git [poll] Latest remote head revision on refs/heads/master is: e9cfddb95d27a46e3048d12e0a83fedd9a3dc2ae Done. Took 1.6 sec Changes found so, how can I ignore changes in `/images-reporting.git` but build on changes in the the `deployed-service-versions.git' repo?

          D B added a comment - - edited

          ehaselwanter, Could you explain a bit how to use your workaround?  I'm not sure where to put DisableRemotePoll or PathRestriction.  I tried putting it under the extensions property of checkout scm but it's still polling the secondary repo.

          Update: You add them between the extensions: [] brackets.

          D B added a comment - - edited ehaselwanter , Could you explain a bit how to use your workaround?  I'm not sure where to put DisableRemotePoll or PathRestriction.  I tried putting it under the extensions property of checkout scm but it's still polling the secondary repo. Update: You add them between the extensions: [] brackets.

          Ygor Almeida added a comment - - edited

          Thanks to ehaselwanter this works for me:

          checkout(
          poll: false,
          scm: [
            $class: 'GitSCM', branches: [[name: '*/master']],
            userRemoteConfigs: [[
              url: MY_URL.git,
              credentialsId: CREDENTIALS_ID]],
            extensions: [
              [$class: 'DisableRemotePoll'],
              [$class: 'PathRestriction', excludedRegions: '', includedRegions: '*']]
          ])
          

           
          GitSCM poll does ignore changes now.
           
           

          Ygor Almeida added a comment - - edited Thanks to ehaselwanter this works for me: checkout( poll: false , scm: [ $class: 'GitSCM' , branches: [[name: '*/master' ]], userRemoteConfigs: [[ url: MY_URL.git, credentialsId: CREDENTIALS_ID]], extensions: [ [$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: ' *']] ])   GitSCM poll does ignore changes now.    

          Alfredo Moreira added a comment - - edited

          Are you guys using: 
          triggers {
          pollSCM 'H/5 * * * *'
          }
          to trigger the builds? ehaselwanter ygorth
           
          I have that and it is still polling the repo I do not wan to be polled. This is my set up:
           

          // pipeline{
          //Pipeline Options
          options{
          buildDiscarder(logRotator(numToKeepStr:'5'))
          disableConcurrentBuilds()
          skipDefaultCheckout()
          }
          //define agent/node to run job on on
          agent {
          node{
          label 'windows'
          }
          }
          triggers {
          pollSCM 'H/2 * * * *'
          }
          //Stages/Build Steps
          stages{
          //Set Source Code Management of code
          stage('Set SCM'){
          steps{
          checkout(
          poll:true,
          scm: [
          $class:'GitSCM',
          branches: [[name:branch]],
          doGenerateSubmoduleConfigurations: false,
          clean:true,
          extensions: [[$class: 'DisableRemotePoll'],
          [$class: 'PathRestriction', excludedRegions: '',
          includedRegions: '*']],
          submoduleCfg: [],
          userRemoteConfigs: [[url: 'link to Repo']],
          credentialsId:'Jenkins Creds',
          ])
          }
          }
          //Pull Groovy Function Library
          stage('Load Jenkins-Code Files'){
          steps{
          dir('CI-CD'){
          checkout(
          poll:false,
          scm: [
          $class:'GitSCM',
          branches: [[name:'master']],
          clean:true,
          extensions: [
          [$class: 'DisableRemotePoll'],
          [$class: 'PathRestriction', excludedRegions: '',
          includedRegions: '*']],
          submoduleCfg: [],
          userRemoteConfigs: [[url: 'GitHub Repo']],
          credentialsId:'Jenkins Creds',
          ])
          }
          }
          }

           

          The Load Jenkins Code is the repo i do not want to poll. Like this, both repos are being polled, and if i remove trigger nothing gets polled. Any help would be much appreciated. Thanks in advance.

          Alfredo Moreira added a comment - - edited Are you guys using:  triggers { pollSCM 'H/5 * * * *' } to trigger the builds?  ehaselwanter ygorth   I have that and it is still polling the repo I do not wan to be polled. This is my set up:   // pipeline{ //Pipeline Options options{ buildDiscarder(logRotator(numToKeepStr: '5' )) disableConcurrentBuilds() skipDefaultCheckout() } //define agent/node to run job on on agent { node{ label 'windows' } } triggers { pollSCM 'H/2 * * * *' } //Stages/Build Steps stages{ //Set Source Code Management of code stage( 'Set SCM' ){ steps{ checkout( poll: true , scm: [ $class: 'GitSCM' , branches: [[name:branch]], doGenerateSubmoduleConfigurations: false , clean: true , extensions: [[$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: '*' ]], submoduleCfg: [], userRemoteConfigs: [[url: 'link to Repo' ]], credentialsId: 'Jenkins Creds' , ]) } } //Pull Groovy Function Library stage( 'Load Jenkins-Code Files' ){ steps{ dir( 'CI-CD' ){ checkout( poll: false , scm: [ $class: 'GitSCM' , branches: [[name: 'master' ]], clean: true , extensions: [ [$class: 'DisableRemotePoll' ], [$class: 'PathRestriction' , excludedRegions: '', includedRegions: '*' ]], submoduleCfg: [], userRemoteConfigs: [[url: 'GitHub Repo' ]], credentialsId: 'Jenkins Creds' , ]) } } }   The Load Jenkins Code is the repo i do not want to poll. Like this, both repos are being polled, and if i remove trigger nothing gets polled. Any help would be much appreciated. Thanks in advance.

          Michael Wong added a comment -

          It seems like polling is always done if changelog is set to true (by default it does get set to true). It should work if you set both to false.

          You also might need to restart your Jenkins server to get it to work (if the job was polling before, since the cached polling baseline never gets reset).

          Michael Wong added a comment - It seems like polling is always done if changelog is set to true (by default it does get set to true). It should work if you set both to false. You also might need to restart your Jenkins server to get it to work (if the job was polling before, since the cached polling baseline never gets reset).

          sradi added a comment -

          For me, the workaround with "DisableRemotePoll" and "PathRestriction" does not work.
          Did anyone successfully use this workaround?

          Is there anybody, who plans to fix this old issue anytime soon?

          sradi added a comment - For me, the workaround with "DisableRemotePoll" and "PathRestriction" does not work. Did anyone successfully use this workaround? Is there anybody, who plans to fix this old issue anytime soon?

          sradi use the option "changelog: false", i had used the Disable with no success, but adding the changelog option fixed everything. You might have to restart Jenkins for the polling to take effect correctly and not use the cache. 

          Here is my git checkout that i dont want to be polled, this repo is cloned into the CI-CD directory inside the project:

          checkout(
          poll:false,
          changelog:false,
          scm: [
          $class:'GitSCM',
          branches: [[name:'master']],
          clean:true,
          extensions: [[$class: 'DisableRemotePoll'],
          [$class: 'PathRestriction', excludedRegions: '*']],
          userRemoteConfigs: [[url: 'GithubURL]],
          credentialsId:'creds-ID,
          ])

          And the polling will look like this:

          Started on Jan 22, 2018 9:43:15 AM Using strategy: Default [poll] Last Built Revision: Revision a2a141227f71769af6e69a0b5d1dacfb6478effe (origin/develop) > git --version # timeout=10 > git ls-remote -h

          ""Github URL"

          1. timeout=10 Found 36 remote heads on

          "Github URL"

          [poll] Latest remote head revision on refs/heads/develop is: a2a141227f71769af6e69a0b5d1dacfb6478effe - already built by 272

          no polling baseline in /Users/mcoeadm/.jenkins/jobs/pe-dev- Davita Dev - Android/workspace/CI-CD on Done. Took 0.69 sec No changes .  --> "THIS IS THE LINE THAT MAKES THE DIFFERENCE"

           

          Hopefully this helps.

          Alfredo Moreira added a comment - sradi use the option "changelog: false", i had used the Disable with no success, but adding the changelog option fixed everything. You might have to restart Jenkins for the polling to take effect correctly and not use the cache.  Here is my git checkout that i dont want to be polled, this repo is cloned into the CI-CD directory inside the project: checkout( poll:false, changelog:false, scm: [ $class:'GitSCM', branches: [ [name:'master'] ], clean:true, extensions: [ [$class: 'DisableRemotePoll'] , [$class: 'PathRestriction', excludedRegions: '*'] ], userRemoteConfigs: [ [url: 'GithubURL] ], credentialsId:'creds-ID, ]) And the polling will look like this: Started on Jan 22, 2018 9:43:15 AM Using strategy: Default [poll] Last Built Revision: Revision a2a141227f71769af6e69a0b5d1dacfb6478effe (origin/develop) > git --version # timeout=10 > git ls-remote -h ""Github URL" timeout=10 Found 36 remote heads on "Github URL" [poll] Latest remote head revision on refs/heads/develop is: a2a141227f71769af6e69a0b5d1dacfb6478effe - already built by 272 no polling baseline in /Users/mcoeadm/.jenkins/jobs/pe-dev- Davita Dev - Android/workspace/CI-CD on Done. Took 0.69 sec No changes .  --> "THIS IS THE LINE THAT MAKES THE DIFFERENCE"   Hopefully this helps.

          Aaron Ling added a comment -

          I hit this issue as well. Setting changelog to false and restarting Jenkins seems to bypass the issue.

          Aaron Ling added a comment - I hit this issue as well. Setting changelog to false and restarting Jenkins seems to bypass the issue.

          Nandlal Deshmukh added a comment - - edited

          @ehaselwanter ,  Thanks a lot, you saved me ..

           Below workaround worked for me 

          extensions: [[$class: 'DisableRemotePoll']]
          
           def scmVars1 = checkout([def scmVars1 = checkout([ $class : 'GitSCM', branches:  [[name: '*/' + "${BRANCH_PARAM}"]], doGenerateSubmoduleConfigurations: false, extensions: [[$class: 'DisableRemotePoll']], submoduleCfg : [], userRemoteConfigs: [[url: 'https://github.com/zbcdeshmukhxys/framework.git']] ])
          

          Nandlal Deshmukh added a comment - - edited @ ehaselwanter  ,  Thanks a lot, you saved me ..  Below workaround worked for me  extensions: [[$class: 'DisableRemotePoll' ]] def scmVars1 = checkout([def scmVars1 = checkout([ $class : 'GitSCM' , branches:  [[name: '*/' + "${BRANCH_PARAM}" ]], doGenerateSubmoduleConfigurations: false , extensions: [[$class: 'DisableRemotePoll' ]], submoduleCfg : [], userRemoteConfigs: [[url: 'https: //github.com/zbcdeshmukhxys/framework.git' ]] ])

          Igor Fraa added a comment -

          According to Git plugin source 
          [$class: 'DisableRemotePoll']
          is not related to disabling polling. It is for repositories with 'ls-remote' disabled and acts as an alias for requiresWorkspaceForPolling parameter. 

          the only workaround to disable polling is to set 

          [$class: 'PathRestriction', excludedRegions: '', includedRegions: 'no/such/path/exists']

          Igor Fraa added a comment - According to Git plugin source   [$class: 'DisableRemotePoll'] is not related to disabling polling. It is for repositories with 'ls-remote' disabled and acts as an alias for requiresWorkspaceForPolling parameter.  the only workaround to disable polling is to set  [$class: 'PathRestriction' , excludedRegions: '', includedRegions: ' no/such/path/exists']

          igora setting path restrictions, as you recommended, did not work for me.

          Stephan Schultchen added a comment - igora setting path restrictions, as you recommended, did not work for me.

            Unassigned Unassigned
            jwillemsen Johnny Willemsen
            Votes:
            17 Vote for this issue
            Watchers:
            27 Start watching this issue

              Created:
              Updated: