Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52816

scm poll triggers even there is no change in repository(bitbucket)

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • git-plugin
    • None
    • we are using Jenkins LTS 2.176.3 on hosted on amazonlinux , job-dsl - 1.74 , git plugin - 4.0.0-rc

      step 1: create a seed job 
      ```
      String basepath = 'Common/Java/testapp'

      String giturl = 'git@bitbucket.org:abc/testapp'

      pipelineJob("$basepath/1.Build-Dev-Int") {
      logRotator {
      numToKeep(10)
      artifactNumToKeep(10)
      }
      parameters {
      stringParam('USE_GIT_BRANCH', 'master')
      }
      throttleConcurrentBuilds {
      maxTotal(1)
      }

      triggers {
      scm('H/15 * * * *')
      }

      definition {
      cpsScm {
      scm {
      git

      { remote \{ url(giturl) }

      branches('master')
      extensions { }
      }
      scriptPath('pipeline/pipeline_dev_int.groovy')
      }
      }
      }
      }```
      step 2 : create pipeline_dev_int.groovy

      ```@Library('sharedlibrary@master')

      node()

      remaining groovy script```
       

      step 3:     configured shared library in manage jenkins
      even though there is no change in repo it is triggering jobs 
       

       

      whenever a branch/ commit made to the shared library even though there is no change in testapp repo it triggering the job for every 15 min and found this in polling history

      Using strategy: Default
      [poll] Last Built Revision: Revision 6717866352d0d5a5e9b41da88a799993ab1d4191 (origin/master)
      No credentials specified
      > git --version # timeout=10
      > git ls-remote -h git@bitbucket.org:abc/testapp # timeout=10
      Found 3 remote heads on git@bitbucket.org:abc/testapp
      [poll] Latest remote head revision on refs/heads/master is: 6717866352d0d5a5e9b41da88a799993ab1d4191 - already built by 97
      Using strategy: Specific revision
      [poll] Last Built Revision: Revision a7cf4fd0c7484765c4a5166a36613c387aee8501 (surya/MCDCA-1412-ECR-NEW-ACCOUNT)
      No credentials specified
      > git --version # timeout=10
      > git ls-remote -h git@bitbucket.org:abc/sharedlibrary # timeout=10
      Found 3 remote heads on git@bitbucket.org:abc/sharedlibrary
      [poll] Latest remote head revision on refs/heads/surya/MCDCA-1412-ECR-NEW-ACCOUNT is: ab8eb071afb815bca04fd4e50a991ae54ea37e8c
      Using strategy: Default
      [poll] Last Built Revision: Revision 6717866352d0d5a5e9b41da88a799993ab1d4191 (origin/master)
      using credential 6e83116e-4fb6-491a-a2c9-9ac9d51d4765
      > git --version # timeout=10
      using GIT_SSH to set credentials
      > git ls-remote -h git@bitbucket.org:abc/testapp # timeout=10
      Found 3 remote heads on git@bitbucket.org:abc/testapp
      [poll] Latest remote head revision on refs/heads/master is: 6717866352d0d5a5e9b41da88a799993ab1d4191 - already built by 97
      Done. Took 8.6 sec
      Changes found

      Polling Log

      This page captures the polling log that triggered this build.
      Started on Jul 30, 2018 7:15:00 PM Using strategy: Default [poll] Last Built Revision: Revision 80ae1a7997f138af5faa0e594eca7e88eb174026 (origin/master) > git --version # timeout=10 > git ls-remote -h git@bitbucket.org:xxx/xxxxxxx # timeout=10 Found 1 remote heads on git@bitbucket.org:xx/xxxxxx[poll] Latest remote head revision on refs/heads/master is: 80ae1a7997f138af5faa0e594eca7e88eb174026 - already built by 185 no polling baseline in /var/lib/jenkins/workspace/Sandbox/xxxx/xxxx/1.Build-Dev-Int@libs/xxx on Using strategy: Default > git --version # timeout=10 using GIT_SSH to set credentials Bitbucket SSH Key > git ls-remote -h git@bitbucket.org:xxxxx.git # timeout=10 Found 1 remote heads on git@bitbucket.org:xx/xxxx.git [poll] Latest remote head revision on refs/heads/master is: 80ae1a7997f138af5faa0e594eca7e88eb174026 Done. Took 2.7 sec Changes found

          [JENKINS-52816] scm poll triggers even there is no change in repository(bitbucket)

          suryatej yaramada created issue -
          Mark Waite made changes -
          Assignee Original: Mark Waite [ markewaite ]

          Mark Waite added a comment - - edited

          Please provide a series of numbered steps that will allow another person to duplicate the problem you're seeing. There are too many details missing from the description to help with the issue you're seeing.

          For example, what is the final definition of the job as created by Job DSL?

          Is there a webhook configured from Bitbucket to Jenkins? If not, why not (since polling is much less efficient than web hooks)?

          If you use a web hook instead of polling, does that resolve the problem?

          Is the problem job a Pipeline (not multibranch) or a multibranch Pipeline?

          What is the branch definition in the jobs?

          What are the branches defined in the repository?

          Mark Waite added a comment - - edited Please provide a series of numbered steps that will allow another person to duplicate the problem you're seeing. There are too many details missing from the description to help with the issue you're seeing. For example, what is the final definition of the job as created by Job DSL? Is there a webhook configured from Bitbucket to Jenkins? If not, why not (since polling is much less efficient than web hooks)? If you use a web hook instead of polling, does that resolve the problem? Is the problem job a Pipeline (not multibranch) or a multibranch Pipeline? What is the branch definition in the jobs? What are the branches defined in the repository?
          suryatej yaramada made changes -
          Description Original: we are using dsl jobs to  create jobs for scripted pipelines and had this function 
          triggers {
          scm('H/10 * * * *')
           
          even though there is no change in repo it is triggering jobs , we are using ecs executors
           
          h1. Polling Log

          This page captures the polling log that triggered this build.
          Started on Jul 30, 2018 7:15:00 PM Using strategy: Default [poll] Last Built Revision: Revision 80ae1a7997f138af5faa0e594eca7e88eb174026 (origin/master) > git --version # timeout=10 > git ls-remote -h git@bitbucket.org:xxx/xxxxxxx # timeout=10 Found 1 remote heads on git@bitbucket.org:xx/xxxxxx[poll] Latest remote head revision on refs/heads/master is: 80ae1a7997f138af5faa0e594eca7e88eb174026 - already built by 185 no polling baseline in /var/lib/jenkins/workspace/Sandbox/xxxx/xxxx/1.Build-Dev-Int@libs/jpl on Using strategy: Default > git --version # timeout=10 using GIT_SSH to set credentials Bitbucket SSH Key > git ls-remote -h git@bitbucket.org:xxxxx.git # timeout=10 Found 1 remote heads on git@bitbucket.org:xx/xxxx.git [poll] Latest remote head revision on refs/heads/master is: 80ae1a7997f138af5faa0e594eca7e88eb174026 Done. Took 2.7 sec Changes found
          New: we are using dsl jobs to  create jobs for scripted pipelines and had this function 
           triggers {
           scm('H/10 * * * *')
            
           even though there is no change in repo it is triggering jobs , we are using ecs executors
            
          h1. Polling Log

          This page captures the polling log that triggered this build.
           Started on Jul 30, 2018 7:15:00 PM Using strategy: Default [poll] Last Built Revision: Revision 80ae1a7997f138af5faa0e594eca7e88eb174026 (origin/master) > git --version # timeout=10 > git ls-remote -h git@bitbucket.org:xxx/xxxxxxx # timeout=10 Found 1 remote heads on git@bitbucket.org:xx/xxxxxx[poll] Latest remote head revision on refs/heads/master is: 80ae1a7997f138af5faa0e594eca7e88eb174026 - already built by 185 no polling baseline in /var/lib/jenkins/workspace/Sandbox/xxxx/xxxx/1.Build-Dev-Int@libs/xxx on Using strategy: Default > git --version # timeout=10 using GIT_SSH to set credentials Bitbucket SSH Key > git ls-remote -h git@bitbucket.org:xxxxx.git # timeout=10 Found 1 remote heads on git@bitbucket.org:xx/xxxx.git [poll] Latest remote head revision on refs/heads/master is: 80ae1a7997f138af5faa0e594eca7e88eb174026 Done. Took 2.7 sec Changes found

          Cenk Tosun added a comment - - edited

          For example, what is the final definition of the job as created by Job DSL?

          •  declerative pipeline jobs using one checkout in job configuration for the jenkinsfile with branch */master and also multiple other git repos in the jenkinsfile with */master

          Is there a webhook configured from Bitbucket to Jenkins? If not, why not (since polling is much less efficient than web hooks)?

          If you use a web hook instead of polling, does that resolve the problem?

          • here it's the other way around, but nevertheless both end up after certain build with no baseline for *** and starting multiple jobs

          Is the problem job a Pipeline (not multibranch) or a multibranch Pipeline?

          • yes, as already mentioned before

          What is the branch definition in the jobs?

          • */branchname, some other jobs contain slashes based on the gitflow pattern

          I hope that is enough.

          Cenk Tosun added a comment - - edited For example, what is the final definition of the job as created by Job DSL?  declerative pipeline jobs using one checkout in job configuration for the jenkinsfile with branch */master and also multiple other git repos in the jenkinsfile with */master Is there a webhook configured from Bitbucket to Jenkins? If not, why not (since polling is much less efficient than web hooks)? Yes, with http://Jenkinsurl/git/notifyCommit?url=repourl  - jobs are receiving the notification from bitbucket If you use a web hook instead of polling, does that resolve the problem? here it's the other way around, but nevertheless both end up after certain build with no baseline for *** and starting multiple jobs Is the problem job a Pipeline (not multibranch) or a multibranch Pipeline? yes, as already mentioned before What is the branch definition in the jobs? */branchname, some other jobs contain slashes based on the gitflow pattern I hope that is enough.

          Cenk Tosun added a comment -

          i have also identical log outputs (even identical sha1) for the builds which are triggered

          Started on Nov 22, 2018 8:37:06 AM

          no polling baseline in <path>@libs\shared-library-repo on
          Using strategy: Default
          [poll] Last Built Revision: Revision <sha1> (refs/remotes/origin/master)
          Using strategy: Default
          [poll] Last Built Revision: Revision <sha1> (refs/remotes/origin/master)
          Using strategy: Default
          [poll] Last Built Revision: Revision <sha1> (refs/remotes/origin/master)
          Done. Took 5 ms
          Changes found

           

           

          Cenk Tosun added a comment - i have also identical log outputs (even identical sha1) for the builds which are triggered Started on Nov 22, 2018 8:37:06 AM no polling baseline in <path>@libs\shared-library-repo on Using strategy: Default [poll] Last Built Revision: Revision <sha1> (refs/remotes/origin/master) Using strategy: Default [poll] Last Built Revision: Revision <sha1> (refs/remotes/origin/master) Using strategy: Default [poll] Last Built Revision: Revision <sha1> (refs/remotes/origin/master) Done. Took 5 ms Changes found    

          Mark Waite added a comment - - edited

          I think you're assuming that I'll do a lot of work to guess the conditions for this problem. When I make guesses, the guesses may be wrong. If I guess incorrectly, I spend time investigating something that is not the problem you are seeing. I'm a volunteer that works on the git plugin during my nights and weekends.

          For example, I would not have guessed that you were using '*/master' to refer to a branch when 'master' is the shorter form and 'origin/master' is the more precise form. Using '*/master' with some configurations will increase the risk that the plugin will incorrectly match a branch name from the wrong remote repository.

          As another example, I would not have guessed that you were using multiple other git repos. I assume those multiple other git repos are being cloned into separate subdirectories, but since you don't say that and don't provide steps to duplicate the problem, I cannot be sure.

          As another example, I asked if the job is a Pipeline or a multibranch Pipeline. You answered "yes, as already mentioned before". That didn't answer my question. A multibranch Pipeline has some important differences compared to a Pipeline. I need to know which of those two types it is.

          When you provide a series of numbered steps that will allow another person to duplicate the problem you're seeing, I'll consider further investigation.

          Mark Waite added a comment - - edited I think you're assuming that I'll do a lot of work to guess the conditions for this problem. When I make guesses, the guesses may be wrong. If I guess incorrectly, I spend time investigating something that is not the problem you are seeing. I'm a volunteer that works on the git plugin during my nights and weekends. For example, I would not have guessed that you were using '*/master' to refer to a branch when 'master' is the shorter form and 'origin/master' is the more precise form. Using '*/master' with some configurations will increase the risk that the plugin will incorrectly match a branch name from the wrong remote repository. As another example, I would not have guessed that you were using multiple other git repos. I assume those multiple other git repos are being cloned into separate subdirectories, but since you don't say that and don't provide steps to duplicate the problem, I cannot be sure. As another example, I asked if the job is a Pipeline or a multibranch Pipeline. You answered "yes, as already mentioned before". That didn't answer my question. A multibranch Pipeline has some important differences compared to a Pipeline. I need to know which of those two types it is. When you provide a series of numbered steps that will allow another person to duplicate the problem you're seeing, I'll consider further investigation.

          Cenk Tosun added a comment - - edited

          so it's not enough...

           

          I'll try to go into more detail:
          we are using pipeline jobs and in the job configuration we checkout the jenksfile
          Branch: */master or */releases/
          Additial Behaviours: Don't trigger a build on commit notifications

          In the jenkinsfile we checkout several git repos
          Branch: */master or */releases/
          Additial Behaviours:

          • useSubfolder
          • localBrach
          • includePath
          • excludePath
          • UserExclusion
               checkout([
                  $class: 'GitSCM',
                  branches: [[name: 'origin/master']],
                  credentialsId: <credtials>,
                  doGenerateSubmoduleConfigurations: false,
                  extensions: 
                          [$class: 'LocalBranch', localBranch: **],
                          [$class: 'ScmName', name: repoName],
                          [$class: 'CloneOption', depth: cloneDepth, honorRefspec: true, noTags: false, reference: '', shallow: false],
                          [$class: 'UserExclusion', excludedUsers: <excluded User>]
                          [$class: 'PathRestriction', excludedRegions: <excludePath>, includedRegions: <includePath>]
                          [$class: 'RelativeTargetDirectory', relativeTargetDir: item]
                          only for one repo:
                            [$class: 'IgnoreNotifyCommit']
                            [$class: 'WipeWorkspace']
                  submoduleCfg: [],
                  userRemoteConfigs: [[url: <repoUrl>]]
                ])
          
          • we are not using the multibranch pipeline plugin
          • most jobs run on slave farm using one common label

          i saw also your answer in https://issues.jenkins-ci.org/browse/JENKINS-50683 and the entry in the docu
          "In the event that Fast Remote Polling is detected as being not possible (branches to build contains wildcards, etc), polling will fallback to requiring a workspace."

          it looks like my problem is exactly related to this behavior by using:

          • wildcard branch names (changing to origin/master and origin/release/branchname is no problem)
          • an exclude message or path definition
          • an include region definition
          • an excluded user definition

          and since we delete the workspace every night, we get this behavior.

          so due this facts it seems that atm frp is not possible for me. any idea how to handle this?

          Cenk Tosun added a comment - - edited so it's not enough...   I'll try to go into more detail: we are using pipeline jobs and in the job configuration we checkout the jenksfile Branch: * /master or */releases/ Additial Behaviours: Don't trigger a build on commit notifications In the jenkinsfile we checkout several git repos Branch: * /master or */releases/ Additial Behaviours: useSubfolder localBrach includePath excludePath UserExclusion checkout([ $class: 'GitSCM' , branches: [[name: 'origin/master' ]], credentialsId: <credtials>, doGenerateSubmoduleConfigurations: false , extensions: [$class: 'LocalBranch' , localBranch: **], [$class: 'ScmName' , name: repoName], [$class: 'CloneOption' , depth: cloneDepth, honorRefspec: true , noTags: false , reference: '', shallow: false ], [$class: 'UserExclusion' , excludedUsers: <excluded User>] [$class: 'PathRestriction' , excludedRegions: <excludePath>, includedRegions: <includePath>] [$class: 'RelativeTargetDirectory' , relativeTargetDir: item] only for one repo: [$class: 'IgnoreNotifyCommit' ] [$class: 'WipeWorkspace' ] submoduleCfg: [], userRemoteConfigs: [[url: <repoUrl>]] ]) we are not using the multibranch pipeline plugin most jobs run on slave farm using one common label i saw also your answer in https://issues.jenkins-ci.org/browse/JENKINS-50683 and the entry in the docu "In the event that Fast Remote Polling is detected as being not possible (branches to build contains wildcards, etc), polling will fallback to requiring a workspace." it looks like my problem is exactly related to this behavior by using: wildcard branch names (changing to origin/master and origin/release/branchname is no problem) an exclude message or path definition an include region definition an excluded user definition and since we delete the workspace every night, we get this behavior. so due this facts it seems that atm frp is not possible for me. any idea how to handle this?
          Cenk Tosun made changes -
          Comment [ one more strange behavior, if the workspaces are missing. No change but two builds with the identical git log
          {code:java}
          Started on Nov 27, 2018 4:39:39 AM
          Using strategy: Default
          [poll] Last Built Revision: Revision fec1077f7d8b5a9b808abd0da6a84966ebe900df (refs/remotes/origin/master)
           > git.exe --version # timeout=10
          using GIT_SSH to set credentials
           > git.exe ls-remote -h <gitUrl> # timeout=10
          Found 21 remote heads on <gitUrl>
          [poll] Latest remote head revision on refs/heads/master is: fec1077f7d8b5a9b808abd0da6a84966ebe900df - already built by 186
          no polling baseline in <buildworkspace>@libs\sharedLibrary on
          Using strategy: Default
          [poll] Last Built Revision: Revision b00d5f0ce8cbf798ff613ac61ec5ad28824afd6b (refs/remotes/origin/master)
          Using strategy: Default
          [poll] Last Built Revision: Revision fec1077f7d8b5a9b808abd0da6a84966ebe900df (refs/remotes/origin/master)
          Using strategy: Default
          [poll] Last Built Revision: Revision 4895bd129cdbd7678807c88f372fdc0b37295cee (refs/remotes/origin/master)
          Done. Took 0.27 sec
          Changes found


          Started on Nov 27, 2018 4:24:39 AM
          Using strategy: Default
          [poll] Last Built Revision: Revision fec1077f7d8b5a9b808abd0da6a84966ebe900df (refs/remotes/origin/master)
           > git.exe --version # timeout=10
          using GIT_SSH to set credentials
           > git.exe ls-remote -h <gitUrl> # timeout=10
          Found 21 remote heads on <gitUrl>
          [poll] Latest remote head revision on refs/heads/master is: fec1077f7d8b5a9b808abd0da6a84966ebe900df - already built by 185
          no polling baseline in <buildworkspace>@libs\sharedLibrary on
          Using strategy: Default
          [poll] Last Built Revision: Revision b00d5f0ce8cbf798ff613ac61ec5ad28824afd6b (refs/remotes/origin/master)
          Using strategy: Default
          [poll] Last Built Revision: Revision fec1077f7d8b5a9b808abd0da6a84966ebe900df (refs/remotes/origin/master)
          Using strategy: Default
          [poll] Last Built Revision: Revision 4895bd129cdbd7678807c88f372fdc0b37295cee (refs/remotes/origin/master)
          Done. Took 0.26 sec
          Changes found
          {code}
           if the workspaces are available, there is no problem and i get outputs like:

          Ignored commit: <sha1> : No paths matched included region whitelist
           Ignored commit: <sha1> : No paths matched included region whitelist

          excluded user: <excludedUser>

           

          {color:#de350b}*And one additinal information:*{color} the jobs run on a farm of various slaves using one common label ]

          Cenk Tosun added a comment -

          markewaite - I guess workspace polling always needs the same workspace and causes problems when the next build runs on a different slave, correct?

          Cenk Tosun added a comment - markewaite - I guess workspace polling always needs the same workspace and causes problems when the next build runs on a different slave, correct?

            daspilker Daniel Spilker
            yrsurya suryatej yaramada
            Votes:
            6 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated: