Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39615

Global Pipeline Libraries triggers the 'poll SCM' of jobs

    XMLWordPrintable

Details

    • Bug
    • Status: Reopened (View Workflow)
    • Major
    • Resolution: Unresolved
    • None
    • linux rhel7.2
      jenkins 2.19.2
      up to date 'workflow' set of plugins (on stable update center)

    Description

      I have this situation where :

      • I have some jobs that gets sources from a SCM, and performs a 'poll SCM' hourly
      • I have a 'global pipeline lib' defined, on another SCM (doesn't matter really, what atters is it's not in the same source tree s the jobs)

      My issue is that any commit to the 'global pipeline lib' SCM will be detected by all the jobs that have a 'poll SCM' configured.

      I wish I can exclude the global lib from polling.

      (and yes, I know, polling is evil, whenever I can I have triggers configured on modern scm to push jobs, but sometimes it's just not possible, believe me you don't want to know all the why's)

      Attachments

        Issue Links

          Activity

            I allow myself to increase criticality. Why ? because in my infrastructure I have dozens of jenkins master with plenty of jobs in each.

            Our target is to share a common lib, and this feature is really welcome.

            Matter is : each commit to the lib leads to hundreds of jobs triggered. So for now we stopped the deployment of this feature. There is no easy workaround in our case. (again : if I could get rid of poll-scm I would, but I can't)

            squalou squalou jenkins added a comment - I allow myself to increase criticality. Why ? because in my infrastructure I have dozens of jenkins master with plenty of jobs in each. Our target is to share a common lib, and this feature is really welcome. Matter is : each commit to the lib leads to hundreds of jobs triggered. So for now we stopped the deployment of this feature. There is no easy workaround in our case. (again : if I could get rid of poll-scm I would, but I can't)
            jglick Jesse Glick added a comment -

            Probably same as JENKINS-38679; needs investigation and a minimal test case.

            jglick Jesse Glick added a comment - Probably same as JENKINS-38679 ; needs investigation and a minimal test case.

            I can explain a very simple testcase if you want.

            setup

            declare a workflow lib, call it 'test-lib', in jenkins global settings, pointing, say, to GIT some-test-ib-repo

            create a workflow job

            Point the pipeline to a Jenkinsfile hosted another GIT repo (some-repo/some-project.git)

            stage ('dummy'){
            println 'I have run'
            }
            

            Tick 'POLL SCM', H/5 * * * * should do

            test

            submit anything on 'some-test-ib-repo', which is NOT the repo of the scm declared in the pipeline

            wait 5 minutes

            the job runs.

            squalou squalou jenkins added a comment - I can explain a very simple testcase if you want. setup declare a workflow lib, call it 'test-lib', in jenkins global settings, pointing, say, to GIT some-test-ib-repo create a workflow job Point the pipeline to a Jenkinsfile hosted another GIT repo (some-repo/some-project.git) stage ( 'dummy' ){ println 'I have run' } Tick 'POLL SCM', H/5 * * * * should do test submit anything on 'some-test-ib-repo', which is NOT the repo of the scm declared in the pipeline wait 5 minutes the job runs.
            roidelapluie Julien Pivotto added a comment - - edited

            I even tried to add the hudson.plugins.git.extensions.impl.IgnoreNotifyCommit() extention without success.

            EDIT: confirmed that it does not work

            roidelapluie Julien Pivotto added a comment - - edited I even tried to add the hudson.plugins.git.extensions.impl.IgnoreNotifyCommit() extention without success. EDIT: confirmed that it does not work
            roidelapluie Julien Pivotto added a comment - - edited

            squalou A workaround that might work:

            EDIT: confirmed that it does not work

            roidelapluie Julien Pivotto added a comment - - edited squalou A workaround that might work: EDIT: confirmed that it does not work

            "(and yes, I know, polling is evil, whenever I can I have triggers configured on modern scm to push jobs, but sometimes it's just not possible, believe me you don't want to know all the why's)"

            It is not evil. We use polling without schedule (https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-Pushnotificationfromrepository) so it is more pushing than polling, and still have the problem.

            roidelapluie Julien Pivotto added a comment - "(and yes, I know, polling is evil, whenever I can I have triggers configured on modern scm to push jobs, but sometimes it's just not possible, believe me you don't want to know all the why's)" It is not evil. We use polling without schedule ( https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin#GitPlugin-Pushnotificationfromrepository ) so it is more pushing than polling, and still have the problem.
            munkyboy Mike added a comment -

            I ran into this problem as well. I found a workaround for my setup.

            I'm using bitbucket which offers a ssh and http clone url for repos. For the global library config, I use the http clone url. For the actual job tied the library (and corresponding bitbucket hook), I use the ssh url.

            munkyboy Mike added a comment - I ran into this problem as well. I found a workaround for my setup. I'm using bitbucket which offers a ssh and http clone url for repos. For the global library config, I use the http clone url. For the actual job tied the library (and corresponding bitbucket hook), I use the ssh url.
            mathieulaude Mathieu LL added a comment -

            Workaround that works for me is Additional Behaviours "Don't trigger a build on commit notifications" on the Global Pipeline Libraries configuration :

            mathieulaude Mathieu LL added a comment - Workaround that works for me is Additional Behaviours "Don't trigger a build on commit notifications" on the Global Pipeline Libraries configuration :

            Can pleeease someone finally fix this?

            Especially when pipeline libraries are under heavy development, changes triggering ALL builds makes it impossible to work efficiently. Since "don't trigger a build on commit notifications" isn't available anymore and we can not use push notifications, there is no workaround.

            lordjaxom Sascha Volkenandt added a comment - Can pleeease someone finally fix this? Especially when pipeline libraries are under heavy development, changes triggering ALL builds makes it impossible to work efficiently. Since "don't trigger a build on commit notifications" isn't available anymore and we can not use push notifications, there is no workaround.
            dolphy David Crouch added a comment - - edited

            Confirmed that poll SCM jobs are still triggered by every change to a shared library. What's the way forward on this? There are several duplicate JIRAs which describe this issue, but I don't see any open, active ones. Does this one need to be reopened, or does a new Major/Critical Bug need to be written?

            dolphy David Crouch added a comment - - edited Confirmed that poll SCM jobs are still triggered by every change to a shared library. What's the way forward on this? There are several duplicate JIRAs which describe this issue, but I don't see any open, active ones. Does this one need to be reopened, or does a new Major/Critical Bug need to be written?
            philmcardlecg Phil McArdle added a comment -

            I don't want to diminish your obvious issue, but the fix for JENKINS-41497 fixed this for a lot of us. As an example, my SCM config looks like the following, if you want to compare differences (if there are any):

            philmcardlecg Phil McArdle added a comment - I don't want to diminish your obvious issue, but the fix for JENKINS-41497 fixed this for a lot of us. As an example, my SCM config looks like the following, if you want to compare differences (if there are any):
            mcrooney mcrooney added a comment - - edited

            Still seeing this issue, even with workflow-cps-global-lib 2.9 (and Jenkins 2.89.2), and it sounds like others are as well. I've verified that "Include @Library changes in job recent changes" is unchecked, and yet pushes are still triggering new builds of every job that uses them for each commit (not just one time after the fix). Obviously incredibly invasive to have potentially dozens or hundreds of jobs take up all your executors when you update your global library.

            mcrooney mcrooney added a comment - - edited Still seeing this issue, even with workflow-cps-global-lib 2.9 (and Jenkins 2.89.2), and it sounds like others are as well. I've verified that "Include @Library changes in job recent changes" is unchecked, and yet pushes are still triggering new builds of every job that uses them for each commit (not just one time after the fix). Obviously incredibly invasive to have potentially dozens or hundreds of jobs take up all your executors when you update your global library.

            Still seeing this as well, what I found interesting is that it seems to check for the current version on the SLAVE which doesn't make sense (our jobs pick slave depending on availability so they will not always run on the same node.)

            tario Patrick Ruckstuhl added a comment - Still seeing this as well, what I found interesting is that it seems to check for the current version on the SLAVE which doesn't make sense (our jobs pick slave depending on availability so they will not always run on the same node.)
            mrkita Morgan Kita added a comment -

            I have the same behavior, where my library is integrated into my main code repository.

            In our Jenkins we have build jobs that are based off of multiple branches, and in each job where the library is included (the master version of the same repo by default), there are two entries in the polling log. One for the Library (at master version), and one for the actual checkout in the pipeline definition.

            This is a big problem, because the polling is marked as run for the library entry, but if we the pipeline checks out a different branch (and polls on it as is intended), it always returns that there are changes.

            This leads to infinitely triggered builds as it never updates the status properly.

            I could pull out the library code into it's own repo, but I assume that means it will still show up in the polling log, which it should not.

            I have unchecked the "Include @Library changes in job recent changes" flag in the global library configuration and since we use the "library step", I have set changelog: false in all the files which use it.

            mrkita Morgan Kita added a comment - I have the same behavior, where my library is integrated into my main code repository. In our Jenkins we have build jobs that are based off of multiple branches, and in each job where the library is included (the master version of the same repo by default), there are two entries in the polling log. One for the Library (at master version), and one for the actual checkout in the pipeline definition. This is a big problem, because the polling is marked as run for the library entry, but if we the pipeline checks out a different branch (and polls on it as is intended), it always returns that there are changes. This leads to infinitely triggered builds as it never updates the status properly. I could pull out the library code into it's own repo, but I assume that means it will still show up in the polling log, which it should not. I have unchecked the "Include @Library changes in job recent changes" flag in the global library configuration and since we use the "library step", I have set changelog: false in all the files which use it.
            skp SK added a comment -

            Can someone finally fix this please? It is really annoying, especially because the "Include @Library changes in job recent changes" option has no effect.

            Everytime I work on my shared library all multibranch pipeline jobs are going to run.

            skp SK added a comment - Can someone finally fix this please? It is really annoying, especially because the "Include @Library changes in job recent changes" option has no effect. Everytime I work on my shared library all multibranch pipeline jobs are going to run.
            jackiexiao Jackie Xiao added a comment -

            Also facing this issue, we have lots of microservice CI/CD jobs configured, which depend on the shared library, any change made to the shared lib will trigger all downstream jobs.

            Looking forward to the fix.

            jackiexiao Jackie Xiao added a comment - Also facing this issue, we have lots of microservice CI/CD jobs configured, which depend on the shared library, any change made to the shared lib will trigger all downstream jobs. Looking forward to the fix.

            For everyone who may still facing this problem, please refer to the workaround mentioned here. I can confirm it solves the problem.

            eocampos Eliseo Ocampos added a comment - For everyone who may still facing this problem, please refer to the workaround mentioned here . I can confirm it solves the problem.

            Running a full pipeline might take a long time... especially if you have 10s or 100s that are affected by the issue. Does anyone know if there's a way to alter the config proactively (manually) via the job's XML file?

            brianjenkins Brian Villanueva added a comment - Running a full pipeline might take a long time... especially if you have 10s or 100s that are affected by the issue. Does anyone know if there's a way to alter the config proactively (manually) via the job's XML file?
            chris_knight Chris Knight added a comment -

            "Include @Library changes in job recent changes" did fix this issue for us.  However, importantly, we had to restart our Jenkins server for this to take effect. 

            chris_knight Chris Knight added a comment - "Include @Library changes in job recent changes" did fix this issue for us.  However, importantly, we had to restart our Jenkins server for this to take effect. 
            richmond Rich added a comment -

            For me, "Include @Library changes in job recent changes" did stop the changes from appearing in change log but a change in shared library is still triggering the build.

            richmond Rich added a comment - For me, "Include @Library changes in job recent changes" did stop the changes from appearing in change log but a change in shared library is still triggering the build.

            In our case we hade changelogs for global shared libraries enabled but wanted to disable them to avoid rebuilds on library changes. However, doing so made all jobs which currently are failing, entering a infinite build loop due to scm poll. I have figured out that the the scm poll seems to always consider all repositories with changelog/state of the last sucessfull build. This can even be a build you don't see in the web interface. I found following cure:

            1. Go to the SCM Poll log of the job and find the "old" state (For us this is subversion: "changed from <xxxxx>")
            2. Find file "permalinks" in  ${JENKINS_HOME}/jobs/<affected project>/builds
            3. Within that file get the build number of last successfull build
            4. Edit ${JENKINS_HOME}/jobs/<affected project>/builds/<build_nr>/build.xml. (Probably make  copy of that file before) In that file, find tag "revisionStates". This tag will have subtags "entry" were one of this subtags will match the affected repo and "old state" determined above. Remove the entire entry. Save file.
            amesser Andreas Messer added a comment - In our case we hade changelogs for global shared libraries enabled but wanted to disable them to avoid rebuilds on library changes. However, doing so made all jobs which currently are failing, entering a infinite build loop due to scm poll. I have figured out that the the scm poll seems to always consider all repositories with changelog/state of the last sucessfull build. This can even be a build you don't see in the web interface. I found following cure: Go to the SCM Poll log of the job and find the "old" state (For us this is subversion: "changed from <xxxxx>") Find file "permalinks" in  ${JENKINS_HOME}/jobs/<affected project>/builds Within that file get the build number of last successfull build Edit ${JENKINS_HOME}/jobs/<affected project>/builds/<build_nr>/build.xml. (Probably make  copy of that file before) In that file, find tag "revisionStates". This tag will have subtags "entry" were one of this subtags will match the affected repo and "old state" determined above. Remove the entire entry. Save file.

            People

              Unassigned Unassigned
              squalou squalou jenkins
              Votes:
              43 Vote for this issue
              Watchers:
              57 Start watching this issue

              Dates

                Created:
                Updated: