Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41497

Poll SCM and Timer triggers include "Changes" for a Pipeline for any/all Shared Libraries

    XMLWordPrintable

    Details

    • Similar Issues:
    • Sprint:
      Pipeline - July/August

      Description

      I noticed this while filing another ticket (JENKINS-41496), but the "Changes" view for the Pipeline run for this project "azure" has the "wrong commits" shown. See the screenshots for more.

      What basically appears to be happening is that any change to a Shared Library will result in newly triggered Pipelines which have "Poll SCM" configured. Pipelines which configure a timer will also have Changes from the Shared Library listed when it executes again.

      I think listing the Shared Library commits in "Changes" is acceptable, but triggering based on an SCM Poll for a Pipeline is very confusing behavior and IMHO incorrect behavior.

      As a shared tooling team, I would not expect my Shared Library changes to trigger a bunch of Pipelines for projects depending on them.

        Attachments

          Issue Links

            Activity

            rtyler R. Tyler Croy created issue -
            Hide
            cliffmeyers Cliff Meyers added a comment - - edited

            Could you please paste the JSON response from the REST API call to .../runs/$runId?

            Show
            cliffmeyers Cliff Meyers added a comment - - edited Could you please paste the JSON response from the REST API call to .../runs/$runId?
            Hide
            rtyler R. Tyler Croy added a comment -

            This looks like it's an underlying behavior in Pipeline.

            Show
            rtyler R. Tyler Croy added a comment - This looks like it's an underlying behavior in Pipeline.
            rtyler R. Tyler Croy made changes -
            Field Original Value New Value
            Attachment classic-changes-view.png [ 35641 ]
            Component/s pipeline [ 21692 ]
            Component/s blueocean-plugin [ 21481 ]
            Description I noticed this while filing another ticket (JENKINS-41496), but the "Changes" view for the Pipeline run for this project "azure" has the **wrong commits** shown. See the screenshots for more.

            I'm not sure why.
            I noticed this while filing another ticket (JENKINS-41496), but the "Changes" view for the Pipeline run for this project "azure" has the "wrong commits" shown. See the screenshots for more.


            What basically appears to be happening is that any change to a Shared Library will result in _newly_ triggered Pipelines which have "Poll SCM" configured. Pipelines which configure a timer will also have Changes from the Shared Library listed when it executes again.


            I think *listing* the Shared Library commits in "Changes" is acceptable, but *triggering* based on an SCM Poll for a Pipeline is very confusing behavior and IMHO incorrect behavior.

            As a shared tooling team, I would not expect my Shared Library changes to trigger a bunch of Pipelines for projects depending on them.
            Priority Major [ 3 ] Minor [ 4 ]
            Summary Blue Ocean "Changes" for a Pipeline are from the completely wrong repository Poll SCM and Timer triggers include "Changes" for a Pipeline for any/all Shared Libraries
            abayer Andrew Bayer made changes -
            Component/s workflow-cps-global-lib-plugin [ 21714 ]
            Component/s workflow-job-plugin [ 21716 ]
            Component/s workflow-scm-step-plugin [ 21717 ]
            Component/s pipeline [ 21692 ]
            Hide
            jglick Jesse Glick added a comment -

            As designed, but an RFE to disable changelog and/or polling for library checkouts.

            Show
            jglick Jesse Glick added a comment - As designed, but an RFE to disable changelog and/or polling for library checkouts.
            jglick Jesse Glick made changes -
            Component/s workflow-job-plugin [ 21716 ]
            Component/s workflow-scm-step-plugin [ 21717 ]
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-41442 [ JENKINS-41442 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-39615 [ JENKINS-39615 ]
            jglick Jesse Glick made changes -
            Link This issue is duplicated by JENKINS-39499 [ JENKINS-39499 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-42320 [ JENKINS-42320 ]
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-38659 [ JENKINS-38659 ]
            Hide
            tsniatowski Tomasz Śniatowski added a comment -

            I think this is a serious scaling problem that a tiny shared library change can trigger a huge number of jobs. Any chance an improvement to make rebuilds optional gets picked up? I know that in our deployment we won't be able to afford such rebuilds.

            Show
            tsniatowski Tomasz Śniatowski added a comment - I think this is a serious scaling problem that a tiny shared library change can trigger a huge number of jobs. Any chance an improvement to make rebuilds optional gets picked up? I know that in our deployment we won't be able to afford such rebuilds.
            Hide
            hogarthj James Hogarth added a comment -

            Agreed ... we're looking to shared libraries to augment existing Jenkinsfile directives/steps (such as cleaner artifactory integration or change management activities in JIRA not possible with the present JIRA plugin) 

            We really don't want to have commits to the implicitly loaded shared library (or even explicitly where a project does that, but implicit is the real problem as it affects all jobs) to add a new feature, or a minor bugfix, to display in the changelog of jobs much less trigger jobs across the estate.

             

            We're using the Bitbucket source plugin with webhook plugin on Bitbucket and a very basic global shared library modern SCM (git) configuration for this.

            I'll be doing some digging for any possible workarounds etc today.

            Show
            hogarthj James Hogarth added a comment - Agreed ... we're looking to shared libraries to augment existing Jenkinsfile directives/steps (such as cleaner artifactory integration or change management activities in JIRA not possible with the present JIRA plugin)  We really don't want to have commits to the implicitly loaded shared library (or even explicitly where a project does that, but implicit is the real problem as it affects all jobs) to add a new feature, or a minor bugfix, to display in the changelog of jobs much less trigger jobs across the estate.   We're using the Bitbucket source plugin with webhook plugin on Bitbucket and a very basic global shared library modern SCM (git) configuration for this. I'll be doing some digging for any possible workarounds etc today.
            Hide
            hogarthj James Hogarth added a comment -

            Actually ... on now doing some initial investigation

            I don't think triggering is actually happening, just the changes of the shared library being included in the changelog

            Tomasz Śniatowski are you sure you saw actual triggering of jobs rather than just a  job properly triggered then also including the changelog of the library in the job detail?

            Show
            hogarthj James Hogarth added a comment - Actually ... on now doing some initial investigation I don't think triggering is actually happening, just the changes of the shared library being included in the changelog Tomasz Śniatowski are you sure you saw actual triggering of jobs rather than just a  job properly triggered then also including the changelog of the library in the job detail?
            Hide
            hogarthj James Hogarth added a comment -

            Now I'm certain the webhook from Bitbucket is not triggering the job when a change is made to the shared library, but not the actual job repo.

            It's also not triggering on a general reindexing by the Bitbucket plugin of that project

            It is triggering a build if the Jenkinsfile has properties set to carry out an SCM poll, but this can be mitigated in configuring the global library but configuring the additional behaviour "polling ignores commits with certain messages" on the SCM config for the global library with a '.*' expression.

            This does still have library changes appear in the recent changes of the job though.

            Show
            hogarthj James Hogarth added a comment - Now I'm certain the webhook from Bitbucket is not triggering the job when a change is made to the shared library, but not the actual job repo. It's also not triggering on a general reindexing by the Bitbucket plugin of that project It is triggering a build if the Jenkinsfile has properties set to carry out an SCM poll, but this can be mitigated in configuring the global library but configuring the additional behaviour "polling ignores commits with certain messages" on the SCM config for the global library with a '.*' expression. This does still have library changes appear in the recent changes of the job though.
            Hide
            philmcardlecg Phil McArdle added a comment - - edited

            It is triggering a build if the Jenkinsfile has properties set to carry out an SCM poll, but this can be mitigated in configuring the global library but configuring the additional behaviour "polling ignores commits with certain messages" on the SCM config for the global library with a '.*' expression.

            I've seen similar suggestions for workarounds in other issues (JENKINS-39615), but some official answer would be nice, especially as many of us use SCM polling (for one reason or another) and the global pipeline library defaults to settings that will trigger builds which use polling.

            Show
            philmcardlecg Phil McArdle added a comment - - edited It is triggering a build if the Jenkinsfile has properties set to carry out an SCM poll, but this can be mitigated in configuring the global library but configuring the additional behaviour "polling ignores commits with certain messages" on the SCM config for the global library with a '.*' expression. I've seen similar suggestions for workarounds in other issues ( JENKINS-39615 ), but some official answer would be nice, especially as many of us use SCM polling (for one reason or another) and the global pipeline library defaults to settings that will trigger builds which use polling.
            hogarthj James Hogarth made changes -
            Issue Type Improvement [ 4 ] Bug [ 1 ]
            hogarthj James Hogarth made changes -
            Issue Type Bug [ 1 ] Improvement [ 4 ]
            Hide
            hogarthj James Hogarth added a comment -

            Jesse Glick could you please take a look at my PR for the changeset part of this.

            https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/34

            Testing this in our internal test instance it behaves as expected so far as I can tell with commits to the shared library not included in the recent changes of teh job or the change details of a build when the checkbox is unticked.

            With the checkbox ticked it behaves as it currently does with the changeset of the shared library, if it has changed so the changeset exists, being added to the changes page of the build and appearing in recent changes of the job.

            I've defaulted it to show include the changeset of the library as that matches the current behaviour.

            Show
            hogarthj James Hogarth added a comment - Jesse Glick could you please take a look at my PR for the changeset part of this. https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/34 Testing this in our internal test instance it behaves as expected so far as I can tell with commits to the shared library not included in the recent changes of teh job or the change details of a build when the checkbox is unticked. With the checkbox ticked it behaves as it currently does with the changeset of the shared library, if it has changed so the changeset exists, being added to the changes page of the build and appearing in recent changes of the job. I've defaulted it to show include the changeset of the library as that matches the current behaviour.
            Hide
            mathieulaude Mathieu LL added a comment -

            James Hogarth on your test instance, if your push some modifications on a shared library, does it trigger a job using this library with SCM polling activated (and your checkbox unticked) ?

            Show
            mathieulaude Mathieu LL added a comment - James Hogarth on your test instance, if your push some modifications on a shared library, does it trigger a job using this library with SCM polling activated (and your checkbox unticked) ?
            Hide
            hogarthj James Hogarth added a comment -

            Mathieu LL when digging in to this I realised that particular problem isn't actually an issue with this plugin.

            You can see here that the plugin actually sets setPoll to false.

            It's actually SCMTrigger that is responsible for that, but I don't (from a cursory look at the code) think it has any way of differentiating an SCM associated with a Job from a library versus any other scm checkout.

            For now I suggest using the workaround on that with the "skip if path affected" or "skip based on message" set to a wildcard of everything.

            I'm hoping this is acceptable to Jesse etc at least though so the confusion from mixed changesets can at least be addressed as we'd like to see this change in our workplace.

            Show
            hogarthj James Hogarth added a comment - Mathieu LL when digging in to this I realised that particular problem isn't actually an issue with this plugin. You can see here  that the plugin actually sets setPoll to false. It's actually  SCMTrigger  that is responsible for that, but I don't (from a cursory look at the code) think it has any way of differentiating an SCM associated with a Job from a library versus any other scm checkout. For now I suggest using the workaround on that with the "skip if path affected" or "skip based on message" set to a wildcard of everything. I'm hoping this is acceptable to Jesse etc at least though so the confusion from mixed changesets can at least be addressed as we'd like to see this change in our workplace.
            Hide
            mathieulaude Mathieu LL added a comment -

            OK, thanks for the explanation. I'll keep the workaround for now.

            Show
            mathieulaude Mathieu LL added a comment - OK, thanks for the explanation. I'll keep the workaround for now.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-42809 [ JENKINS-42809 ]
            Hide
            ghenzler Georg Henzler added a comment -

            For us it is not only the triggers that cause the pain - we use the changes as reported also to set the build name (and do some other magic with the derived JIRA IDs). Obviously those changes can be filtered, but it results in unnecessary complicated code and inconsistent UI (since the changes as reported always show up the changes list in the very left column of the pipeline stage view).

            If someone could merge https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/34 it would be highly appreciated!

            Show
            ghenzler Georg Henzler added a comment - For us it is not only the triggers that cause the pain - we use the changes as reported also to set the build name (and do some other magic with the derived JIRA IDs). Obviously those changes can be filtered, but it results in unnecessary complicated code and inconsistent UI (since the changes as reported always show up the changes list in the very left column of the pipeline stage view). If someone could merge https://github.com/jenkinsci/workflow-cps-global-lib-plugin/pull/34 it would be highly appreciated!
            casz Joseph Petersen (old) made changes -
            Remote Link This issue links to "PR 34 (Web Link)" [ 16918 ]
            Hide
            maxfields2000 Maxfield Stewart added a comment - - edited

            I can definitely confirm this problem is real and incredibly challenging to deal with (running an instance with thousands of jobs, of which hundreds use the library to do things like deploys etc ).  

            I created a simple test case where I make a Pipeline job and just have it use a simple pipeline:

            echo "Hello World"!

            I then configure my global library and implicitly included.  On the piplein job I turn on polling. Then I commit a simple readme chagne to my pipeline library and watch polling trip my job when it changes.

            I even have :

            "Ignore on push notifications" AND

            "Don't trigger a build on commit notifications" (the Advanced clone behaviors) set

            Changes to the library will /always/ trigger the job.  Which makes the feature basically useless at scale. So I'm all for the PR above that allows us to disable including it in the polling logs from the central config page in the implicitly loaded case.  Without it, we basically have to sit and hope that some day this will work correctly. We're falling back to the old way of having the library pushed to the central location via the SSH/exposed git endpoint (which never had this problem because it was never included in the polling logs).

            Show
            maxfields2000 Maxfield Stewart added a comment - - edited I can definitely confirm this problem is real and incredibly challenging to deal with (running an instance with thousands of jobs, of which hundreds use the library to do things like deploys etc ).   I created a simple test case where I make a Pipeline job and just have it use a simple pipeline: echo "Hello World"! I then configure my global library and implicitly included.  On the piplein job I turn on polling. Then I commit a simple readme chagne to my pipeline library and watch polling trip my job when it changes. I even have : "Ignore on push notifications" AND "Don't trigger a build on commit notifications" (the Advanced clone behaviors) set Changes to the library will /always/ trigger the job.  Which makes the feature basically useless at scale. So I'm all for the PR above that allows us to disable including it in the polling logs from the central config page in the implicitly loaded case.  Without it, we basically have to sit and hope that some day this will work correctly. We're falling back to the old way of having the library pushed to the central location via the SSH/exposed git endpoint (which never had this problem because it was never included in the polling logs).
            Hide
            hogarthj James Hogarth added a comment -

            Maxfield Stewart - did you try the workaround I suggested a little way above?

            In the testing I did this way did disable polling of the share library when polling was set on the job itself.

            For now I suggest using the workaround on that with the "skip if path affected" or "skip based on message" set to a wildcard of everything.
            Show
            hogarthj James Hogarth added a comment - Maxfield Stewart - did you try the workaround I suggested a little way above? In the testing I did this way did disable polling of the share library when polling was set on the job itself. For now I suggest using the workaround on that with the "skip if path affected" or "skip based on message" set to a wildcard of everything.
            Hide
            maxfields2000 Maxfield Stewart added a comment - - edited

            That workaround does not work if you have no workspace. The git plugin requires creating a workspace in order to do that kind of filtering.  We have a use case where in many cases there is no workspace (it's by design, we dynamically create space for jobs to run) until the job is triggered.  When you use the filtering feature and you have no worksapce it will trigger your job even if your central scripts don't change (because it has to make a workspace) which then results in infinitely triggering jobs. So really this is a direct ask to please make a simple toggle that says "Do not include in polling or change history"  or better yet have that the default behavior and have an option that says "include in polling/change history".

            Prior to this "Global Library" feature, the "central library" approach by using SSH/Git server did not include itself in any polling logs or results. This new implementation of Global Libraries should also not include itself, or it's changelogs, as part of the "job" by default (I can see why you might want to have it do so in some cases)

            Otherwise, this feature is a nice toy but not really scalable (you really don't want 5,000 jobs firing because you changed a shared function).  Filters (even if you have permanent workspaces) are wholey inconvenient. The work arounds here for others in this shape are to revert to using the old way of globally sharing code (which isn't branch aware or referenceable). Be advised though that any functions in that library that have the same name as other libraries will trump the other libraries.

            For now we're furiously working to tell our job owners to convert to web hooks entirely and stop using Polling (I wish I could disable polling entirely as a jenkins wide thing).

            Show
            maxfields2000 Maxfield Stewart added a comment - - edited That workaround does not work if you have no workspace. The git plugin requires creating a workspace in order to do that kind of filtering.  We have a use case where in many cases there is no workspace (it's by design, we dynamically create space for jobs to run) until the job is triggered.  When you use the filtering feature and you have no worksapce it will trigger your job even if your central scripts don't change (because it has to make a workspace) which then results in infinitely triggering jobs. So really this is a direct ask to please make a simple toggle that says "Do not include in polling or change history"  or better yet have that the default behavior and have an option that says "include in polling/change history". Prior to this "Global Library" feature, the "central library" approach by using SSH/Git server did not include itself in any polling logs or results. This new implementation of Global Libraries should also not include itself, or it's changelogs, as part of the "job" by default (I can see why you might want to have it do so in some cases) Otherwise, this feature is a nice toy but not really scalable (you really don't want 5,000 jobs firing because you changed a shared function).  Filters (even if you have permanent workspaces) are wholey inconvenient. The work arounds here for others in this shape are to revert to using the old way of globally sharing code (which isn't branch aware or referenceable). Be advised though that any functions in that library that have the same name as other libraries will trump the other libraries. For now we're furiously working to tell our job owners to convert to web hooks entirely and stop using Polling (I wish I could disable polling entirely as a jenkins wide thing).
            Hide
            endeepak Deepak Narayana Rao added a comment -
            this can be mitigated in configuring the global library but configuring the additional behaviour "polling ignores commits with certain messages" on the SCM config for the global library with a '.*' expression
            

            The above workaround doesn't seem to be working in Jenkins 2.58. Is there a specific version jenkins and a plugin needed for this workaround?

            Show
            endeepak Deepak Narayana Rao added a comment - this can be mitigated in configuring the global library but configuring the additional behaviour "polling ignores commits with certain messages" on the SCM config for the global library with a '.*' expression The above workaround doesn't seem to be working in Jenkins 2.58. Is there a specific version jenkins and a plugin needed for this workaround?
            Hide
            adityacs Aditya C S added a comment -

            Workaround suggested doesn't work. Using Jenkins 2.60.1. This is a critical issue for us as we are not able to scale using shared library. Hope this issue get's fixed soon.

            Show
            adityacs Aditya C S added a comment - Workaround suggested doesn't work. Using Jenkins 2.60.1. This is a critical issue for us as we are not able to scale using shared library. Hope this issue get's fixed soon.
            Hide
            dolphy David Crouch added a comment -

            All workarounds suggested only make the issue worse for us. With no nothing additional configured in settings for the global shared library, Jenkins will kick every build using that shared library when any change is committed. When any kind of filtering is attempted (Polling ignores commits, Don't trigger a build on commits, etc), Jenkins will kick every build using that shared library every single polling interval, regardless of whether or not anything changed.

            This is a critical issue for; we have completely stopped using shared libraries because of this issue.

             

            Show
            dolphy David Crouch added a comment - All workarounds suggested only make the issue worse for us. With no nothing additional configured in settings for the global shared library, Jenkins will kick every build using that shared library when any change is committed. When any kind of filtering is attempted (Polling ignores commits, Don't trigger a build on commits, etc), Jenkins will kick every build using that shared library  every single polling interval , regardless of whether or not anything changed. This is a critical issue for; we have completely stopped using shared libraries because of this issue.  
            Hide
            dolphy David Crouch added a comment -

            We found a workaround that works for us: stop controlling the shared library as a GitHub project and control it in a legacy SCM (i.e. Perforce) instead. Polling directives (Polling ignores changes based on <X>) do seem to work just fine when we house the shared library in a legacy SCM, whereas those directives don't seem to work with the GitHub retrieval implementation (or make it worse, as described above).

            Show
            dolphy David Crouch added a comment - We found a workaround that works for us: stop controlling the shared library as a GitHub project and control it in a legacy SCM (i.e. Perforce) instead. Polling directives (Polling ignores changes based on <X>) do seem to work just fine when we house the shared library in a legacy SCM, whereas those directives don't seem to work with the GitHub retrieval implementation (or make it worse, as described above).
            Hide
            tknerr Torben Knerr added a comment -

            Wondering if "minor" is the correct priority / criticality for this.

            After all, this renders the whole global-shared-library feature useless. It's the showstopper why we didn't upgrade our jenkins since 2.32.1...

            Does anyone at cloudbees actually use global shared libraries like this, or are we doing it wrong?

            /ping Jesse Glick

            Show
            tknerr Torben Knerr added a comment - Wondering if "minor" is the correct priority / criticality for this. After all, this renders the whole global-shared-library feature useless. It's the showstopper why we didn't upgrade our jenkins since 2.32.1... Does anyone at cloudbees actually use global shared libraries like this, or are we doing it wrong? /ping Jesse Glick
            Hide
            xoue Guillermo Solla added a comment - - edited

            Torben Knerr The best way to get rid of this issue is changing the pipeline trigger model to webhooks, this way there is no trigger on the Library repo itself, just in the main projects. But as this is an issue for the polling method. It does not apply... but it's fine to remind it I guess.

             

            Although the issue of not been able to refine changeset still applies when using webhooks.

            Show
            xoue Guillermo Solla added a comment - - edited Torben Knerr The best way to get rid of this issue is changing the pipeline trigger model to webhooks, this way there is no trigger on the Library repo itself, just in the main projects. But as this is an issue for the polling method. It does not apply... but it's fine to remind it I guess.   Although the issue of not been able to refine changeset still applies when using webhooks.
            Hide
            tknerr Torben Knerr added a comment - - edited

            I wasn't aware this is a polling only issue, thanks for the pointer Guillermo Solla!

            Still, we can't use webhooks due to our firewall setup currently  

            Show
            tknerr Torben Knerr added a comment - - edited I wasn't aware this is a polling only issue, thanks for the pointer  Guillermo Solla ! Still, we can't use webhooks due to our firewall setup currently  
            Hide
            jglick Jesse Glick added a comment -

            I am aware of the issue but have not had a chance to work on it.

            Show
            jglick Jesse Glick added a comment - I am aware of the issue but have not had a chance to work on it.
            jamesdumay James Dumay made changes -
            Labels cloudbees-internal-pipeline
            Hide
            technolo_g Matthew Bajor added a comment - - edited

            From my understanding (with the git/github plugins anyways), a PUSH event from the webhook triggers the polling mechanism. This can be seen in the polling log on a job that is only configured for GHPush. The webhook was received:

            Jul 27, 2017 11:26:31 AM org.jenkinsci.plugins.github_branch_source.PushGHEventSubscriber onEvent
            INFO: Received PUSH for https://ghe.domain.com/my-org/code-repo from 10.52.7.244 ? http://jenkins.domain.com:8080/build-system-1/github-webhook/
            Jul 27, 2017 11:26:31 AM com.cloudbees.jenkins.GitHubPushTrigger$1 run
            INFO: SCM changes detected in Pipelines/code-repo. Triggering #5

             

            Then this output can be found in the polling log:

            Polling Log
            View as plain text
            This page captures the polling log that triggered this build.
            Started on Jul 27, 2017 11:26:24 AM
            Started by event from 10.52.7.244 ? http://jenkins.domain.com:8080/build-system-1/github-webhook/ on Thu Jul 27 11:26:24 MDT 2017
            Using strategy: Specific revision
            [poll] Last Built Revision: Revision caabb3d7fd98ea63ad76339eed6633590e78072b (master)
            using GIT_SSH to set credentials GitHub SSH Key ssh-poller-creds
             > git ls-remote -h git@ghe.domain.com:my-org/jenkins-libraries.git # timeout=10
            Found 1 remote heads on git@ghe.domain.com:my-org/jenkins-libraries.git
            [poll] Latest remote head revision on refs/heads/master is: 7ae6d912bcad95b6baffcd9bd24580b3dd1cf78d
            Using strategy: Default
            [poll] Last Built Revision: Revision 6e872eb81537e239fc7b00c8a1ac5878aafabff2 (refs/remotes/origin/master)
            using GIT_SSH to set credentials GitHub SSH Key ssh-poller-creds
             > git ls-remote -h git@ghe.domain.com:cban/code-repo.git # timeout=10
            Found 5 remote heads on git@ghe.domain.com:cban/code-repo.git
            [poll] Latest remote head revision on refs/heads/master is: 6e872eb81537e239fc7b00c8a1ac5878aafabff2 - already built by 4
            Done. Took 0.24 sec
            Changes found

             

            We may have a misconfiguration, but the webhook from a PR (from GHPRB) also triggers this mechanism so on every PR build, the master kicks off too when there are library changes. 

            Show
            technolo_g Matthew Bajor added a comment - - edited From my understanding (with the git/github plugins anyways), a PUSH event from the webhook triggers the polling mechanism. This can be seen in the polling log on a job that is only configured for GHPush. The webhook was received: Jul 27, 2017 11:26:31 AM org.jenkinsci.plugins.github_branch_source.PushGHEventSubscriber onEvent INFO: Received PUSH for https: //ghe.domain.com/my-org/code-repo from 10.52.7.244 ? http://jenkins.domain.com:8080/build-system-1/github-webhook/ Jul 27, 2017 11:26:31 AM com.cloudbees.jenkins.GitHubPushTrigger$1 run INFO: SCM changes detected in Pipelines/code-repo. Triggering #5   Then this output can be found in the polling log: Polling Log View as plain text This page captures the polling log that triggered this build. Started on Jul 27, 2017 11:26:24 AM Started by event from 10.52.7.244 ? http: //jenkins.domain.com:8080/build-system-1/github-webhook/ on Thu Jul 27 11:26:24 MDT 2017 Using strategy: Specific revision [poll] Last Built Revision: Revision caabb3d7fd98ea63ad76339eed6633590e78072b (master) using GIT_SSH to set credentials GitHub SSH Key ssh-poller-creds > git ls-remote -h git@ghe.domain.com:my-org/jenkins-libraries.git # timeout=10 Found 1 remote heads on git@ghe.domain.com:my-org/jenkins-libraries.git [poll] Latest remote head revision on refs/heads/master is: 7ae6d912bcad95b6baffcd9bd24580b3dd1cf78d Using strategy: Default [poll] Last Built Revision: Revision 6e872eb81537e239fc7b00c8a1ac5878aafabff2 (refs/remotes/origin/master) using GIT_SSH to set credentials GitHub SSH Key ssh-poller-creds > git ls-remote -h git@ghe.domain.com:cban/code-repo.git # timeout=10 Found 5 remote heads on git@ghe.domain.com:cban/code-repo.git [poll] Latest remote head revision on refs/heads/master is: 6e872eb81537e239fc7b00c8a1ac5878aafabff2 - already built by 4 Done. Took 0.24 sec Changes found   We may have a misconfiguration, but the webhook from a PR (from GHPRB) also triggers this mechanism so on every PR build, the master kicks off too when there are library changes. 
            michaelneale Michael Neale made changes -
            Priority Minor [ 4 ] Major [ 3 ]
            michaelneale Michael Neale made changes -
            Sprint Pipeline - July/August [ 371 ]
            michaelneale Michael Neale made changes -
            Assignee Andrew Bayer [ abayer ]
            Hide
            michaelneale Michael Neale added a comment -

            Andrew Bayer to investigate (may punt to sam if needed)

            Show
            michaelneale Michael Neale added a comment - Andrew Bayer to investigate (may punt to sam if needed)
            Hide
            jglick Jesse Glick added a comment -

            There is a proposed PR; last time I had any time to look at it, it needed some design work. Not sure of current status.

            Show
            jglick Jesse Glick added a comment - There is a proposed PR; last time I had any time to look at it, it needed some design work. Not sure of current status.
            Hide
            mathieulaude Mathieu LL added a comment - - edited

            With the latest update of Git Plugin, the trick of setting the "polling ignores commits with certain messages" option (as described by James Hogarth in comment above) is not possible anymore.

            Every push on my shared library triggers my multi-branch pipeline again this morning (15 branches of 1 hour each)... This issue becomes critical for me.

            Show
            mathieulaude Mathieu LL added a comment - - edited With the latest update of Git Plugin , the trick of setting the "polling ignores commits with certain messages" option (as described by James Hogarth in comment above) is not possible anymore. Every push on my shared library triggers my multi-branch pipeline again this morning (15 branches of 1 hour each)... This issue becomes critical for me.
            Hide
            kodstark Kamil Demecki added a comment - - edited

            It is critical issue for my setup too.

            Problem is even more complicated when custom branch pipelines are using shared library with checkout command for few git projects in parallel section (some of them are calling checkout with poll:true and some with poll:false). I use checkout command with refspec + checkout to separate directotry + repo name because of repeated checkouts. In such scenario polling log is pretty hard to use - it shows only information "Changes found" and a lot of entries "Ignoring refs/heads/...as it doesn't match any of the configured refspecs" - it is confusing as today single pipeline was triggered 30 times in two hours when number of commits was only few. How do you advise to narrow down the issue or potential not optimal using of checkout commands.

            Show
            kodstark Kamil Demecki added a comment - - edited It is critical issue for my setup too. Problem is even more complicated when custom branch pipelines are using shared library with checkout command for few git projects in parallel section (some of them are calling checkout with poll:true and some with poll:false). I use checkout command with refspec + checkout to separate directotry + repo name because of repeated checkouts. In such scenario polling log is pretty hard to use - it shows only information "Changes found" and a lot of entries "Ignoring refs/heads/...as it doesn't match any of the configured refspecs" - it is confusing as today single pipeline was triggered 30 times in two hours when number of commits was only few. How do you advise to narrow down the issue or potential not optimal using of checkout commands.
            Hide
            xy_alan Alan Yang added a comment -

            I have same problem.

            Show
            xy_alan Alan Yang added a comment - I have same problem.
            Hide
            initialzero Marc Benstein added a comment -

            At a minimum it would be nice to changelog and polling toggles for shared library. Only having ignore push notification isn't useful for polling.

            Show
            initialzero Marc Benstein added a comment - At a minimum it would be nice to changelog and polling toggles for shared library. Only having ignore push notification isn't useful for polling.
            Hide
            abayer Andrew Bayer added a comment -

            Merged the PR, releasing shortly.

            Show
            abayer Andrew Bayer added a comment - Merged the PR, releasing shortly.
            abayer Andrew Bayer made changes -
            Resolution Fixed [ 1 ]
            Status Open [ 1 ] Resolved [ 5 ]
            Hide
            abayer Andrew Bayer added a comment -

            Released as workflow-cps-global-lib 2.9.

            Show
            abayer Andrew Bayer added a comment - Released as workflow-cps-global-lib 2.9.
            abayer Andrew Bayer made changes -
            Link This issue is duplicated by JENKINS-45133 [ JENKINS-45133 ]
            giorgiosironi Giorgio Sironi made changes -
            Attachment global-pipeline-configuration.png [ 39811 ]
            Attachment triggered-build-log.png [ 39812 ]
            Hide
            giorgiosironi Giorgio Sironi added a comment - - edited

            Upgraded to 2.9, but still experience the same problem. See attached screenshot of the configuration of the global library and of one of the dozens of triggered builds.


            I can confirm I do not see the library commit in the triggered builds changelogs, but that seemed a minor problem with respect to this ticket's description, which is not to trigger all jobs everywhere.

            Show
            giorgiosironi Giorgio Sironi added a comment - - edited Upgraded to 2.9, but still experience the same problem. See attached screenshot of the configuration of the global library and of one of the dozens of triggered builds. I can confirm I do not see the library commit in the triggered builds changelogs, but that seemed a minor problem with respect to this ticket's description, which is not to trigger all jobs everywhere.
            Hide
            giorgiosironi Giorgio Sironi added a comment - - edited

            I tried switching Modern SCM from Git to Github and back, and recreating the configuration of the global library fearing some state was lingering from the previous version of the plugin.

            Sample output from one of the triggered builds demonstrating the commit message is coming from the shared library: https://gist.github.com/giorgiosironi/ef4f326d0e959cbd826d9a6cef769b44

            Show
            giorgiosironi Giorgio Sironi added a comment - - edited I tried switching Modern SCM from Git to Github and back, and recreating the configuration of the global library fearing some state was lingering from the previous version of the plugin. Sample output from one of the triggered builds demonstrating the commit message is coming from the shared library: https://gist.github.com/giorgiosironi/ef4f326d0e959cbd826d9a6cef769b44
            Hide
            giorgiosironi Giorgio Sironi added a comment -

            Rolled back to the internal /workflowLibs.git repository approach, the only workaround I know.

            Show
            giorgiosironi Giorgio Sironi added a comment - Rolled back to the internal /workflowLibs.git repository approach, the only workaround I know.
            Hide
            giorgiosironi Giorgio Sironi added a comment -

            Doesn't solve the problem - for some reason when I push to the Github equivalent of the internal /workflowLibs.git (used as a backup: https://github.com/elifesciences/elife-jenkins-workflow-libs), a web hook triggers all builds again. Now going through organizational and repository hooks to check what's happening.

            Show
            giorgiosironi Giorgio Sironi added a comment - Doesn't solve the problem - for some reason when I push to the Github equivalent of the internal /workflowLibs.git (used as a backup: https://github.com/elifesciences/elife-jenkins-workflow-libs ), a web hook triggers all builds again. Now going through organizational and repository hooks to check what's happening.
            Hide
            giorgiosironi Giorgio Sironi added a comment -

            The log says:

            Sep 25, 2017 9:55:48 AM INFO org.jenkinsci.plugins.github_branch_source.PushGHEventSubscriber onEvent
            
            Received PUSH for https://github.com/elifesciences/elife-jenkins-workflow-libs from 192.30.252.40 ⇒ http://ci--alfred.elifesciences.org:8080/github-webhook/
            
            Sep 25, 2017 9:55:48 AM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run
            
            SCM changes detected in library-api-validator-php. Triggering #22
            
            Sep 25, 2017 9:55:49 AM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run
            
            SCM changes detected in library-patterns-php. Triggering #434
            
            Sep 25, 2017 9:55:49 AM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run
            

            Since the web hook is organization-wide, I don't even know how does it understand that the repository is a global workflow library. I have gone through workspaces to delete the ...@libs folders to no avail.

            Show
            giorgiosironi Giorgio Sironi added a comment - The log says: Sep 25, 2017 9:55:48 AM INFO org.jenkinsci.plugins.github_branch_source.PushGHEventSubscriber onEvent Received PUSH for https: //github.com/elifesciences/elife-jenkins-workflow-libs from 192.30.252.40 ⇒ http://ci--alfred.elifesciences.org:8080/github-webhook/ Sep 25, 2017 9:55:48 AM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run SCM changes detected in library-api-validator-php. Triggering #22 Sep 25, 2017 9:55:49 AM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run SCM changes detected in library-patterns-php. Triggering #434 Sep 25, 2017 9:55:49 AM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run Since the web hook is organization-wide, I don't even know how does it understand that the repository is a global workflow library. I have gone through workspaces to delete the ...@libs folders to no avail.
            Hide
            giorgiosironi Giorgio Sironi added a comment -

            Deleted all workspaces too, no change. The log of a triggered build continues to say:

             > git ls-remote -h git@github.com:elifesciences/elife-jenkins-workflow-libs.git # timeout=10
            Found 1 remote heads on git@github.com:elifesciences/elife-jenkins-workflow-libs.git
            [poll] Latest remote head revision on refs/heads/master is: 9f5110ead60321557cdc0e5394ec299d70983938
            Using strategy: Default
            [poll] Last Built Revision: Revision 966f0ebb926fbe6e2db5d2de68a86aec4ff86676 (refs/remotes/origin/develop)
            

            and I don't know where it's getting that repository URL from.

            Show
            giorgiosironi Giorgio Sironi added a comment - Deleted all workspaces too, no change. The log of a triggered build continues to say: > git ls-remote -h git@github.com:elifesciences/elife-jenkins-workflow-libs.git # timeout=10 Found 1 remote heads on git@github.com:elifesciences/elife-jenkins-workflow-libs.git [poll] Latest remote head revision on refs/heads/master is: 9f5110ead60321557cdc0e5394ec299d70983938 Using strategy: Default [poll] Last Built Revision: Revision 966f0ebb926fbe6e2db5d2de68a86aec4ff86676 (refs/remotes/origin/develop) and I don't know where it's getting that repository URL from.
            Hide
            giorgiosironi Giorgio Sironi added a comment -

            I have found these references:

            elife@ci--alfred:/var/lib/jenkins$ grep -r elife-jenkins-workflow-libs jobs/test-journal/builds/1056/build.xml                            
                    <string>git@github.com:elifesciences/elife-jenkins-workflow-libs.git</string>
                        <url>git@github.com:elifesciences/elife-jenkins-workflow-libs.git</url>
            

            which are on a previous build rather than on the jobs definition, it is possible they are used to determine whether to trigger the job again. Will try to remove it on a test job.

            Show
            giorgiosironi Giorgio Sironi added a comment - I have found these references: elife@ci--alfred:/ var /lib/jenkins$ grep -r elife-jenkins-workflow-libs jobs/test-journal/builds/1056/build.xml <string>git@github.com:elifesciences/elife-jenkins-workflow-libs.git</string> <url>git@github.com:elifesciences/elife-jenkins-workflow-libs.git</url> which are on a previous build rather than on the jobs definition, it is possible they are used to determine whether to trigger the job again. Will try to remove it on a test job.
            Hide
            giorgiosironi Giorgio Sironi added a comment - - edited

            To delete these references, I found out you have to run at least 1 (successful?) build for each job without the global pipeline library being defined in Global Settings. This seems to remove from the jobs/ metadata the reference to the external repository, so that when the Github hook triggers it is ignored. I am now running all builds once to clear all references before I am able to make changes to the library and push it to /workflowLibs.git.

            Show
            giorgiosironi Giorgio Sironi added a comment - - edited To delete these references, I found out you have to run at least 1 (successful?) build for each job without the global pipeline library being defined in Global Settings. This seems to remove from the jobs/ metadata the reference to the external repository, so that when the Github hook triggers it is ignored. I am now running all builds once to clear all references before I am able to make changes to the library and push it to /workflowLibs.git.
            Hide
            dolphy David Crouch added a comment -

            Confirmed that poll SCM jobs are still triggered by every change to a shared library. What's the way forward on this? There are several duplicate JIRAs which describe this issue, but I don't see any open, active ones. Does this one need to be reopened, or does a new Major/Critical Bug need to be written?

            Show
            dolphy David Crouch added a comment - Confirmed that poll SCM jobs are still triggered by every change to a shared library. What's the way forward on this? There are several duplicate JIRAs which describe this issue, but I don't see any open, active ones. Does this one need to be reopened, or does a new Major/Critical Bug need to be written?
            Hide
            michaelneale Michael Neale added a comment -

            David Crouch possibly a new one (with repro details) cc Andrew Bayer

            Show
            michaelneale Michael Neale added a comment - David Crouch possibly a new one (with repro details) cc Andrew Bayer
            Hide
            allan_burdajewicz Allan BURDAJEWICZ added a comment -

            As far as I can tell, the fix works. I noticed that existing jobs will need to be built at least once for this to take effect. This will "fix" the metadata recorded the SCM used. So when you upgrade to that version, the first next change to the shared library code might still triggers many jobs. Next changes will not.

            Show
            allan_burdajewicz Allan BURDAJEWICZ added a comment - As far as I can tell, the fix works. I noticed that existing jobs will need to be built at least once for this to take effect. This will "fix" the metadata recorded the SCM used. So when you upgrade to that version, the first next change to the shared library code might still triggers many jobs. Next changes will not.
            Hide
            michaelneale Michael Neale added a comment -
            Show
            michaelneale Michael Neale added a comment - ah thanks Allan BURDAJEWICZ !   cc David Crouch
            Hide
            dolphy David Crouch added a comment -

            Allan BURDAJEWICZ Michael Neale

            You're right, and I'm wrong! I had based my comment solely off of seeing the jobs kick again, but never verified that they will continually kick. After the first time, they no longer trigger on a shared library commit.

             

            Sorry for the hasty assumption, and thanks!

            Show
            dolphy David Crouch added a comment - Allan BURDAJEWICZ Michael Neale You're right, and I'm wrong! I had based my comment solely off of seeing the jobs kick again, but never verified that they will continually kick. After the first time, they no longer trigger on a shared library commit.   Sorry for the hasty assumption, and thanks!
            Hide
            mnsa Matthias Nodeland added a comment -

            The global configuration option only for works for implicitly loaded libraries. If I include a library with the library step, I have to specify

            library identifier: "MyLibrary", changelog: false

            I would expect the default behavior

            library "MyLibrary"
            

            to be controlled by the global LibraryConfiguration option. As it is, I have to edit a large number of Jenkinsfiles to enable the new feature.

            It seems like my expected behavior might have been intended originally. In the implementation in LibraryStep.java

            public static class Execution extends AbstractSynchronousNonBlockingStepExecution<LoadedClasses> {
                @Override protected LoadedClasses run() throws Exception {
                    // ...
                    Boolean changelog = step.getChangelog(); // returns default value LibraryStep.changelog = true, if not provided
                    // ...
                    changelog = cfg.defaultedChangelogs(changelog); // returns changelog if != null, else global configuration option
                    // ...
                }
            }
            

            a function is called that loads the global configuration value if the argument is null, but due to the default initialization, this statement only has effect if changelog = null is specified explicitly. However, a comment for LibraryStep.setChangelog states: default to including changes of the library in job recent changes.

            Is this working as intended or should I open a PR for this?

             

            Show
            mnsa Matthias Nodeland added a comment - The global configuration option only for works for implicitly loaded libraries. If I include a library with the library step, I have to specify library identifier: "MyLibrary" , changelog: false I would expect the default behavior library "MyLibrary" to be controlled by the global LibraryConfiguration option. As it is, I have to edit a large number of Jenkinsfiles to enable the new feature. It seems like my expected behavior might have been intended originally. In the implementation in LibraryStep.java public static class Execution extends AbstractSynchronousNonBlockingStepExecution<LoadedClasses> { @Override protected LoadedClasses run() throws Exception { // ... Boolean changelog = step.getChangelog(); // returns default value LibraryStep.changelog = true , if not provided // ... changelog = cfg.defaultedChangelogs(changelog); // returns changelog if != null , else global configuration option // ... } } a function is called that loads the global configuration value if the argument is null, but due to the default initialization, this statement only has effect if changelog = null is specified explicitly. However, a comment for LibraryStep.setChangelog states: default to including changes of the library in job recent changes . Is this working as intended or should I open a PR for this?  
            Hide
            michaelneale Michael Neale added a comment -

            Matthias Nodeland do you mind opening that as a new feature JIRA for this and a PR that goes with it (if you want to) - and could be discussed there (or at least the ticket, where can be discussed). Changing default usually needs discussion but it seems reasonable in this case that the default is probably not right. 

            Show
            michaelneale Michael Neale added a comment - Matthias Nodeland  do you mind opening that as a new feature JIRA for this and a PR that goes with it (if you want to) - and could be discussed there (or at least the ticket, where can be discussed). Changing default usually needs discussion but it seems reasonable in this case that the default is probably not right. 
            Hide
            mcrooney mcrooney added a comment -

            With workflow-cps-global-lib 2.9 still seeing that jobs are re-built multiple times, with the option unchecked to include library changes in the config UI. I would agree that changing the default makes sense in this case because it is very unintuitive and can cause pretty large issues to your cluster throughput and stability, while if you wanted it and it gets turned off, you can easily turn it on.

            Show
            mcrooney mcrooney added a comment - With workflow-cps-global-lib 2.9 still seeing that jobs are re-built multiple times, with the option unchecked to include library changes in the config UI. I would agree that changing the default makes sense in this case because it is very unintuitive and can cause pretty large issues to your cluster throughput and stability, while if you wanted it and it gets turned off, you can easily turn it on.
            Hide
            mcrooney mcrooney added a comment - - edited

            From JENKINS-41497, "The global configuration option only for works for implicitly loaded libraries." For non-implicit libraries, is it expected that we have to set
            changelog: false
            ..in every Jenkinsfile? If so, then the global setting should be a child checkbox under the implicit checkbox.

            Show
            mcrooney mcrooney added a comment - - edited From JENKINS-41497 , "The global configuration option only for works for implicitly loaded libraries." For non-implicit libraries, is it expected that we have to set changelog: false ..in every Jenkinsfile? If so, then the global setting should be a child checkbox under the implicit checkbox.
            Hide
            mnsa Matthias Nodeland added a comment -

            I opened issue JENKINS-48487 for this specific problem. It is actually only a problem of the the library step, using the '@Library' decorator, it works fine. For the library step you would have to specify changelog: false in every Jenkinsfile (and change it to true, if you should change your mind later on). Since it works for the decorator syntax, I don't think it should be a child checkbox of the implicit checkbox. I filed it as a bug because the behavior is different between the decorator and the step.

            Show
            mnsa Matthias Nodeland added a comment - I opened issue JENKINS-48487 for this specific problem. It is actually only a problem of the the library step, using the '@Library' decorator, it works fine. For the library step you would have to specify changelog: false in every Jenkinsfile (and change it to true, if you should change your mind later on). Since it works for the decorator syntax, I don't think it should be a child checkbox of the implicit checkbox. I filed it as a bug because the behavior is different between the decorator and the step.
            abayer Andrew Bayer made changes -
            Link This issue is duplicated by JENKINS-42320 [ JENKINS-42320 ]
            abayer Andrew Bayer made changes -
            Link This issue is duplicated by JENKINS-42067 [ JENKINS-42067 ]
            Hide
            doronshai Doron Shai added a comment - - edited

            I see this issue once again with Jenkins 2.107.2, on 2.107.1 it is working just fine

            Show
            doronshai Doron Shai added a comment - - edited I see this issue once again with Jenkins 2.107.2 , on 2.107.1 it is working just fine
            doronshai Doron Shai made changes -
            Resolution Fixed [ 1 ]
            Status Resolved [ 5 ] Reopened [ 4 ]
            doronshai Doron Shai made changes -
            Comment [ I see this issue again on plugin 2.9 and Jenkins 2.107.2.

            Does someone knows if this can be solved somehow?  ]
            Hide
            eocampos Eliseo Ocampos added a comment -

            This bug is still present on version 2.116. I found that the workaround mentioned here is the way to go for the moment. Thanks Allan BURDAJEWICZ!

            Show
            eocampos Eliseo Ocampos added a comment - This bug is still present on version 2.116. I found that the workaround mentioned here is the way to go for the moment. Thanks Allan BURDAJEWICZ !
            Hide
            lakeend Petri Järvenpää added a comment -

            Hello and sorry for being bit thick, but where is this "global configuration" located and changed?

            I am trying to have implicitly loaded library for certain folder, but I cant figure how to stop the builds triggering continuously from shared lib commits.

            Show
            lakeend Petri Järvenpää added a comment - Hello and sorry for being bit thick, but where is this "global configuration" located and changed? I am trying to have implicitly loaded library for certain folder, but I cant figure how to stop the builds triggering continuously from shared lib commits.
            Hide
            eocampos Eliseo Ocampos added a comment -

            Go to Manage Jenkins -> Configure System and there find the section "Global Pipeline Libraries"

            Show
            eocampos Eliseo Ocampos added a comment - Go to Manage Jenkins -> Configure System and there find the section "Global Pipeline Libraries"
            Hide
            lakeend Petri Järvenpää added a comment -

            "Dont trigger build a build on commit notifications" this setting for the library should stop the triggering?

            This is what I already tested, and no matter if I let jobs trigger and build, they will trigger second, third and fourth time as well from commits to shared library.

            Show
            lakeend Petri Järvenpää added a comment - "Dont trigger build a build on commit notifications" this setting for the library should stop the triggering? This is what I already tested, and no matter if I let jobs trigger and build, they will trigger second, third and fourth time as well from commits to shared library.
            Hide
            eocampos Eliseo Ocampos added a comment -

            Petri Järvenpää please pay close attention to this comment and this other comment above.

            Show
            eocampos Eliseo Ocampos added a comment - Petri Järvenpää please pay close attention to this comment and this other comment above.
            Hide
            shakor Shahin Kordasti added a comment -

            I am confused as to what the "fix" is. I enabled the setting "Dont trigger build a build on commit notifications" for all our Jenkins jobs and they still get triggered whenever the global library SCM has a commit and this keeps happening after the first time. Do I need to remove the global library and re-add it again for this to work?

            Show
            shakor Shahin Kordasti added a comment - I am confused as to what the "fix" is. I enabled the setting "Dont trigger build a build on commit notifications" for all our Jenkins jobs and they still get triggered whenever the global library SCM has a commit and this keeps happening after the first time. Do I need to remove the global library and re-add it again for this to work?
            unlikelyzero John Hill made changes -
            Comment [ This is a pretty significant issue. We've removed our Global Pipeline Library config, restarted jenkins and it's still triggering events ]
            Hide
            whatsdevops Angelo Loria added a comment -

            Seeing this exact issue on Jenkins 2.150.1, Build Pipeline 1.5.8. Have a global library set up in one repo/configured under Global Configuration, and a multibranch job that polls a different repo for changes to specific branches. Upon committing anything to any file in the global library repo, the multibranch job is triggered. I've read every thread I can find on the issue but cannot figure out how to not have jobs triggered when committing changes to the global library repo. It's basically unusable at this point, as I can't continue to trigger all jobs over and over. 

            Show
            whatsdevops Angelo Loria added a comment - Seeing this exact issue on Jenkins 2.150.1, Build Pipeline 1.5.8. Have a global library set up in one repo/configured under Global Configuration, and a multibranch job that polls a different repo for changes to specific branches. Upon committing anything to any file in the global library repo, the multibranch job is triggered. I've read every thread I can find on the issue but cannot figure out how to not have jobs triggered when committing changes to the global library repo. It's basically unusable at this point, as I can't continue to trigger all jobs over and over. 
            Hide
            michaelneale Michael Neale added a comment -

            Angelo Loria not sure if that is related to this issue: but I think that may be possible by using "when" and looking at the cause/trigger. So when global libraries are updated, the job will run, but terminate early (green) as it can skip the rest of the build. Kind of a work around. 

            Show
            michaelneale Michael Neale added a comment - Angelo Loria not sure if that is related to this issue: but I think that may be possible by using "when" and looking at the cause/trigger. So when global libraries are updated, the job will run, but terminate early (green) as it can skip the rest of the build. Kind of a work around. 
            Hide
            swtch3k Cenk Tosun added a comment - - edited

            valide workaround for me was to add an additional behavior at the pipeline definition for the jenkinsfile or at the shared library repo definition, where to use ignore certain path with an inclusion of "ignoreRepo". no more additional builds. hope that bug get fixed or the option will disabled...

            Show
            swtch3k Cenk Tosun added a comment - - edited valide workaround for me was to add an additional behavior at the pipeline definition for the jenkinsfile or at the shared library repo definition, where to use ignore certain path with an inclusion of "ignoreRepo". no more additional builds. hope that bug get fixed or the option will disabled...
            Hide
            wbrode William Brode added a comment -

            Cenk Tosun another solution to your issue is to use "lightweight" checkout when defining the Jenkinsfile SCM.  I think it was intended behavior to allow polling on changes to the jenkinsfile scm - but lightweight will disable that (if you are using an SCM that supports it).

            Show
            wbrode William Brode added a comment - Cenk Tosun another solution to your issue is to use "lightweight" checkout when defining the Jenkinsfile SCM.  I think it was intended behavior to allow polling on changes to the jenkinsfile scm - but lightweight will disable that (if you are using an SCM that supports it).
            Hide
            doronshai Doron Shai added a comment -

            William Brode I do not see that the lightweight checkout actually solve this issue.

            I do not understand why and how this issue is still open. this is a MAJOR problem with Jenkins....

            Show
            doronshai Doron Shai added a comment - William Brode I do not see that the lightweight checkout actually solve this issue. I do not understand why and how this issue is still open. this is a MAJOR problem with Jenkins....
            Hide
            swtch3k Cenk Tosun added a comment -

            William Brode as Doron Shai has already said, this unfortunately does not help to solve the problem...

            Show
            swtch3k Cenk Tosun added a comment - William Brode  as Doron Shai has already said, this unfortunately does not help to solve the problem...
            hjink94 Harrison Kim made changes -
            Attachment image-2019-04-08-13-45-36-893.png [ 46656 ]
            Hide
            delphym Daniel Mladek added a comment - - edited

            Just would like to confirm, that I'm observing this undesirable behaviour too on Jenkins v2.161 and all the relevant pipeline plugins (up-to-date). All source code including Jenkinsfiles are in SVN, shared libs in Git/Bitbucket.

             

            Nearly after month of migrating legacy builds to new pipelines and going through the loop of learning and refactoring my Jenkinsfiles to make them leaner and started using a [shared library|https://jenkins.io/doc/book/pipeline/shared-libraries/], each time I do a change in a shared lib, it triggers all the pipeline jobs. It's really nightmare.

             

            I understand that there might be a need for this behaviour if a sort of part of a build script has been changed, to validate the rest of the code it is still buildable, OTOH if you're in a building stage of setting up your build system, this is incredible annoying. (For the same reason, I'm almost thinking to setup external repo just for the Jenkinsfiles to avoid unnecessary triggers of build jobs when modified Jenkinsfile in SCM. Having it inside the job configuration or using old fashioned freestyle jobs, exactly achieve this, but without additional benefits of using pipelines.)

             

            So ideally, this behaviour would be configurable (like as it is now for Include @Library changes in job recent changes.)

             

             

             

             

             

            Show
            delphym Daniel Mladek added a comment - - edited Just would like to confirm, that I'm observing this undesirable behaviour too on Jenkins v2.161 and all the relevant pipeline plugins (up-to-date). All source code including Jenkinsfiles are in SVN, shared libs in Git/Bitbucket.   Nearly after month of migrating legacy builds to new pipelines and going through the loop of learning and refactoring my Jenkinsfiles to make them leaner and started using a [shared library| https://jenkins.io/doc/book/pipeline/shared-libraries/ ], each time I do a change in a shared lib, it triggers all the pipeline jobs. It's really nightmare.   I understand that there might be a need for this behaviour if a sort of part of a build script has been changed, to validate the rest of the code it is still buildable, OTOH if you're in a building stage of setting up your build system, this is incredible annoying. (For the same reason, I'm almost thinking to setup external repo just for the Jenkinsfiles to avoid unnecessary triggers of build jobs when modified Jenkinsfile in SCM. Having it inside the job configuration or using old fashioned freestyle jobs, exactly achieve this, but without additional benefits of using pipelines.)   So ideally, this behaviour would be configurable (like as it is now for Include @Library changes in job recent changes .)          
            Hide
            doronshai Doron Shai added a comment -

            MOVING TO GITLAB CI - THANKS FOR THE AMAZING SUPPORT

            Show
            doronshai Doron Shai added a comment - MOVING TO GITLAB CI - THANKS FOR THE AMAZING SUPPORT
            Hide
            jackiexiao Jackie Xiao added a comment -

            No matter how many new features being added to Jenkins, lack of support and issue fixing for such basic issues will eventually drove users away.

            Goodbye Jenkins

            Show
            jackiexiao Jackie Xiao added a comment - No matter how many new features being added to Jenkins, lack of support and issue fixing for such basic issues will eventually drove users away. Goodbye Jenkins
            Hide
            gordin Christoph Vogtländer added a comment -

            I'm also affected by this issue running Jenkins ver. 2.176.2 with "Pipeline: Shared Groovy Libraries" version 2.14

            I just recently switched to shared libs from internal workflowLibs.git. I added a global shared library using Subversion defined with "Load implicitly: true", "Allow default version to be overridden: true" and "Include @Library changes in job recent changes: true".
            After the global library was used in the build for the first time, subsequent builds are now triggered whenever something changes in the global library.

            I was not able to resolve this by simply deactivating "Include @Library changes in job recent changes". Still, builds were triggered for every commit in the shared lib. Only after a Jenkins restart ("Include @Library changes in job recent changes" is still false) the job is no longer triggered.

            Why are polling and changelog bound to each other in shared libraries in the first place? In the "checkout" step these are two different options (or am I misreading something?).

            In my case, the project that is triggered does NOT poll the SCM but is configured to only react on push notifications. So, this might be a different issue? Should I open a new bug report?

            Show
            gordin Christoph Vogtländer added a comment - I'm also affected by this issue running Jenkins ver. 2.176.2 with "Pipeline: Shared Groovy Libraries" version 2.14 I just recently switched to shared libs from internal workflowLibs.git. I added a global shared library using Subversion defined with "Load implicitly: true", "Allow default version to be overridden: true" and "Include @Library changes in job recent changes: true". After the global library was used in the build for the first time, subsequent builds are now triggered whenever something changes in the global library. I was not able to resolve this by simply deactivating "Include @Library changes in job recent changes". Still, builds were triggered for every commit in the shared lib. Only after a Jenkins restart ("Include @Library changes in job recent changes" is still false) the job is no longer triggered. Why are polling and changelog bound to each other in shared libraries in the first place? In the "checkout" step these are two different options (or am I misreading something?). In my case, the project that is triggered does NOT poll the SCM but is configured to only react on push notifications. So, this might be a different issue? Should I open a new bug report?
            Hide
            yrsurya suryatej yaramada added a comment -

            Even we are facing same issue , opened a ticket - https://issues.jenkins-ci.org/browse/JENKINS-52816

            Show
            yrsurya suryatej yaramada added a comment - Even we are facing same issue , opened a ticket -  https://issues.jenkins-ci.org/browse/JENKINS-52816
            abayer Andrew Bayer made changes -
            Assignee Andrew Bayer [ abayer ]
            Hide
            allan_burdajewicz Allan BURDAJEWICZ added a comment - - edited

            Is this still happening even after jobs are rebuilt at least once (see my comment above) ?

            If still hitting the problem, https://issues.jenkins-ci.org/browse/JENKINS-61415 (workflow-job:2.37) may help as it does not require a rebuild anymore. (nor a restart of Jenkins)

            Show
            allan_burdajewicz Allan BURDAJEWICZ added a comment - - edited Is this still happening even after jobs are rebuilt at least once (see my comment above) ? If still hitting the problem, https://issues.jenkins-ci.org/browse/JENKINS-61415 (workflow-job:2.37) may help as it does not require a rebuild anymore. (nor a restart of Jenkins)
            Hide
            dordor dor s added a comment - - edited

            in version 2.9 of workflow-cps-global-lib-plugin you added "allow excluding shared libraries from changelogs" as part of this commit https://github.com/jenkinsci/workflow-cps-global-lib-plugin/commit/64df779249ac69981a17f46e2698791fe95fa06d , but you never mentioned of how to do it in jenkins declarative pipeline?

            well, this is the syntax of loading shared library into the pipeline

            libraries {
              lib('some-library')
            }
            

            but I couldn't find any way to add the annotation of changelog=false in the jenkins declarative pipeline syntax. any idea how?

             and I couldn't find anything here: https://www.jenkins.io/doc/book/pipeline/shared-libraries/

            some examples can be found here for jenkins scripted pipeline: https://stackoverflow.com/questions/46735908/jenkins-pipeline-global-pipeline-library-updates-show-up-in-jobs-list-of-rece
             

            Devin Nusbaum Tim Jacomb any chance that you can help here?

            Show
            dordor dor s added a comment - - edited in version 2.9 of  workflow-cps-global-lib-plugin  you added "allow excluding shared libraries from changelogs" as part of this commit https://github.com/jenkinsci/workflow-cps-global-lib-plugin/commit/64df779249ac69981a17f46e2698791fe95fa06d  , but you never mentioned of how to do it in jenkins declarative pipeline? well, this is the syntax of loading shared library into the pipeline libraries {   lib( 'some-library' ) } but I couldn't find any way to add the annotation of changelog=false in the jenkins declarative pipeline syntax. any idea how?  and I couldn't find anything here:  https://www.jenkins.io/doc/book/pipeline/shared-libraries/ some examples can be found here for jenkins scripted pipeline: https://stackoverflow.com/questions/46735908/jenkins-pipeline-global-pipeline-library-updates-show-up-in-jobs-list-of-rece   Devin Nusbaum Tim Jacomb any chance that you can help here?
            Hide
            timja Tim Jacomb added a comment -

            You can configure it in the global config (which is what we do) no idea about in the pipeline

            Show
            timja Tim Jacomb added a comment - You can configure it in the global config (which is what we do) no idea about in the pipeline
            Hide
            dordor dor s added a comment -

            Hi Tim Jacomb , thank you for your reply. I forgot to mention it but I already tried to configure it in the global config but it's not affect on my declarative pipeline which located in github organization. So in case that there are new commits in the shared library git repo I see those changes as part of the pipeline which can be confusing or affect the change set array that I use to detect what files has been changed 

            Show
            dordor dor s added a comment - Hi Tim Jacomb , thank you for your reply. I forgot to mention it but I already tried to configure it in the global config but it's not affect on my declarative pipeline which located in github organization. So in case that there are new commits in the shared library git repo I see those changes as part of the pipeline which can be confusing or affect the change set array that I use to detect what files has been changed 
            Hide
            ahammar Anders Hammar added a comment - - edited

            dor s We have the same issue with declarative pipelines where I can't figure out how to turn off changelog. The global configuration doesn't seem to work. It's turned off but changelogs is still included for shared libraries.
            We're using v2.17 of workflow-cps-global-lib-plugin.

            Show
            ahammar Anders Hammar added a comment - - edited dor s We have the same issue with declarative pipelines where I can't figure out how to turn off changelog. The global configuration doesn't seem to work. It's turned off but changelogs is still included for shared libraries. We're using v2.17 of workflow-cps-global-lib-plugin.
            Hide
            richmond Rich added a comment -

            I am facing similar issue. When I unselect 'Include @Library changes in job recent changes' and restart Jenkins, changelog does not show changes made to shared library but any change in shared library is still triggering the build.

            Show
            richmond Rich added a comment - I am facing similar issue. When I unselect 'Include @Library changes in job recent changes' and restart Jenkins, changelog does not show changes made to shared library but any change in shared library is still triggering the build.
            Hide
            dordor dor s added a comment -

            I still can't find a way to disable the change log for the shared library changes (which is not relevant for me) in declerative pipelines

             

            is seems that the default is true

            apperantly thhis is the relevant code:

             

                private Boolean changelog = true;
                ...
                // default to including changes of the library in job recent changes
                @DataBoundSetter public void setChangelog(Boolean changelog) {
                    this.changelog = changelog;
                }

             

            https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/libs/LibraryStep.java#L111-L114

             

            and this

             

                static void doRetrieve(String name, boolean changelog, @Nonnull SCM scm, FilePath target, Run<?, ?> run, TaskListener listener) throws Exception {
                    ...
                    delegate.setChangelog(changelog);

             

            https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/libs/SCMSourceRetriever.java#L139

             

            more info here: https://issues.jenkins.io/browse/JENKINS-41497?focusedCommentId=394574&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-394574

            Devin Nusbaum any chance that you can have a look?

             

            Show
            dordor dor s added a comment - I still can't find a way to disable the change log for the shared library changes (which is not relevant for me) in declerative  pipelines   is seems that the default is true apperantly thhis is the relevant code:   private Boolean changelog = true ; ... // default to including changes of the library in job recent changes @DataBoundSetter public void setChangelog( Boolean changelog) { this .changelog = changelog; }   https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/libs/LibraryStep.java#L111-L114   and this   static void doRetrieve( String name, boolean changelog, @Nonnull SCM scm, FilePath target, Run<?, ?> run, TaskListener listener) throws Exception { ... delegate.setChangelog(changelog);   https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/master/src/main/java/org/jenkinsci/plugins/workflow/libs/SCMSourceRetriever.java#L139   more info here:  https://issues.jenkins.io/browse/JENKINS-41497?focusedCommentId=394574&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-394574 Devin Nusbaum  any chance that you can have a look?  
            Hide
            falloutphil Philip Beadling added a comment - - edited

            I have the same issue in declarative pipeline with fresh install of 2.263.3.

            With a multibranch declarative pipeline (Bitbucket Source Plugin) my job changelog contains very confusingly Git revision changes for my global pipeline library (loaded via git scm), but lists no changes from Bitbucket.

            Following advice I have unticked:

              Include @Library changes in job recent changes

             

            And also tried changing my Jenkinsfile annotation to:

            @Library(value='my-lib', changelog=false)

             

            But neither prevents the details appearing on clicking on a the build page for a completed job.

            I get GIt revision details on every job run from the last library update - irrespective if the library has been updated from the last job or not.

             

            Also I note that the declarative way of loading a libraries - using the libraries directive has no provision for setting the changelog parameter, as evidenced by this note in the source code:

            https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/15978cd172a8d34e2cd6bf8a6bfeaa5514d44dad/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/ModelParser.groovy#L743

             

            As Bitbucket doesn't use polling I don't have the problem of my jenkins library kicking off builds.

             

            FWIW - I think both kicking off builds and even reporting library changes should be turned off by default, it's taken me a fair bit of head-scratching to even understand where the commit ids came from - I had just assumed they were from my project source control for months.

             

            Show
            falloutphil Philip Beadling added a comment - - edited I have the same issue in declarative pipeline with fresh install of 2.263.3. With a multibranch declarative pipeline (Bitbucket Source Plugin) my job changelog contains very confusingly Git revision changes for my global pipeline library (loaded via git scm), but lists no changes from Bitbucket. Following advice I have unticked:   Include @Library changes in job recent changes   And also tried changing my Jenkinsfile annotation to: @Library(value='my-lib', changelog=false)   But neither prevents the details appearing on clicking on a the build page for a completed job. I get GIt revision details on every job run from the last library update - irrespective if the library has been updated from the last job or not.   Also I note that the declarative way of loading a libraries - using the libraries directive has no provision for setting the changelog parameter, as evidenced by this note in the source code: https://github.com/jenkinsci/pipeline-model-definition-plugin/blob/15978cd172a8d34e2cd6bf8a6bfeaa5514d44dad/pipeline-model-definition/src/main/groovy/org/jenkinsci/plugins/pipeline/modeldefinition/parser/ModelParser.groovy#L743   As Bitbucket doesn't use polling I don't have the problem of my jenkins library kicking off builds.   FWIW - I think both kicking off builds and even reporting library changes should be turned off by default, it's taken me a fair bit of head-scratching to even understand where the commit ids came from - I had just assumed they were from my project source control for months.  
            Hide
            gartoty Tyler added a comment -

            My company recently moved from GitLab to GitHub and this exact issue is happening now where it triggers all consumers of our shared library. The problem is that we have such a large consumer base that it is literally triggering 100s of pipeline jobs. We have tried everything like others have mentioned with adding additional behaviours to the lib pipeline definition: "Don't trigger a build on commit notifications", "Polling ignored from certain users", "dot not check Include @Library changes in job recent changes" and nothing works. We are upgrading our Jenkins and plugins this weekend which is a little dated but I don't think it will resolve anything based on the open issues. I'm hoping some sort of solution is found because this is not ideal and we are having to do some creative things to get by at the moment 

            Show
            gartoty Tyler added a comment - My company recently moved from GitLab to GitHub and this exact issue is happening now where it triggers all consumers of our shared library. The problem is that we have such a large consumer base that it is literally triggering 100s of pipeline jobs. We have tried everything like others have mentioned with adding additional behaviours to the lib pipeline definition: "Don't trigger a build on commit notifications", "Polling ignored from certain users", "dot not check Include @Library changes in job recent changes" and nothing works. We are upgrading our Jenkins and plugins this weekend which is a little dated but I don't think it will resolve anything based on the open issues. I'm hoping some sort of solution is found because this is not ideal and we are having to do some creative things to get by at the moment 
            Hide
            richmond Rich added a comment -

            Tyler, I had similar issues. Try this:

            1. Did you restart Jenkins after you made changes to the Shared Library definition ? If not, do it now.
            2. All your pipeline jobs which are getting triggered, will need reset of 'GitHub hook trigger for GITScm polling' i.e. go to each job, disable the trigger, save job, enable the trigger, save job.

            That should fix. You may try #2 on a few jobs first before doing it for all 100s of them.

            Show
            richmond Rich added a comment - Tyler , I had similar issues. Try this: Did you restart Jenkins after you made changes to the Shared Library definition ? If not, do it now. All your pipeline jobs which are getting triggered, will need reset of 'GitHub hook trigger for GITScm polling' i.e. go to each job, disable the trigger, save job, enable the trigger, save job. That should fix. You may try #2 on a few jobs first before doing it for all 100s of them.
            mcascone Max Cascone made changes -
            Link This issue relates to JENKINS-65649 [ JENKINS-65649 ]

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              rtyler R. Tyler Croy
              Votes:
              85 Vote for this issue
              Watchers:
              94 Start watching this issue

                Dates

                Created:
                Updated: