Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40606

Add ability to perform actions on branch removal

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      This is similar request to https://issues.jenkins-ci.org/browse/JENKINS-37907, but for workflow-multibranch-plugin.

      Rationale:
      Team is using short term feature branches and for each of them SonarQube analysis is executed. SonarQube treats branch builds as separate SonarQube projects. I would like to have a possibility to hook in when branch is being deleted and call SonarQube API to delete this branch-specific project.

        Attachments

          Issue Links

            Activity

            dziesio Grzegorz Poznachowski created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Link This issue relates to JENKINS-37907 [ JENKINS-37907 ]
            Hide
            jglick Jesse Glick added a comment -

            Use ItemListener.onDeleted from a plugin. I suppose you are asking for some sort of “userspace” hook, like a shell script or something, which you could configure perhaps as a JobProperty. If so, this would better be moved to plugin-proposals.

            Seems like a straight-up duplicate of JENKINS-37907, which seems to have been closed improperly.

            Show
            jglick Jesse Glick added a comment - Use ItemListener.onDeleted from a plugin. I suppose you are asking for some sort of “userspace” hook, like a shell script or something, which you could configure perhaps as a JobProperty . If so, this would better be moved to plugin-proposals . Seems like a straight-up duplicate of JENKINS-37907 , which seems to have been closed improperly.
            Hide
            dziesio Grzegorz Poznachowski added a comment - - edited

            I assume that would require writing a separate Jenkins plugin for that (like here)? Or, could I use it in Jenkinsfile? And yes, I was asking if there is any posibility to achieve that (i.e. via script) beside coding a dedicated plugin.

            Show
            dziesio Grzegorz Poznachowski added a comment - - edited I assume that would require writing a separate Jenkins plugin for that (like here )? Or, could I use it in Jenkinsfile? And yes, I was asking if there is any posibility to achieve that (i.e. via script) beside coding a dedicated plugin.
            Hide
            jglick Jesse Glick added a comment -

            could I use it in Jenkinsfile?

            No, because this is outside a build context.

            Show
            jglick Jesse Glick added a comment - could I use it in Jenkinsfile? No, because this is outside a build context.
            jglick Jesse Glick made changes -
            Component/s plugin-proposals [ 15491 ]
            Component/s workflow-multibranch-plugin [ 21465 ]
            Labels multibranch
            Hide
            dziesio Grzegorz Poznachowski added a comment -

            Is something changed in the context of this ticket with 2.x plugins available?

            Show
            dziesio Grzegorz Poznachowski added a comment - Is something changed in the context of this ticket with 2.x plugins available?
            Hide
            jglick Jesse Glick added a comment -

            Not that I am aware of.

            Show
            jglick Jesse Glick added a comment - Not that I am aware of.
            Hide
            jgeluk Jacobus Geluk added a comment -

            Ideally, you would want to be able to run a special section in your Jenkinsfile that gets executed just before a job and its workspace gets deleted (only when it actually has one). Even though at that point the git-branch is already deleted and a fetch for the latest version would be impossible, it would allow cleanup scripts that still reside in the workspace from its last execution to be executed.

            Show
            jgeluk Jacobus Geluk added a comment - Ideally, you would want to be able to run a special section in your Jenkinsfile that gets executed just before a job and its workspace gets deleted (only when it actually has one). Even though at that point the git-branch is already deleted and a fetch for the latest version would be impossible, it would allow cleanup scripts that still reside in the workspace from its last execution to be executed.
            Hide
            stephenconnolly Stephen Connolly added a comment -

            Jacobus Geluk ah but where would you log that execution? How would you find out that it failed?

            I think this could be more easily handled by an extension plugin.

            You have the listener, a multibranch project property could add an action that shows the tidy-up logs and you could have some pipeline step that defines the tidy-up steps of a branch in the pipeline. Then the listener looks at the branch, sees that it has the tidy-up steps action, grabs them and allows the delete to continue...meanwhile it schedules an execution of the tidy-up that is attached to the multibranch project rather than the job.

            Now the above is not trivial... but it is all straighforward (i.e. I do not envision any major pitfalls along the way) and I think it would fit into the overall design for how all these things work quite well

            Show
            stephenconnolly Stephen Connolly added a comment - Jacobus Geluk ah but where would you log that execution? How would you find out that it failed? I think this could be more easily handled by an extension plugin. You have the listener, a multibranch project property could add an action that shows the tidy-up logs and you could have some pipeline step that defines the tidy-up steps of a branch in the pipeline. Then the listener looks at the branch, sees that it has the tidy-up steps action, grabs them and allows the delete to continue...meanwhile it schedules an execution of the tidy-up that is attached to the multibranch project rather than the job. Now the above is not trivial... but it is all straighforward (i.e. I do not envision any major pitfalls along the way) and I think it would fit into the overall design for how all these things work quite well
            Hide
            larsskj Lars Skjærlund added a comment -

            I'd like to vote for this as well.

            We're building and creating a Docker environment for each branch to run tests on the branch - and would like to tear down this environment properly when a branch is deleted.

            Show
            larsskj Lars Skjærlund added a comment - I'd like to vote for this as well. We're building and creating a Docker environment for each branch to run tests on the branch - and would like to tear down this environment properly when a branch is deleted.
            Hide
            mgreco2k Michael Greco added a comment - - edited

            I'd like to see this available from a GitHubOrg job ... when a Branch or PR is deleted I'd like to be able to run pipeline code on a different branch that then has instructions on removing infrastructure. Lets say all my GH repos in my GH organization have a special branch called "unprovision" which never goes away, and they all have a JenkinsFile called "JenkinsfileUnprovisionInfrastructure". So I set up GitHubOrg job to find all these repos in my organization and importantly I set the refspec to always checkout the "unprovision" branch. At this point the scan runs and discovery completes. Perfect. Now the idea is a person at some later point in Github does a BRANCH DELETE or PR CLOSE and in turn the GH webhook on that repo sends Jenkins this event AND the GitHubOrg job gets the REMOVED event that comes in. My job then executes by checking out the branch identified in the refspec ("unprovision" branch) and runs the Jenkinsfile identified (which is JenkinsfileUnprovisionInfrastructure). That pipeline code (JenkinsfileUnprovisionInfrastructure) which I've written then cleans up the infrastructure related to the GH BRANCH and/or PR that was closed. NOTE: I'm not trying to or even asking to check out something that is gone from GH ! The idea is when the development of something begins we provision infrastrcture from our pipelines ... SO WHEN ITS CLOSED WE NEED TO FREE UP THOSE RESOURCES and would like to do it from pipelines as well.

             

            Right now when I close a PR in GH the GH event related to that PR close comes into Jenkins and this message shows up in the "repository events" log of my GHOrg job :

             

            [Thu Nov 30 15:11:36 UTC 2017] Received Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒ http://jenkins:8080/github-webhook/ with timestamp Thu Nov 30 15:11:00 UTC 2017

            Examining myorg/pipeline-app

            Checking branches...

            Getting remote branch story/unprovision1...

               Checking branch story/unprovision1

                  ‘cicd/JenkinsfileUnprovisionAWS’ found

                Met criteria

            No changes detected: story/unprovision1 (still at 76c77601680005daeb4)

               1 branches were processed

                Checking pull-requests...

                Getting remote pull request #30...

                0 pull requests were processed

            Finished examining myorg/pipeline-app

            [Thu Nov 30 15:12:04 UTC 2017] Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒http://jenkins:8080/github-webhook/ with timestamp Thu Nov 30 15:11:00 UTC 2017 processed in 27 sec

             

            So making sense of those log messages : PR#30 is a pr that I closed in GH on a branch called story/unprovision1 on a repo in myorg called pipeline-app. There's obviously an event raised in the hook that gets shown in the log as a REMOVED event and then you clearly see "0 pull requests were processed" ...

            Show
            mgreco2k Michael Greco added a comment - - edited I'd like to see this available from a GitHubOrg job ... when a Branch or PR is deleted I'd like to be able to run pipeline code on a different branch that then has instructions on removing infrastructure. Lets say all my GH repos in my GH organization have a special branch called "unprovision" which never goes away, and they all have a JenkinsFile called "JenkinsfileUnprovisionInfrastructure". So I set up GitHubOrg job to find all these repos in my organization and importantly I set the refspec to always checkout the "unprovision" branch. At this point the scan runs and discovery completes. Perfect. Now the idea is a person at some later point in Github does a BRANCH DELETE or PR CLOSE and in turn the GH webhook on that repo sends Jenkins this event AND the GitHubOrg job gets the REMOVED event that comes in. My job then executes by checking out the branch identified in the refspec ("unprovision" branch) and runs the Jenkinsfile identified (which is JenkinsfileUnprovisionInfrastructure). That pipeline code (JenkinsfileUnprovisionInfrastructure) which I've written then cleans up the infrastructure related to the GH BRANCH and/or PR that was closed. NOTE: I'm not trying to or even asking to check out something that is gone from GH ! The idea is when the development of something begins we provision infrastrcture from our pipelines ... SO WHEN ITS CLOSED WE NEED TO FREE UP THOSE RESOURCES and would like to do it from pipelines as well.   Right now when I close a PR in GH the GH event related to that PR close comes into Jenkins and this message shows up in the "repository events" log of my GHOrg job :   [Thu Nov 30 15:11:36 UTC 2017] Received Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒ http://jenkins:8080/github-webhook / with timestamp Thu Nov 30 15:11:00 UTC 2017 Examining myorg/pipeline-app Checking branches... Getting remote branch story/unprovision1...    Checking branch story/unprovision1       ‘cicd/JenkinsfileUnprovisionAWS’ found     Met criteria No changes detected: story/unprovision1 (still at 76c77601680005daeb4)    1 branches were processed     Checking pull-requests...     Getting remote pull request #30...     0 pull requests were processed Finished examining myorg/pipeline-app [Thu Nov 30 15:12:04 UTC 2017] Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒ http://jenkins:8080/github-webhook / with timestamp Thu Nov 30 15:11:00 UTC 2017 processed in 27 sec   So making sense of those log messages : PR#30 is a pr that I closed in GH on a branch called story/unprovision1 on a repo in myorg called pipeline-app. There's obviously an event raised in the hook that gets shown in the log as a REMOVED event and then you clearly see "0 pull requests were processed" ...
            Hide
            integer Kanstantsin Shautsou added a comment -

            Multibranch can't handle such case because of it's design with spaghetti triggering decision logic from different places and tones of optimisations.

            Multibranch api is too locked and tied on triggering on hashes.
            SCMHeadObserver is limiting everything and hiding childObserver like i mentioned in https://groups.google.com/forum/#!topic/jenkinsci-dev/gEH1EdzMblM

            The other problem is that it impossible to have branch specific triggering settings, scmfiltering and etc extensions works only one side and only on hashes.

            Probably some other generic multi job should be designed instead (only categories, project factory, action storage (instead of every time updating transient), few bas jellys) and that's all needed for trigger developers.

            Show
            integer Kanstantsin Shautsou added a comment - Multibranch can't handle such case because of it's design with spaghetti triggering decision logic from different places and tones of optimisations. Multibranch api is too locked and tied on triggering on hashes. SCMHeadObserver is limiting everything and hiding childObserver like i mentioned in https://groups.google.com/forum/#!topic/jenkinsci-dev/gEH1EdzMblM The other problem is that it impossible to have branch specific triggering settings, scmfiltering and etc extensions works only one side and only on hashes. Probably some other generic multi job should be designed instead (only categories, project factory, action storage (instead of every time updating transient), few bas jellys) and that's all needed for trigger developers.
            Hide
            giorgiosironi Giorgio Sironi added a comment -

            I have an idea for a workaround for this that I want to try out: write a Groovy script that is executed periodically as part of a pipeline (through the Jenkins CLI to get the necessary permissions) and scans the deleted pull requests/branches that are in the grace period in which they have not been deleted yet from the Jenkins multiproject folder.

            If I can get that list, it would be easy to then idempotently run cleanup jobs. They won't run as soon as the pull request/branch is gone, but the frequency of this cleanup can be tuned.

            I use a similar approach to clean up old agents:

            https://github.com/elifesciences/elife-alfred-formula/blob/master/jenkinsfiles/ec2-plugin-nodes-retirement.groovy

            although here it's more difficult to understand the API to traverse and whether it's possible to list all of these.

            Show
            giorgiosironi Giorgio Sironi added a comment - I have an idea for a workaround for this that I want to try out: write a Groovy script that is executed periodically as part of a pipeline (through the Jenkins CLI to get the necessary permissions) and scans the deleted pull requests/branches that are in the grace period in which they have not been deleted yet from the Jenkins multiproject folder. If I can get that list, it would be easy to then idempotently run cleanup jobs. They won't run as soon as the pull request/branch is gone, but the frequency of this cleanup can be tuned. I use a similar approach to clean up old agents: https://github.com/elifesciences/elife-alfred-formula/blob/master/jenkinsfiles/ec2-plugin-nodes-retirement.groovy although here it's more difficult to understand the API to traverse and whether it's possible to list all of these.
            jglick Jesse Glick made changes -
            Link This issue relates to JENKINS-37220 [ JENKINS-37220 ]
            Hide
            jglick Jesse Glick added a comment -

            Something along the lines of JENKINS-37220 could be useful for this sort of specialized use case.

            Show
            jglick Jesse Glick added a comment - Something along the lines of JENKINS-37220 could be useful for this sort of specialized use case.
            Hide
            aytuncbeken Aytunc BEKEN added a comment -

            Long time passed but if anyone still requires this feature can use the plugin MultiBranch Action Triggers
            https://github.com/jenkinsci/multibranch-action-triggers-plugin

            Show
            aytuncbeken Aytunc BEKEN added a comment - Long time passed but if anyone still requires this feature can use the plugin MultiBranch Action Triggers https://github.com/jenkinsci/multibranch-action-triggers-plugin

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              dziesio Grzegorz Poznachowski
              Votes:
              21 Vote for this issue
              Watchers:
              26 Start watching this issue

                Dates

                Created:
                Updated: