Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-40606

Add ability to perform actions on branch removal

      This is similar request to https://issues.jenkins-ci.org/browse/JENKINS-37907, but for workflow-multibranch-plugin.

      Rationale:
      Team is using short term feature branches and for each of them SonarQube analysis is executed. SonarQube treats branch builds as separate SonarQube projects. I would like to have a possibility to hook in when branch is being deleted and call SonarQube API to delete this branch-specific project.

          [JENKINS-40606] Add ability to perform actions on branch removal

          Is something changed in the context of this ticket with 2.x plugins available?

          Grzegorz Poznachowski added a comment - Is something changed in the context of this ticket with 2.x plugins available?

          Jesse Glick added a comment -

          Not that I am aware of.

          Jesse Glick added a comment - Not that I am aware of.

          Jacobus Geluk added a comment -

          Ideally, you would want to be able to run a special section in your Jenkinsfile that gets executed just before a job and its workspace gets deleted (only when it actually has one). Even though at that point the git-branch is already deleted and a fetch for the latest version would be impossible, it would allow cleanup scripts that still reside in the workspace from its last execution to be executed.

          Jacobus Geluk added a comment - Ideally, you would want to be able to run a special section in your Jenkinsfile that gets executed just before a job and its workspace gets deleted (only when it actually has one). Even though at that point the git-branch is already deleted and a fetch for the latest version would be impossible, it would allow cleanup scripts that still reside in the workspace from its last execution to be executed.

          jgeluk ah but where would you log that execution? How would you find out that it failed?

          I think this could be more easily handled by an extension plugin.

          You have the listener, a multibranch project property could add an action that shows the tidy-up logs and you could have some pipeline step that defines the tidy-up steps of a branch in the pipeline. Then the listener looks at the branch, sees that it has the tidy-up steps action, grabs them and allows the delete to continue...meanwhile it schedules an execution of the tidy-up that is attached to the multibranch project rather than the job.

          Now the above is not trivial... but it is all straighforward (i.e. I do not envision any major pitfalls along the way) and I think it would fit into the overall design for how all these things work quite well

          Stephen Connolly added a comment - jgeluk ah but where would you log that execution? How would you find out that it failed? I think this could be more easily handled by an extension plugin. You have the listener, a multibranch project property could add an action that shows the tidy-up logs and you could have some pipeline step that defines the tidy-up steps of a branch in the pipeline. Then the listener looks at the branch, sees that it has the tidy-up steps action, grabs them and allows the delete to continue...meanwhile it schedules an execution of the tidy-up that is attached to the multibranch project rather than the job. Now the above is not trivial... but it is all straighforward (i.e. I do not envision any major pitfalls along the way) and I think it would fit into the overall design for how all these things work quite well

          I'd like to vote for this as well.

          We're building and creating a Docker environment for each branch to run tests on the branch - and would like to tear down this environment properly when a branch is deleted.

          Lars Skjærlund added a comment - I'd like to vote for this as well. We're building and creating a Docker environment for each branch to run tests on the branch - and would like to tear down this environment properly when a branch is deleted.

          Michael Greco added a comment - - edited

          I'd like to see this available from a GitHubOrg job ... when a Branch or PR is deleted I'd like to be able to run pipeline code on a different branch that then has instructions on removing infrastructure. Lets say all my GH repos in my GH organization have a special branch called "unprovision" which never goes away, and they all have a JenkinsFile called "JenkinsfileUnprovisionInfrastructure". So I set up GitHubOrg job to find all these repos in my organization and importantly I set the refspec to always checkout the "unprovision" branch. At this point the scan runs and discovery completes. Perfect. Now the idea is a person at some later point in Github does a BRANCH DELETE or PR CLOSE and in turn the GH webhook on that repo sends Jenkins this event AND the GitHubOrg job gets the REMOVED event that comes in. My job then executes by checking out the branch identified in the refspec ("unprovision" branch) and runs the Jenkinsfile identified (which is JenkinsfileUnprovisionInfrastructure). That pipeline code (JenkinsfileUnprovisionInfrastructure) which I've written then cleans up the infrastructure related to the GH BRANCH and/or PR that was closed. NOTE: I'm not trying to or even asking to check out something that is gone from GH ! The idea is when the development of something begins we provision infrastrcture from our pipelines ... SO WHEN ITS CLOSED WE NEED TO FREE UP THOSE RESOURCES and would like to do it from pipelines as well.

           

          Right now when I close a PR in GH the GH event related to that PR close comes into Jenkins and this message shows up in the "repository events" log of my GHOrg job :

           

          [Thu Nov 30 15:11:36 UTC 2017] Received Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒ http://jenkins:8080/github-webhook/ with timestamp Thu Nov 30 15:11:00 UTC 2017

          Examining myorg/pipeline-app

          Checking branches...

          Getting remote branch story/unprovision1...

             Checking branch story/unprovision1

                ‘cicd/JenkinsfileUnprovisionAWS’ found

              Met criteria

          No changes detected: story/unprovision1 (still at 76c77601680005daeb4)

             1 branches were processed

              Checking pull-requests...

              Getting remote pull request #30...

              0 pull requests were processed

          Finished examining myorg/pipeline-app

          [Thu Nov 30 15:12:04 UTC 2017] Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒http://jenkins:8080/github-webhook/ with timestamp Thu Nov 30 15:11:00 UTC 2017 processed in 27 sec

           

          So making sense of those log messages : PR#30 is a pr that I closed in GH on a branch called story/unprovision1 on a repo in myorg called pipeline-app. There's obviously an event raised in the hook that gets shown in the log as a REMOVED event and then you clearly see "0 pull requests were processed" ...

          Michael Greco added a comment - - edited I'd like to see this available from a GitHubOrg job ... when a Branch or PR is deleted I'd like to be able to run pipeline code on a different branch that then has instructions on removing infrastructure. Lets say all my GH repos in my GH organization have a special branch called "unprovision" which never goes away, and they all have a JenkinsFile called "JenkinsfileUnprovisionInfrastructure". So I set up GitHubOrg job to find all these repos in my organization and importantly I set the refspec to always checkout the "unprovision" branch. At this point the scan runs and discovery completes. Perfect. Now the idea is a person at some later point in Github does a BRANCH DELETE or PR CLOSE and in turn the GH webhook on that repo sends Jenkins this event AND the GitHubOrg job gets the REMOVED event that comes in. My job then executes by checking out the branch identified in the refspec ("unprovision" branch) and runs the Jenkinsfile identified (which is JenkinsfileUnprovisionInfrastructure). That pipeline code (JenkinsfileUnprovisionInfrastructure) which I've written then cleans up the infrastructure related to the GH BRANCH and/or PR that was closed. NOTE: I'm not trying to or even asking to check out something that is gone from GH ! The idea is when the development of something begins we provision infrastrcture from our pipelines ... SO WHEN ITS CLOSED WE NEED TO FREE UP THOSE RESOURCES and would like to do it from pipelines as well.   Right now when I close a PR in GH the GH event related to that PR close comes into Jenkins and this message shows up in the "repository events" log of my GHOrg job :   [Thu Nov 30 15:11:36 UTC 2017] Received Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒ http://jenkins:8080/github-webhook / with timestamp Thu Nov 30 15:11:00 UTC 2017 Examining myorg/pipeline-app Checking branches... Getting remote branch story/unprovision1...    Checking branch story/unprovision1       ‘cicd/JenkinsfileUnprovisionAWS’ found     Met criteria No changes detected: story/unprovision1 (still at 76c77601680005daeb4)    1 branches were processed     Checking pull-requests...     Getting remote pull request #30...     0 pull requests were processed Finished examining myorg/pipeline-app [Thu Nov 30 15:12:04 UTC 2017] Pull request #30 closed in repository myorg/pipeline-app REMOVED event from 192.30.252.42 ⇒ http://jenkins:8080/github-webhook / with timestamp Thu Nov 30 15:11:00 UTC 2017 processed in 27 sec   So making sense of those log messages : PR#30 is a pr that I closed in GH on a branch called story/unprovision1 on a repo in myorg called pipeline-app. There's obviously an event raised in the hook that gets shown in the log as a REMOVED event and then you clearly see "0 pull requests were processed" ...

          Multibranch can't handle such case because of it's design with spaghetti triggering decision logic from different places and tones of optimisations.

          Multibranch api is too locked and tied on triggering on hashes.
          SCMHeadObserver is limiting everything and hiding childObserver like i mentioned in https://groups.google.com/forum/#!topic/jenkinsci-dev/gEH1EdzMblM

          The other problem is that it impossible to have branch specific triggering settings, scmfiltering and etc extensions works only one side and only on hashes.

          Probably some other generic multi job should be designed instead (only categories, project factory, action storage (instead of every time updating transient), few bas jellys) and that's all needed for trigger developers.

          Kanstantsin Shautsou added a comment - Multibranch can't handle such case because of it's design with spaghetti triggering decision logic from different places and tones of optimisations. Multibranch api is too locked and tied on triggering on hashes. SCMHeadObserver is limiting everything and hiding childObserver like i mentioned in https://groups.google.com/forum/#!topic/jenkinsci-dev/gEH1EdzMblM The other problem is that it impossible to have branch specific triggering settings, scmfiltering and etc extensions works only one side and only on hashes. Probably some other generic multi job should be designed instead (only categories, project factory, action storage (instead of every time updating transient), few bas jellys) and that's all needed for trigger developers.

          I have an idea for a workaround for this that I want to try out: write a Groovy script that is executed periodically as part of a pipeline (through the Jenkins CLI to get the necessary permissions) and scans the deleted pull requests/branches that are in the grace period in which they have not been deleted yet from the Jenkins multiproject folder.

          If I can get that list, it would be easy to then idempotently run cleanup jobs. They won't run as soon as the pull request/branch is gone, but the frequency of this cleanup can be tuned.

          I use a similar approach to clean up old agents:

          https://github.com/elifesciences/elife-alfred-formula/blob/master/jenkinsfiles/ec2-plugin-nodes-retirement.groovy

          although here it's more difficult to understand the API to traverse and whether it's possible to list all of these.

          Giorgio Sironi added a comment - I have an idea for a workaround for this that I want to try out: write a Groovy script that is executed periodically as part of a pipeline (through the Jenkins CLI to get the necessary permissions) and scans the deleted pull requests/branches that are in the grace period in which they have not been deleted yet from the Jenkins multiproject folder. If I can get that list, it would be easy to then idempotently run cleanup jobs. They won't run as soon as the pull request/branch is gone, but the frequency of this cleanup can be tuned. I use a similar approach to clean up old agents: https://github.com/elifesciences/elife-alfred-formula/blob/master/jenkinsfiles/ec2-plugin-nodes-retirement.groovy although here it's more difficult to understand the API to traverse and whether it's possible to list all of these.

          Jesse Glick added a comment -

          Something along the lines of JENKINS-37220 could be useful for this sort of specialized use case.

          Jesse Glick added a comment - Something along the lines of JENKINS-37220 could be useful for this sort of specialized use case.

          Aytunc BEKEN added a comment -

          Long time passed but if anyone still requires this feature can use the plugin MultiBranch Action Triggers
          https://github.com/jenkinsci/multibranch-action-triggers-plugin

          Aytunc BEKEN added a comment - Long time passed but if anyone still requires this feature can use the plugin MultiBranch Action Triggers https://github.com/jenkinsci/multibranch-action-triggers-plugin

            Unassigned Unassigned
            dziesio Grzegorz Poznachowski
            Votes:
            21 Vote for this issue
            Watchers:
            27 Start watching this issue

              Created:
              Updated: