Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37727

Multibranch pipeline project jobs are not deleted when branches are deleted

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • git-plugin
    • None
    • git-plugin compiled from master.
      This bug does not exist in git-plugin 2.5.3

      Problem

      If you compile git-plugin from the source, and set 'Discard old items' in the multibranch pipeline, deleted branches on the SCM are not deleted from the job list. If you install the 2.5.3 version of this plugin and restart Jenkins, these branches are deleted.

      How to reproduce it

      Configure a Jenkins installation

      1. Download a fresh Jenkins (2.7.2), rename your current home
      2. Start it: java -jar jenkins.war
      3. Choose the default packages option during the first boot
      4. Clone git-plugin repository, compile it (mvn.cmd '-Dmaven.test.skip=true' clean install hpi:hpi)
      5. Import your git-plugin (the one you compiled) in Jenkins (advanced tab)
      6. Restart Jenkins

      See the error

      1. Create a test git repository with a Jenkinsfile on many branches
      2. Add a new multibranch pipeline, select "Discard old items"
      3. Save, check that your branches appear in the list
      4. Delete a branch from your repository. git push origin :features/a-random-branch
      5. Run a branch indexing
      6. See that your branch was not deleted

      Check that 2.5.3 is working

      1. Restore the git-plugin in its original version, 2.5.3
      2. Restart Jenkins
      3. Rerun branch indexing, see that your branch has disappeared !

      Investigations

      I didn't have the time to run a Jenkins instance in debug, or to read the commits yet. So I've no idea why this bug happens. I'll update this section later if I've time.

          [JENKINS-37727] Multibranch pipeline project jobs are not deleted when branches are deleted

          Mark Waite added a comment -

          Included in git plugin 2.6.0, released 2 Sep 2016. Was never visible in any released version of the plugin.

          Mark Waite added a comment - Included in git plugin 2.6.0 , released 2 Sep 2016. Was never visible in any released version of the plugin.

          Leibo added a comment - - edited

          Hi, I just installed 2.7.3 , using Git plugin 2.6.0, configured a multi-branch job, included feature* branches where I have 3 of them.
          3 jobs were created as expected.

          I then deleted one branch but I still see it in the indexing log...
          Seen branch in repository origin/feature/XAVI_MULTI_Demo1
          Seen branch in repository origin/feature/XAVI_MULTI_Demo2
          Seen branch in repository origin/feature/XAVI_MULTI_Demo3

          While on git there's no such branch

          $ git branch -a|grep -i demo
            feature/XAVI_MULTI_Demo1
            feature/XAVI_MULTI_Demo2
            remotes/origin/feature/XAVI_MULTI_Demo1
            remotes/origin/feature/XAVI_MULTI_Demo2
          

          On the master I see the workspace folders

          feature%2FXAVI_MULTI_Demo1@script
          feature%2FXAVI_MULTI_Demo2@script
          feature%2FXAVI_MULTI_Demo3@script
          

          and the deleted branch folder is empty...

          After I removed the git-xxxxx folder from Jenkins caches folder - reindexing was ok.
          Is it related to this issue as well?

          Leibo added a comment - - edited Hi, I just installed 2.7.3 , using Git plugin 2.6.0, configured a multi-branch job, included feature* branches where I have 3 of them. 3 jobs were created as expected. I then deleted one branch but I still see it in the indexing log... Seen branch in repository origin/feature/XAVI_MULTI_Demo1 Seen branch in repository origin/feature/XAVI_MULTI_Demo2 Seen branch in repository origin/feature/XAVI_MULTI_Demo3 While on git there's no such branch $ git branch -a|grep -i demo feature/XAVI_MULTI_Demo1 feature/XAVI_MULTI_Demo2 remotes/origin/feature/XAVI_MULTI_Demo1 remotes/origin/feature/XAVI_MULTI_Demo2 On the master I see the workspace folders feature%2FXAVI_MULTI_Demo1@script feature%2FXAVI_MULTI_Demo2@script feature%2FXAVI_MULTI_Demo3@script and the deleted branch folder is empty... After I removed the git-xxxxx folder from Jenkins caches folder - reindexing was ok. Is it related to this issue as well?

          Mark Waite added a comment - - edited

          assafl, that sounds like the same condition which shows the problem (sometimes) in my Docker test instance.

          Could you try switching that specific pipeline job definition from using the "Default" git implementation to using "jgit" as the implementation? Prior to git plugin 2.6.0, the only implementation available to the pipeline branch indexing was the "jgit" implementation and it seemed well behaved unless "checkout to local branch" was specified. The switch from "Default" to "jgit" should be available in the Jenkins job definition with the label "Git executable". Choices in that pick list should be "Default" or "jgit".

          Could you also try without the "Checkout to local branch" extension enabled?

          Removing the git-xxxx folder from the Jenkins cache directory will (at least temporarily) fix the issue, since that will remove the deleted branch from the cache repository. However, it seems likely that the problem will reappear the next time a branch is deleted, since the prune() call in the cached repository seems to not be removing that deleted branch.

          Mark Waite added a comment - - edited assafl , that sounds like the same condition which shows the problem (sometimes) in my Docker test instance. Could you try switching that specific pipeline job definition from using the "Default" git implementation to using "jgit" as the implementation? Prior to git plugin 2.6.0, the only implementation available to the pipeline branch indexing was the "jgit" implementation and it seemed well behaved unless "checkout to local branch" was specified. The switch from "Default" to "jgit" should be available in the Jenkins job definition with the label "Git executable". Choices in that pick list should be "Default" or "jgit". Could you also try without the "Checkout to local branch" extension enabled? Removing the git-xxxx folder from the Jenkins cache directory will (at least temporarily) fix the issue, since that will remove the deleted branch from the cache repository. However, it seems likely that the problem will reappear the next time a branch is deleted, since the prune() call in the cached repository seems to not be removing that deleted branch.

          Leibo added a comment -

          I don't see any options to change the definition... for the 'Add source' config item I can only pick from Git, Github, Single repo or subversion

          Leibo added a comment - I don't see any options to change the definition... for the 'Add source' config item I can only pick from Git, Github, Single repo or subversion

          Mark Waite added a comment - - edited

          If you don't see the phrase "Git executable" in your job definition page (it is inside the Git configuration section of the job definition page), then that means you haven't yet enabled "jgit" as a globally available git implementation in your Jenkins server.

          Add jgit as an available global git implementation in your Jenkins server by clicking "Manage Jenkins", then "Global Tool Configuration". In the "Git" section of that page, click the "Add" button and select "jgit" from the dropdown list.

          Mark Waite added a comment - - edited If you don't see the phrase "Git executable" in your job definition page (it is inside the Git configuration section of the job definition page), then that means you haven't yet enabled "jgit" as a globally available git implementation in your Jenkins server. Add jgit as an available global git implementation in your Jenkins server by clicking "Manage Jenkins", then "Global Tool Configuration". In the "Git" section of that page, click the "Add" button and select "jgit" from the dropdown list.

          Leibo added a comment - - edited

          Hi,

          Added the JGit as explained and got authentication error... after some issues searching I created a new user based on our SSH private key authentication and it seems working!!!

          So we're good I hope.

          Thanks a lot for the fast response

          Leibo added a comment - - edited Hi, Added the JGit as explained and got authentication error... after some issues searching I created a new user based on our SSH private key authentication and it seems working!!! So we're good I hope. Thanks a lot for the fast response

          Mark Waite added a comment -

          I'm glad it is working for you, though I'm skeptical that the issue is fully resolved. I don't yet understand the conditions which caused you (and me) to see the issue, while superboum and I to not see the issue. I realize it is strange that I've both seen the problem, and not seen the problem, but that just means I don't really understand the circumstances which show the bug.

          Mark Waite added a comment - I'm glad it is working for you, though I'm skeptical that the issue is fully resolved. I don't yet understand the conditions which caused you (and me) to see the issue, while superboum and I to not see the issue. I realize it is strange that I've both seen the problem, and not seen the problem, but that just means I don't really understand the circumstances which show the bug.

          Øystein Kjærnet added a comment - - edited

          I experience exactly the same as Leibo - old jobs from deleted branches not being deleted, even though I've selected "Discard old items" in the build config - choosing JGit as suggested by Mark fixed this. Jenkins version 2.20, Git plugin 3.0.0, Git client plugin 2.0.0, Build Pipeline 1.5.4, Pipeline: Multibranch 2.8, git version on system: 1.7.1. I'd be glad to help debugging this in any way...

          Øystein Kjærnet added a comment - - edited I experience exactly the same as Leibo - old jobs from deleted branches not being deleted, even though I've selected "Discard old items" in the build config - choosing JGit as suggested by Mark fixed this. Jenkins version 2.20, Git plugin 3.0.0, Git client plugin 2.0.0, Build Pipeline 1.5.4, Pipeline: Multibranch 2.8, git version on system: 1.7.1. I'd be glad to help debugging this in any way...

          Mark Waite added a comment - - edited

          gelefisk can you confirm that in your environment git plugin 2.5.3 does not show the problem, while git plugin 2.6.0 and git plugin 3.0.0 show the problem?

          Can you also explore further to see if you can detect the set of steps which would show the problem on a fresh installation?

          If that works, then could you explore placing that fresh set of steps into a job on a docker instance (preferably as a pull request to the bugs folder of my lts-with-plugins branch)?

          Mark Waite added a comment - - edited gelefisk can you confirm that in your environment git plugin 2.5.3 does not show the problem, while git plugin 2.6.0 and git plugin 3.0.0 show the problem? Can you also explore further to see if you can detect the set of steps which would show the problem on a fresh installation? If that works, then could you explore placing that fresh set of steps into a job on a docker instance (preferably as a pull request to the bugs folder of my lts-with-plugins branch )?

          Marius Balteanu added a comment - - edited

          I've the same issue and downgrading the Git plugin to 2.5.3 didn't solve the issue. I'll try the jgit workaround.

          • git client plugin 2.1.0
          • git plugin 2.5.3 / 3.0.0
          • pipeline multibranch 2.9.2
          • git version on the server 2.9.2

          Please ignore my above comment because my problem was from the job configuration.

          Marius Balteanu added a comment - - edited I've the same issue and downgrading the Git plugin to 2.5.3 didn't solve the issue. I'll try the jgit workaround. git client plugin 2.1.0 git plugin 2.5.3 / 3.0.0 pipeline multibranch 2.9.2 git version on the server 2.9.2 Please ignore my above comment because my problem was from the job configuration.

            markewaite Mark Waite
            superboum Quentin Dufour
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: