Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62486

Multibranch jobs rebuild and lose build history when source control is down

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      We used Mercurial (hg) for source control and most of our Jenkins jobs are multi-branch jobs.  They all have skipDefaultCheckout set to false and have a build step that calls `deleteDir` and then `checkout scm` (because it doesn't appear as though default checkout can do a clean checkout.)

      When our version control server is goes down and then comes back up (like a reboot), all the multibranch builds are triggered even though nothing in the code has changed.  Additionally, the lose all their build history

       

        Attachments

          Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-48867 Mercurial multibranch pipeline deletes all branches when repository is down

          • Major - Major loss of function.
          • Resolved

            Activity

            Hide
            Permalink
            jessebs jessebs added a comment -

            Some more details.  We're no longer deleting the directory but instead doing a clean checkout.  The problem still exists. 

            It appears as though Jenkins will delete all known details of branches in the time our source control server is down.

            Here's a relevant log:

            Started by timer
[Wed May 27 17:50:14 UTC 2020] Starting branch indexing...
Acquired master cache lock.
[85701BDF9C7C8AEC476538CEBD36486C4DA433CD-repo] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" pull
pulling from https://example.com/scm/hg/repo
abort: HTTP Error 502: Bad Gateway
ERROR: Failed to update /var/lib/jenkins/hgcache/85701BDF9C7C8AEC476538CEBD36486C4DA433CD-repo
Master cache lock released.
ERROR: Could not use caches, not fetching branch heads
[Wed May 27 17:50:14 UTC 2020] Finished branch indexing. Indexing took 0.28 sec
Evaluating orphaned items in repo
Will remove default
Finished: SUCCESS

            I would think that the unsuccessful response would cause the job to fail, thereby avoiding branch indexing and removal of branches

            Show
            jessebs jessebs added a comment - Some more details.  We're no longer deleting the directory but instead doing a clean checkout.  The problem still exists.  It appears as though Jenkins will delete all known details of branches in the time our source control server is down. Here's a relevant log: Started by timer [Wed May 27 17:50:14 UTC 2020] Starting branch indexing... Acquired master cache lock. [85701BDF9C7C8AEC476538CEBD36486C4DA433CD-repo] $ hg --config auth.jenkins.prefix=* --config ******** --config ******** --config "auth.jenkins.schemes=http https" pull pulling from https://example.com/scm/hg/repo abort: HTTP Error 502: Bad Gateway ERROR: Failed to update /var/lib/jenkins/hgcache/85701BDF9C7C8AEC476538CEBD36486C4DA433CD-repo Master cache lock released. ERROR: Could not use caches, not fetching branch heads [Wed May 27 17:50:14 UTC 2020] Finished branch indexing. Indexing took 0.28 sec Evaluating orphaned items in repo Will remove default Finished: SUCCESS I would think that the unsuccessful response would cause the job to fail, thereby avoiding branch indexing and removal of branches

              People

              Assignee:
              vaimr Denis Saponenko
              Reporter:
              jessebs jessebs
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: