Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68444

Jobs deleted during GitHub Organization Scan and Github Enterprise down

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • Jenkins version: 2.319.2
      github-branch-source: 2.11.4
      github-api: 1.301-378.v9807bd746da5
      branch-api: 2.7.0

      We are using Github Enterprise 3.3.1 on-prem and using the github-branch-source plugin to manage the creation of Jenkins pipeline jobs.

      It seems anytime our Github Enterprise server is down due to maintenance/problems, we end up with a small percentage of deleted branch jobs/deleted build history and reset build numbers.  So far there is a 100% correlation with some form of Github Enterprise maintenance/down time.

      All because it was unable to find the Jenkinsfile when the Organization Scan runs while Github down for maintenance or having issues.    Here is an example of the scan not finding Jenkinsfiles even thou they do exist in Github and showing the removal of the branch jobs:

      Started by timer
      [Thu Jan 27 17:16:00 CST 2022] Starting branch indexing...
      17:16:01 Connecting to https://github.foobar.com/api/v3 using GitHub IGX API token for githuborg/automated tasks
      Examining iegc/bar  Checking branches...  Getting remote branches...    Checking branch master
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch AUL
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch AUL-testing
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch D3333
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch D4444
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch D4444-OneOff
            ‘Jenkinsfile’ found
          Met criteria
      No changes detected: D4444-OneOff (still at 8e899399ad1eb593e14f162e19c608b817d031ed)    Checking branch D5111
            ‘Jenkinsfile’ found
          Met criteria
      No changes detected: D5111 (still at e43cfc44627c088962f5d0b892060e1f4fc47c5f)    Checking branch D8888
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch D9999
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch D9222
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch F14570-development
            ‘Jenkinsfile’ found
          Met criteria
      No changes detected: F14570-development (still at 2ea9b0764160ec2d5e6be984cfdeec1faa29d14d)    Checking branch FeatureToggleEnableDisable
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch S11111
            ‘Jenkinsfile’ not found
          Does not meet criteria    Checking branch clazy-fixes
            ‘Jenkinsfile’ found
          Met criteria
      No changes detected: clazy-fixes (still at 81c9cc25dff71daf96a69a30bab1d70ce2c4d99a)    Checking branch bar_build_hack
            ‘Jenkinsfile’ not found
          Does not meet criteria  17 branches were processed  Checking pull-requests...  Getting remote pull requests...  0 pull requests were processedFinished examining iegc/bar[Thu Jan 27 17:16:06 CST 2022] Finished branch indexing. Indexing took 5.7 sec
      Evaluating orphaned items in iegci » bar
      Will not remove S11111 because it is new
      Will not remove D8888 because it is new
      Will not remove D4444 because it is new
      Will not remove D9999 because it is new
      Will not remove master because it is new
      Will remove AUL-testing as it is too old
      Will remove AUL as it is too old
      Will remove D9222 as it is too old
      Will remove D3333 as it is too old
      Will remove D9000 as it is too old
      Will remove FeatureToggleEnableDisable as it is too old
      Will remove bar_build_hack as it is too old
      Finished: SUCCESS

      We have Organization Scans set to run once per day and the logs indicated that it had run during the maintenance window which is why I think it only affects a small number of our Github Orgs.

      I am wondering is this is some caching issue in github-api or issue with Github Enterprise.  If Github Enerprise is down for maintenance, how can these API calls return 200.  We have a scheduled maintenance window in the next week and want to collect/more information to help track this issue down.

            Unassigned Unassigned
            johnlengeling John Lengeling
            Votes:
            3 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: