-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Jenkins version: 2.319.2
github-branch-source: 2.11.4
github-api: 1.301-378.v9807bd746da5
branch-api: 2.7.0
We are using Github Enterprise 3.3.1 on-prem and using the github-branch-source plugin to manage the creation of Jenkins pipeline jobs.
It seems anytime our Github Enterprise server is down due to maintenance/problems, we end up with a small percentage of deleted branch jobs/deleted build history and reset build numbers. So far there is a 100% correlation with some form of Github Enterprise maintenance/down time.
All because it was unable to find the Jenkinsfile when the Organization Scan runs while Github down for maintenance or having issues. Here is an example of the scan not finding Jenkinsfiles even thou they do exist in Github and showing the removal of the branch jobs:
Started by timer
[Thu Jan 27 17:16:00 CST 2022] Starting branch indexing...
17:16:01 Connecting to https://github.foobar.com/api/v3 using GitHub IGX API token for githuborg/automated tasks
Examining iegc/bar Checking branches... Getting remote branches... Checking branch master
‘Jenkinsfile’ not found
Does not meet criteria Checking branch AUL
‘Jenkinsfile’ not found
Does not meet criteria Checking branch AUL-testing
‘Jenkinsfile’ not found
Does not meet criteria Checking branch D3333
‘Jenkinsfile’ not found
Does not meet criteria Checking branch D4444
‘Jenkinsfile’ not found
Does not meet criteria Checking branch D4444-OneOff
‘Jenkinsfile’ found
Met criteria
No changes detected: D4444-OneOff (still at 8e899399ad1eb593e14f162e19c608b817d031ed) Checking branch D5111
‘Jenkinsfile’ found
Met criteria
No changes detected: D5111 (still at e43cfc44627c088962f5d0b892060e1f4fc47c5f) Checking branch D8888
‘Jenkinsfile’ not found
Does not meet criteria Checking branch D9999
‘Jenkinsfile’ not found
Does not meet criteria Checking branch D9222
‘Jenkinsfile’ not found
Does not meet criteria Checking branch F14570-development
‘Jenkinsfile’ found
Met criteria
No changes detected: F14570-development (still at 2ea9b0764160ec2d5e6be984cfdeec1faa29d14d) Checking branch FeatureToggleEnableDisable
‘Jenkinsfile’ not found
Does not meet criteria Checking branch S11111
‘Jenkinsfile’ not found
Does not meet criteria Checking branch clazy-fixes
‘Jenkinsfile’ found
Met criteria
No changes detected: clazy-fixes (still at 81c9cc25dff71daf96a69a30bab1d70ce2c4d99a) Checking branch bar_build_hack
‘Jenkinsfile’ not found
Does not meet criteria 17 branches were processed Checking pull-requests... Getting remote pull requests... 0 pull requests were processedFinished examining iegc/bar[Thu Jan 27 17:16:06 CST 2022] Finished branch indexing. Indexing took 5.7 sec
Evaluating orphaned items in iegci » bar
Will not remove S11111 because it is new
Will not remove D8888 because it is new
Will not remove D4444 because it is new
Will not remove D9999 because it is new
Will not remove master because it is new
Will remove AUL-testing as it is too old
Will remove AUL as it is too old
Will remove D9222 as it is too old
Will remove D3333 as it is too old
Will remove D9000 as it is too old
Will remove FeatureToggleEnableDisable as it is too old
Will remove bar_build_hack as it is too old
Finished: SUCCESS
We have Organization Scans set to run once per day and the logs indicated that it had run during the maintenance window which is why I think it only affects a small number of our Github Orgs.
I am wondering is this is some caching issue in github-api or issue with Github Enterprise. If Github Enerprise is down for maintenance, how can these API calls return 200. We have a scheduled maintenance window in the next week and want to collect/more information to help track this issue down.