We are seeing a weird issue with our Jenkins / GitHub Org integration. It seems that randomly, all the build history for pipelines inside a GitHub Org folder disappear. They also trigger builds on all the pipelines inside the org folder. I have looked through the Jenkins log and haven't found anything that is obviously triggering this. 

      Edit: It also is triggering all PRs in all repos in the Org to rebuild.

          [JENKINS-63555] GitHub Org being reset randomly

          Following up on this issue, it looks like whatever triggers this is causing all the open PRs to build, causing hundreds of build jobs to build at the same time, spiking CPU on the head node.

          Cesar Covarrubias added a comment - Following up on this issue, it looks like whatever triggers this is causing all the open PRs to build, causing hundreds of build jobs to build at the same time, spiking CPU on the head node.

          Liam Newman added a comment -

          ccovarru
          What is likely happening is an edge case where the scan fails but doesn't know that it failed and marks all children for deletion. The next scan succeeds and believe it has found all new repos (with new branches and PRs).

          If you set your "Orphaned Item Strategy" to retain old items for 1 day and leave the number of items blank (unlimited), that should at least keep your repos, branches and PRs from being deleted. They may still all run with then are found again, but maybe not.

          Some additional questions:
          Are you using github app credentials?
          Are you using github.com or GitHub Enterprise?
          Do your Jenkins System logs have warnings or alerts related to repo scanning?

          Liam Newman added a comment - ccovarru What is likely happening is an edge case where the scan fails but doesn't know that it failed and marks all children for deletion. The next scan succeeds and believe it has found all new repos (with new branches and PRs). If you set your "Orphaned Item Strategy" to retain old items for 1 day and leave the number of items blank (unlimited), that should at least keep your repos, branches and PRs from being deleted. They may still all run with then are found again, but maybe not. Some additional questions: Are you using github app credentials? Are you using github.com or GitHub Enterprise? Do your Jenkins System logs have warnings or alerts related to repo scanning?

          We did have Orphaned Strategy set to retain items for 10 days and 10 items. I just set the number of items for unlimited. I anticipate this happening again  since it's been happening every day or 2. 

          Yes we are using GitHub App creds.
          Our org is on github.com, but with a business account. 

          The main log we see is "GitHub webhooks activated for job OrgName/repo" for all our repos that match.

          Cesar Covarrubias added a comment - We did have Orphaned Strategy set to retain items for 10 days and 10 items. I just set the number of items for unlimited. I anticipate this happening again  since it's been happening every day or 2.  Yes we are using GitHub App creds. Our org is on github.com, but with a business account.  The main log we see is "GitHub webhooks activated for job OrgName/repo" for all our repos that match.

            Unassigned Unassigned
            ccovarru Cesar Covarrubias
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: