Details
-
Bug
-
Status: Open (View Workflow)
-
Minor
-
Resolution: Unresolved
-
None
-
jenkins 2.176.1
latest versions of all plugins in LTS
Description
Jenkins handles a monorepo well enough when using git scm with ssh clone. If we have the following repo structure:
// code placeholder
api-one/
Jenkinsfile
api-two/
Jenkinsfile
...
api-n/
Jenkinsfile
We create one multibranch pipeline per Jenkinsfile in order to have many independent pipelines (CI + CD) for our projects. These projects all share common code through jenkins shared libraries and everything works ok.
However, if we want to use the github branch source and want to enable webhooks from github we immediately run into rate limit issues. We have a large repository - thousands of pushes a day. Every time a webhook is fired from github and hits jenkins all N multibranch pipelines begin scanning ALL remote refs for the repository. In our repository there are 10s of thousands of tags.
A single scan on a single multibranch pipeline eliminates our entire github rate limit budget.
I am not entirely sure why jenkins must rescan the entire repo. It seems at the very least the webhook from github contains all information about the ref to be built and the individual jobs could simply take the webhook at face value and build/delete/etc without requiring a full repository scan.
Is this possible? Why does Jenkins currently scan the entire repo?
Attachments
Activity
connor poole
created issue -
thinkinfinite cpoole
Is this still happening using the latest version of github-branch-source-plugin?
could you include the config.xml for one of you multibranch jobs? Triggers should only be pulling one branch not all of them.
Hi,
I think we have similar issue that we have a job that triggering on tags with github webhook, but now it can be triggered by almost all Github actions, such as creating/deleting branches, committing codes/pulling request to branches, etc. However the other jobs are working well even those are defined to be triggered on tags. I've tried to copy the working job's Jenkinsfile onto the issue job but had no luck as I got the same result.
In the past, this issue happened sometimes, and it can be resolved by temporary disabling the job or unchecking "GitHub hook trigger for GITScm polling" under "Build Triggers", but this time it doesn't work anymore.
Below is the checkout code:
checkout([$class: 'GitSCM', branches: [[name: '**/tags/**']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '$MyCredentialsId', refspec: '+refs/tags/*:refs/remotes/origin/tags/*', url: '$Github_Url']]])