Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-75765

Bitbucket SCM initialization is Not Performant

XMLWordPrintable

      We have a Jenkins instance with thousands of jobs on it. It looks as if the initialize method is called on each BitbucketSCMSource serially. Causing it to take a long time to start up the instance atlassian-bitbucket-server-integration-plugin/src/main/java/com/atlassian/bitbucket/jenkins/internal/scm/BitbucketSCMSource.java at b56e566df8082ef1bd526708d1f552e84c2ea984 · jenkinsci/atlassian-bitbucket-server-integration-plugin Approximately 10-15 minutes for a large instance. You can see Jenkins logging out information about it's progress loading jobs.

      Logs:

      2025-06-11 16:49:07.994+0000 [id=84]	INFO	c.c.h.p.folder.AbstractFolder#lambda$onLoad$4: Loading job folder-all/job-46 (55.2%)6/11/2025 10:49:23 AM 2025-06-11 16:49:23.000+0000 [id=84]	INFO	c.c.h.p.folder.AbstractFolder#lambda$onLoad$4: Loading job folder-all/job-71/aidaleuc%2Ftest-large-file (93.5%)6/11/2025 10:49:38 AM 2025-06-11 16:49:38.212+0000 [id=84]	INFO	c.c.h.p.folder.AbstractFolder#lambda$onLoad$4: Loading job folder-all/job-103 (123.9%)6/11/2025 10:49:53 AM 2025-06-11 16:49:53.224+0000 [id=84]	INFO	c.c.h.p.folder.AbstractFolder#lambda$onLoad$4: Loading job folder-all/job-127/aidaleuc%2Fjunit (156.7%)6/11/2025 10:50:08 AM 2025-06-11 16:50:08.638+0000 [id=84]	INFO	c.c.h.p.folder.AbstractFolder#lambda$onLoad$4: Loading job folder-all/job-149/aidaleuc%2Fjunit (193.0%)6/11/2025 10:50:23 AM 2025-06-11 16:50:23.912+0000 [id=84]	INFO	c.c.h.p.folder.AbstractFolder#lambda$onLoad$4: Loading job folder-all/job-180/aidaleuc%2Fjunit (242.8%)

      This may be a multi-faceted issue as it appears the Cloudbees folder plugin loads each job in a folder serially. 

      Reproduction Steps

      I was able to reproduce this issue on my test instance using Job DSL to add a bunch of duplicate jobs into a folder. Note I am just using Job DSL to reproduce the issue and the issue is present regardless of whether a job was generated via Job DSL.

      1. Run the following job dsl script

      def folderCount = 200
 folder("folder-all"){
    displayName("folder-all")
  }
for(def i = 0; i < 200; i++){
  multibranchPipelineJob("folder-all/job-${i}"){
    displayName("pod-test-${i}")
    branchSources {
        branchSource {
            source {
                BbS {
                    id('test-${i}')
                    projectName('<project-here>')
                    repositoryName('<repo>')
                    credentialsId('<svc-account>')
                    mirrorName(null)
                    sshCredentialsId(null)
                    serverId('<server-id>')
                    traits {
                        gitBranchDiscovery()
                    }
                }
            }
        }
    }
    factory{
        workflowBranchProjectFactory{
            scriptPath("Jenkinsfile")
        }
    }
  }
}

      2. Restart the Jenkins instance

      3. Notice that startup times are slower and you see the job loading method. It adds approximately 10-20 seconds of startup but as more jobs are added the issue gets worse.

      Edit: 

      I am fairly certain this commit is reponsible
      Fix the unstable config (#449) · jenkinsci/atlassian-bitbucket-server-integration-plugin@5b38280

      I have opened a PR for this and it fixes my immediate issue, however I don't have a great understanding of the potential implications. In any case totally re initializing the SCM source every time it loads from memory seems expensive?
      [JENKINS-75765] Don't Re-Initialize on Serialization by aidanleuck · Pull Request #468 · jenkinsci/atlassian-bitbucket-server-integration-plugin

        1. access-logs.txt
          235 kB

            Unassigned Unassigned
            aidaleuc Aidan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: