Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-72360

Pipeline job seed dsl is very slow

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • core, job-dsl-plugin
    • None

      Jenkins job seed dsl is extremely slow. We are using version Jenkins 2.414.3.
      Job DSL Version1.87

      We have a main pipeline that creates multiple downstream jobs using such syntax:

                              for (String jobName : pipelineQualityGatesJobs) {
                                  jobDsl scriptText: """pipelineJob("${env.JOB_NAME}_${devJobPrefix}${jobName}") {
                                           parameters {
                                              stringParam('branch',"${gitBranch}",'branch')
                                              stringParam('buildmajor',"${pipelineMajor}",'release major identifier')
                                              stringParam('buildminor',"${pipelineMinor}",'release minor identifier')
                                              stringParam('fix', "${pipelineFix}",'fix level')
                                              stringParam('commitID', "${gitCommitID}",'git commit id')
                                              stringParam('buildnumber', "${pipelineBuildNumber}",'build number')
                                              stringParam('parentjob', "${pipelineName}",'optional parent job')
                                              stringParam('devuser', "${devuser}")
                                              stringParam('pipelinesBranch', "${jenkinsLibsBranch}")
                                              stringParam('shallowCloneDepth', "${shallowCloneDepthParam}")
                                          }
                                          removedJobAction: 'IGNORE'
      									ignoreExisting: 'TRUE'
                                          definition {
                                              cpsScm {
                                                      scm {
                                                          git {
                                                              remote {
                                                                  url("${jenkinsRepository}")
                                                                  credentials('UM.GH.read.write')
                                                              }
                                                              branch("${jenkinsLibsBranch}")
                                                          }
                                                      }
                                                      scriptPath('jenkinsfiles/${jobName}/jenkinsfile.groovy') 
                                              }
                                          }
                                      }
                                      """.stripIndent()
                              }
      

      The downstream jobs are approximately 20. This sometimes takes 4 hours.

      2023-11-22 12:55:10  Processing provided DSL script
      2023-11-22 12:56:20  Unreferenced items:
      

      Here is a thread dump:

      "org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution [#786]" Id=107307 Group=main BLOCKED on javaposse.jobdsl.plugin.DescriptorImpl@f128c3d owned by "org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution [#838]" Id=114970
      	at hudson.model.Descriptor.save(Descriptor.java:910)
      	-  *blocked on javaposse.jobdsl.plugin.DescriptorImpl@f128c3d*
      	at javaposse.jobdsl.plugin.ExecuteDslScripts.updateGeneratedJobMap(ExecuteDslScripts.java:559)
      	at javaposse.jobdsl.plugin.ExecuteDslScripts.updateGeneratedJobs(ExecuteDslScripts.java:520)
      	at javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:380)
      	at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
      	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
      	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
      	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$$Lambda$1155/0x000000080177c840.run(Unknown Source)
      	at java.base@11.0.14.1/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      	at java.base@11.0.14.1/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base@11.0.14.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base@11.0.14.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base@11.0.14.1/java.lang.Thread.run(Thread.java:829)
       
      	Number of locked synchronizers = 1
      	- java.util.concurrent.ThreadPoolExecutor$Worker@545761b4
      org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution [#838] 
      "org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution [#838]" Id=114970 Group=main RUNNABLE
      	at java.base@11.0.14.1/java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
      	at java.base@11.0.14.1/java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:248)
      	at java.base@11.0.14.1/java.io.File.exists(File.java:831)
      	at hudson.plugins.jobConfigHistory.HistoryFileFilter.accept(HistoryFileFilter.java:54)
      	at java.base@11.0.14.1/java.io.File.listFiles(File.java:1342)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.getRevisionsFromHistoriesDir(FileHistoryDao.java:553)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.getRevisions(FileHistoryDao.java:540)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.getRevisions(FileHistoryDao.java:534)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.hasDuplicateHistory(FileHistoryDao.java:879)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.checkDuplicate(FileHistoryDao.java:905)
      	at hudson.plugins.jobConfigHistory.FileHistoryDao.saveItem(FileHistoryDao.java:420)
      	at hudson.plugins.jobConfigHistory.JobConfigHistorySaveableListener.onChange(JobConfigHistorySaveableListener.java:62)
      	at hudson.model.listeners.SaveableListener.fireOnChange(SaveableListener.java:82)
      	at hudson.model.Descriptor.save(Descriptor.java:913)
      	-  locked javaposse.jobdsl.plugin.DescriptorImpl@f128c3d
      	at javaposse.jobdsl.plugin.ExecuteDslScripts.updateGeneratedJobMap(ExecuteDslScripts.java:559)
      	at javaposse.jobdsl.plugin.ExecuteDslScripts.updateGeneratedJobs(ExecuteDslScripts.java:520)
      	at javaposse.jobdsl.plugin.ExecuteDslScripts.perform(ExecuteDslScripts.java:380)
      	at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
      	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
      	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
      	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
      	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$$Lambda$1155/0x000000080177c840.run(Unknown Source)
      	at java.base@11.0.14.1/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      	at java.base@11.0.14.1/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base@11.0.14.1/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base@11.0.14.1/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base@11.0.14.1/java.lang.Thread.run(Thread.java:829)
       
      	Number of locked synchronizers = 1
      	- java.util.concurrent.ThreadPoolExecutor$Worker@409fda9a
      

      Looking for hints how to avoid this or what causes it?

            jamietanna Jamie Tanna
            vassilena Vassilena Treneva
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: