Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-45131

Locking Resources is slow




      Locking and Unlocking takes very long for no apparent reason.

      Take this example pipeline:

      node {
          // change workspace to shared directory
          exws(extWorkspace) {
              timeout(time: 30, unit: 'MINUTES') {
                  stage('build') {
                      checkout scm
                      nodeEnv {
                          sh 'ant build'
                      milestone 1
                  } // stage build
              } // timeout
          } // exws
      } // node
      lock(resource: 'deploy', inversePrecedence: true) {
          // do not deploy older versions of the artifacts (NON-UNIQUE-SNAPSHOTS)
          milestone 2
          node {
              exws(extWorkspace) {
                  stage('deploy') {
                      sh 'ant deploy'
                  } // stage deploy
              } // extws
          }  // node
      } // lock

       This is how the Build-Log looks (sometimes):

      12:04:42 BUILD SUCCESSFUL
      12:04:42 Total time: 3 minutes 37 seconds
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] milestone
      12:04:42 Trying to pass milestone 1
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      [Pipeline] // exws
      [Pipeline] }
      [Pipeline] // node

      here is the first problem. I see the job leaving the node, but not that it tries to lock a resource, that message will be printed after the lock is acquired (even though the timestamp implies something else)!

      [Pipeline] lock
      12:04:42 Trying to acquire lock on [deploy]
      12:13:36 Lock acquired on [deploy]
      [Pipeline] {
      [Pipeline] milestone
      12:13:36 Trying to pass milestone 2
      [Pipeline] node

      and the second problem is that it took nearly 10 minutes to get the lock while there was only one other non related job running. So the lock should be obtainable super fast and the overall workload of the system was low.

      Not only the jobs seem to have these performance drops, but also when going to <yourjenkins.url>/lockable-resources/ there are times when it takes minutes to see the currently locked resources.




            Unassigned Unassigned
            dageissl Daniel Geißler
            7 Vote for this issue
            7 Start watching this issue