Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24376

Queue constantly locked when rapidly scheduling builds

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      If there is a job that gets very rapidly scheduled, since a Run.id currently must be unique to the second, a lot of the time the Queue will be locked:

      ... waiting on condition [...]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
          at java.lang.Thread.sleep(Native Method)
          at hudson.model.AbstractProject.newBuild(AbstractProject.java:1121)
          - locked <...> (a hudson.model.FreeStyleProject)
          at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1349)
          at hudson.model.AbstractProject.createExecutable(AbstractProject.java:156)
          at hudson.model.Executor.run(Executor.java:211)
          - locked <...> (a hudson.model.Queue)
      

      which can prevent other things from happening, such as routine queue maintenance, or even rendering of the queue widget (which expects a fresh snapshot every second):

      "Handling ..." ... waiting for monitor entry [...]
         java.lang.Thread.State: BLOCKED (on object monitor)
          at hudson.model.Queue.getItems(Queue.java:685)
          - waiting to lock <...> (a hudson.model.Queue)
          at hudson.model.Queue$CachedItemList.get(Queue.java:217)
          at hudson.model.Queue.getApproximateItemsQuickly(Queue.java:715)
          at hudson.model.View.getApproximateQueueItemsQuickly(View.java:483)
      

      (This also artificially slows down some functional tests.)

        Attachments

          Issue Links

            Activity

            Hide
            jglick Jesse Glick added a comment -

            The only apparent solution is to switch to an ID format that can encode milliseconds. This could also be made to use UTC rather than local time, solving JENKINS-19977.

            Show
            jglick Jesse Glick added a comment - The only apparent solution is to switch to an ID format that can encode milliseconds. This could also be made to use UTC rather than local time, solving JENKINS-19977 .
            Hide
            jglick Jesse Glick added a comment -

            Perhaps new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss-SSS").setTimeZone(TimeZone.getTimeZone("UTC"))? Would use a similar format to what we already use, without introducing new characters. ISO-8601 etc. are not ideal for this purpose.

            Show
            jglick Jesse Glick added a comment - Perhaps new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss-SSS").setTimeZone(TimeZone.getTimeZone("UTC")) ? Would use a similar format to what we already use, without introducing new characters. ISO-8601 etc. are not ideal for this purpose.
            Hide
            jglick Jesse Glick added a comment - - edited

            Compatibility implications: ftppublisher-plugin, jobConfigHistory-plugin, and support-core-plugin seem to assume something about the current ID format and so might be broken. (Just grepping sources for yyyy-MM-dd_HH-mm-ss and ID_FORMATTER.)

            Show
            jglick Jesse Glick added a comment - - edited Compatibility implications: ftppublisher-plugin , jobConfigHistory-plugin , and support-core-plugin seem to assume something about the current ID format and so might be broken. (Just grepping sources for yyyy-MM-dd_HH-mm-ss and ID_FORMATTER .)
            Hide
            jglick Jesse Glick added a comment -

            Oops, forgot about getIDFormatter(), which is also used by build-publisher-plugin and disk-usage-plugin.

            Show
            jglick Jesse Glick added a comment - Oops, forgot about getIDFormatter() , which is also used by build-publisher-plugin and disk-usage-plugin .
            Hide
            jglick Jesse Glick added a comment -

            Filed a PR with proposed core changes.

            Show
            jglick Jesse Glick added a comment - Filed a PR with proposed core changes.
            Hide
            jglick Jesse Glick added a comment -

            The fix of JENKINS-24380 should have covered this.

            Show
            jglick Jesse Glick added a comment - The fix of JENKINS-24380 should have covered this.

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              jglick Jesse Glick
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: