Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24376

Queue constantly locked when rapidly scheduling builds

    XMLWordPrintable

Details

    Description

      If there is a job that gets very rapidly scheduled, since a Run.id currently must be unique to the second, a lot of the time the Queue will be locked:

      ... waiting on condition [...]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
    at java.lang.Thread.sleep(Native Method)
    at hudson.model.AbstractProject.newBuild(AbstractProject.java:1121)
    - locked <...> (a hudson.model.FreeStyleProject)
    at hudson.model.AbstractProject.createExecutable(AbstractProject.java:1349)
    at hudson.model.AbstractProject.createExecutable(AbstractProject.java:156)
    at hudson.model.Executor.run(Executor.java:211)
    - locked <...> (a hudson.model.Queue)

      which can prevent other things from happening, such as routine queue maintenance, or even rendering of the queue widget (which expects a fresh snapshot every second):

      "Handling ..." ... waiting for monitor entry [...]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at hudson.model.Queue.getItems(Queue.java:685)
    - waiting to lock <...> (a hudson.model.Queue)
    at hudson.model.Queue$CachedItemList.get(Queue.java:217)
    at hudson.model.Queue.getApproximateItemsQuickly(Queue.java:715)
    at hudson.model.View.getApproximateQueueItemsQuickly(View.java:483)

      (This also artificially slows down some functional tests.)

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. JENKINS-19977 Build records get confused by timezone changes

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. JENKINS-24380 Use build numbers as IDs

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link PR 1376

          Activity

            Descending order - Click to sort in ascending order
            jglick Jesse Glick added a comment -

            The fix of JENKINS-24380 should have covered this.

            jglick Jesse Glick added a comment - The fix of JENKINS-24380 should have covered this.
            jglick Jesse Glick added a comment -

            Filed a PR with proposed core changes.

            jglick Jesse Glick added a comment - Filed a PR with proposed core changes.
            jglick Jesse Glick added a comment -

            Oops, forgot about getIDFormatter(), which is also used by build-publisher-plugin and disk-usage-plugin.

            jglick Jesse Glick added a comment - Oops, forgot about getIDFormatter() , which is also used by build-publisher-plugin and disk-usage-plugin .
            jglick Jesse Glick added a comment - - edited

            Compatibility implications: ftppublisher-plugin, jobConfigHistory-plugin, and support-core-plugin seem to assume something about the current ID format and so might be broken. (Just grepping sources for yyyy-MM-dd_HH-mm-ss and ID_FORMATTER.)

            jglick Jesse Glick added a comment - - edited Compatibility implications: ftppublisher-plugin , jobConfigHistory-plugin , and support-core-plugin seem to assume something about the current ID format and so might be broken. (Just grepping sources for yyyy-MM-dd_HH-mm-ss and ID_FORMATTER .)
            jglick Jesse Glick added a comment -

            Perhaps new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss-SSS").setTimeZone(TimeZone.getTimeZone("UTC"))? Would use a similar format to what we already use, without introducing new characters. ISO-8601 etc. are not ideal for this purpose.

            jglick Jesse Glick added a comment - Perhaps new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss-SSS").setTimeZone(TimeZone.getTimeZone("UTC")) ? Would use a similar format to what we already use, without introducing new characters. ISO-8601 etc. are not ideal for this purpose.
            jglick Jesse Glick added a comment -

            The only apparent solution is to switch to an ID format that can encode milliseconds. This could also be made to use UTC rather than local time, solving JENKINS-19977.

            jglick Jesse Glick added a comment - The only apparent solution is to switch to an ID format that can encode milliseconds. This could also be made to use UTC rather than local time, solving JENKINS-19977 .

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: