Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-31039

Queue items are ephemeral leading to api users to lose track of builds.

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • core
    • None
    • Jenkins ver. 1.565.3

      This issue is in response to an issue while utilizing the python API. The corresponding bug for the python api is located at: https://github.com/salimfadhley/jenkinsapi/issues/335
      The issue is that after a build has been invoked, we have a queue item url returned which can then be polled for new/more information like build number and where to find more information about the build after an actual build has started. However, after the job has starts building the queue item URL is removed after about 30 or so seconds (I assume because the queue has not been garbage collected yet). This leaves api users who haven't yet polled the queue item for what job their queue item turned into scratching their head wondering what happened to their queue item. In lay man's terms, "Their build job disappeared".
      example:
      Invoke a build using curl.

      $ curl -i -X POST  http://jenkins.example.com/job/Blank_Job/buildWithParameters --user $username:$password
      HTTP/1.1 201 Created
      Server: Apache-Coyote/1.1
      Location: http://jenkins.example.com/queue/item/336/
      Content-Length: 0
      Date: Thu, 18 Dec 2014 03:03:40 GMT
      

      Noticed that a queue item has been returned back as the location. Provided that the build hasn't started or the queue item garbage collected. Querying the URL returns the build number as well as the build URL.

      $ curl -X POST   http://jenkins.example.com/queue/item/336/api/xml --user $username:$password | grep -i '<executable>.*</executable>' --color -o
        % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                       Dload  Upload   Total   Spent    Left  Speed
      100   992  100   992    0     0   2278      0 --:--:-- --:--:-- --:--:--  2517
      <executable><number>487</number><url>http://jenkins/job/Blank_Job/487/</url></executable>
      

      However, querying the same queue item moments later returns an error 404. Once again, to reiterate that there is some timing to this issue, if the api user queries the queue item on relatively fast interval (about every second or so), before the queue item can be garbage collected, this doesn't normally become a problem. However, it does become a problem for api users who don't want or can't poll on a secondly basis in an attempt to beat the race condition.

      I can see several solutions to this:

      1. Items from the queue are never removed
      2. Guarantee that the url will still return information for a much greater interval (24 HRS might be overkill, but several minutes shouldn't be too difficult).
      3. Leave the removal of items in the queue to the api users
      4. Allow api users to lock the queue item with a time stamp signaling jenkins that the queue item needs to stay alive at least until that time stamp before removing the item. This way, if an api user knows that he isn't going to be asking for the information of the job for quite some time, he can tell jenkins about this before hand. Kind of a, "Hey, I will be checking on this around this time" type of a deal.

      This will allow api users to pick up the bread crumbs on what is happening to their build.

      See also the following issue:
      https://issues.jenkins-ci.org/browse/JENKINS-12827
      Particularly the comment:
      https://issues.jenkins-ci.org/browse/JENKINS-12827?focusedCommentId=218307&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-218307

            Unassigned Unassigned
            rusty Russell Weber
            Votes:
            13 Vote for this issue
            Watchers:
            28 Start watching this issue

              Created:
              Updated: