Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37338

Pipeline scripts that should appear in /queue are returned in /runs API

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Minor Minor
    • blueocean-plugin
    • None

      from cliffmeyers:
      Hitting the /runs REST API for a Pipeline shows all the job runs, including those that probably should be in the /queue. The status of all jobs is "RUNNING" which also feels a bit misleading.

      from vivek
      What needs to happen is that if a build item is in queue, should not be returned. We should not be mixing run object with queue because they are vastly different. There is already an API /pipeline/:id/activities Ivan implemented to give all runs and items in queue, so if this is what you look for you can use that.

          [JENKINS-37338] Pipeline scripts that should appear in /queue are returned in /runs API

          Michael Neale added a comment -

          I think the missing piece is showing the activity of Jenkins in the not-yet-implemented GUI, this would make it clear...

          otherwise, if there was a way we could enquire as to what "parts" of a pipeline are doing... we could report more intelligently why it is waiting when it is in the RUNNING state...

          IN a sense we don't really want "Queued" anyway - it is an artifact of Jenkins internals that we don't always want to expose.

          What we may want to expose is that their build is in a bad way because of exahusted executors or mis configuration - that is really what it is about.

          Michael Neale added a comment - I think the missing piece is showing the activity of Jenkins in the not-yet-implemented GUI, this would make it clear... otherwise, if there was a way we could enquire as to what "parts" of a pipeline are doing... we could report more intelligently why it is waiting when it is in the RUNNING state... IN a sense we don't really want "Queued" anyway - it is an artifact of Jenkins internals that we don't always want to expose. What we may want to expose is that their build is in a bad way because of exahusted executors or mis configuration - that is really what it is about.

          Michael Neale added a comment -

          Summary: for pipeline this is actually "correct" behavior, as there is no queue on a per pipeline basis. Much more work is needed to show what parts of it are in a piepline and where it is blocked.

          Michael Neale added a comment - Summary: for pipeline this is actually "correct" behavior, as there is no queue on a per pipeline basis. Much more work is needed to show what parts of it are in a piepline and where it is blocked.

          James Dumay added a comment -

          cliffmeyers vivek if this requires extensive frontend work lets punt this to the next sprint

          James Dumay added a comment - cliffmeyers vivek if this requires extensive frontend work lets punt this to the next sprint

          Luis Farzati added a comment - - edited

          michaelneale I get your point about why "Queued" shouldn't be relevant, but I'm a bit lost on how would I be able to map a queue item to a run item then.

          When I do a POST to /runs, the HAL link points to a queue item, and while it's true that I also get a expectedBuildNumber, I guess this is not 100% guaranteed. Am I right? So my initial reaction was, ok no prob I query /runs and lookup by queue item id. But there's no evidence of the originating queue item.

          Am I missing something here?

          Luis Farzati added a comment - - edited michaelneale I get your point about why "Queued" shouldn't be relevant, but I'm a bit lost on how would I be able to map a queue item to a run item then. When I do a POST to /runs , the HAL link points to a queue item, and while it's true that I also get a expectedBuildNumber , I guess this is not 100% guaranteed. Am I right? So my initial reaction was, ok no prob I query /runs and lookup by queue item id. But there's no evidence of the originating queue item. Am I missing something here?

          Michael Neale added a comment -

          luisfarzati yes I think you are correct (perhaps cliffmeyers can confirm).

          There is a small window where there are things queued, and the expected number is what you say.

          Michael Neale added a comment - luisfarzati yes I think you are correct (perhaps cliffmeyers can confirm). There is a small window where there are things queued, and the expected number is what you say.

          Daniel Beck added a comment -

          I think JENKINS-12827 applies to this discussion.

          Build numbers don't get assigned to queued builds, and for good reason: Between QueueSorter, QueueTaskDispatcher, and plugins like Set Next Build Number, this is impossible to determine before actually starting the build. Then there's the issue of canceling queue items – they don't (and shouldn't) use up build numbers, which this change would necessitate (or result in wrong results, as a specific build number could be queued and canceled again multiple times).

          Daniel Beck added a comment - I think JENKINS-12827 applies to this discussion. Build numbers don't get assigned to queued builds, and for good reason: Between QueueSorter, QueueTaskDispatcher, and plugins like Set Next Build Number, this is impossible to determine before actually starting the build. Then there's the issue of canceling queue items – they don't (and shouldn't) use up build numbers, which this change would necessitate (or result in wrong results, as a specific build number could be queued and canceled again multiple times).

          Vivek Pandey added a comment - - edited

          luisfarzati No there is no guarantee for that build number to be the same, its for informational purpose to clients and must not be used in /runs API calls.

          Vivek Pandey added a comment - - edited luisfarzati No there is no guarantee for that build number to be the same, its for informational purpose to clients and must not be used in /runs API calls.

          Michael Neale added a comment -

          Right, so there really is no way to launch a build (request) and get a number back immediately. vivek so is there a way to take a queue item and "eventually" resolve it to a run number? (as per the question)

          Michael Neale added a comment - Right, so there really is no way to launch a build (request) and get a number back immediately. vivek so is there a way to take a queue item and "eventually" resolve it to a run number? (as per the question)

          Cliff Meyers added a comment - - edited

          vivek michaelneale Sorry to introduce a bit of a tangent but this issue is relevant to some of the changes we want to make in the UI for JENKINS-37425. When the user initiates a run, we want to immediately display a Toast with an "OPEN" link that will launch the Run Details UI. In order for Run Details to work as it's built today, we need the build number. For reasons listed above we can't always count on it, so I just wanted to outline a few scenarios to see what you guys think:

          • My understanding is that all pipelines basically begin running immediately, so could we assume that the build number returned for a pipeline is reliable? If not, what scenario would this not hold true in?
          • For freestyle jobs, my understanding is that there can be only one item in the queue at once. Similarly then could we assume that the build number would be "eventually correct" once the item in the queue actually moves to the run state?
          • I don't know enough about other job types (e.g. Matrix) to know if there would be problems there.

          Cliff Meyers added a comment - - edited vivek michaelneale Sorry to introduce a bit of a tangent but this issue is relevant to some of the changes we want to make in the UI for JENKINS-37425 . When the user initiates a run, we want to immediately display a Toast with an "OPEN" link that will launch the Run Details UI. In order for Run Details to work as it's built today, we need the build number. For reasons listed above we can't always count on it, so I just wanted to outline a few scenarios to see what you guys think: My understanding is that all pipelines basically begin running immediately, so could we assume that the build number returned for a pipeline is reliable? If not, what scenario would this not hold true in? For freestyle jobs, my understanding is that there can be only one item in the queue at once. Similarly then could we assume that the build number would be "eventually correct" once the item in the queue actually moves to the run state? I don't know enough about other job types (e.g. Matrix) to know if there would be problems there.

          Vivek Pandey added a comment -

          cliffmeyers:

          >My understanding is that all pipelines basically begin running immediately, so could we assume that the build number returned for a pipeline is reliable? If not, what scenario would this not hold true in?

          Generally yes, pipeline almost immediately get the run id. However in certain cases where you are throttling a pipeline build, it will end up in queue. For example try this pipeline script and click on build now few times and you will see it ends up in queue:

          node {
             echo 'Hello World'
             properties([[$class: 'JobPropertyImpl', throttle: [count: 1, durationName: 'hour']], pipelineTriggers([])])
          }
          

          >For freestyle jobs, my understanding is that there can be only one item in the queue at once.

          No, you can have any number of items in queue for the same pipeline jobs. Each with its own projected/estimated build id and queue id - these are queued items.

          >Similarly then could we assume that the build number would be "eventually correct" once the item in the queue actually moves to the run state?

          yes.

          >I don't know enough about other job types (e.g. Matrix) to know if there would be problems there.

          jobs resulting from matrix jobs could be either pipeline or regular jobs so same rules apply. we are not displaying them so we can hold off on them for now.

          Vivek Pandey added a comment - cliffmeyers : >My understanding is that all pipelines basically begin running immediately, so could we assume that the build number returned for a pipeline is reliable? If not, what scenario would this not hold true in? Generally yes, pipeline almost immediately get the run id. However in certain cases where you are throttling a pipeline build, it will end up in queue. For example try this pipeline script and click on build now few times and you will see it ends up in queue: node { echo 'Hello World' properties([[$class: 'JobPropertyImpl' , throttle: [count: 1, durationName: 'hour' ]], pipelineTriggers([])]) } >For freestyle jobs, my understanding is that there can be only one item in the queue at once. No, you can have any number of items in queue for the same pipeline jobs. Each with its own projected/estimated build id and queue id - these are queued items. >Similarly then could we assume that the build number would be "eventually correct" once the item in the queue actually moves to the run state? yes. >I don't know enough about other job types (e.g. Matrix) to know if there would be problems there. jobs resulting from matrix jobs could be either pipeline or regular jobs so same rules apply. we are not displaying them so we can hold off on them for now.

            vivek Vivek Pandey
            cliffmeyers Cliff Meyers
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: