We use a pipeline of 40+ VMs to run our tests. Our goal is to find the total time it takes for all the VMs to run all of our tests - this is all available through the flowGraphTable. But the flowGraphTable is not easily available to obtain through the API, so we use api/json?depth=2 to find all of the nodes that have ran in each build. We use their link and add the endpoint "wfapi/describe" (code below) to retrieve the time it took for that node to run. However, there are two major issues that arise by using that workaround

       

      {
      "_links":{
           "self":{
                "href":"http://<jenkins_url>/job/<jenkins_server_info>/wfapi/describe"
           }
      },
      "id":"",
      "name":"Build",
      "status":"FAILED",
      "startTimeMillis":1000000,
      "endTimeMillis":2000000,
      "durationMillis":51,
      "queueDurationMillis":5,
      "pauseDurationMillis":0,
      "stages":[{
           "_links":{
                "self":{
                     "href":""
                }
           },
           "id":"5",
           "name":"Resource Check",
           "execNode":"",
           "status":"SUCCESS",
           "startTimeMillis":10000,
           "durationMillis":508,
           "pauseDurationMillis":0
      }
      

       

       

      1. It takes a few minutes to parse through all the nodes and make each request
      2. Many nodes lead to a JSON file that is null - no data is part of the response

      The 2nd point is bold as that is our major roadblock. The nodes that should be providing us the most useful information are giving us a response of null.

      We have worked with adding the endpoint "wfapi/describe" after the build number. We iterate through all of the stages and go through their individual responses. The issue here, however, is that only first 100 tests, including their time, are shown - everything else is not there. For us, this is the most valuable information as it allows us to see how long each VM took to run tests. 

      I am proposing three new added keys and values to the endpoint "api/json?depth=2" as shown below:

       

      {
      "_class":"org.jenkinsci.plugins.workflow.cps.nodes.StepStartNode",
      "actions":[{},{},{}],
      "displayName":"",
      "iconColor":"",
      "id":"",
      "parents":[""],
      "running":false,
      "durationMillis":0,
      "startTimeMillis":100000,
      "queueDurationMillis":0,
      "pauseDurationMillis":0,
      "url":""
      }
      

       

      1. "durationMillis" is the time it took for the rest to run - in milliseconds
      2. "startTimeMillis" is the time that process started - in UNIX milliseconds
      3. "queueDurationMillis" is the time the process was waiting in queue before being ran - in milliseconds
      4. "pauseDurationMillis" is the time that the process paused for before running again - in milliseconds

       

          [JENKINS-52394] api/json?depth=2 - Duration not available

          Oleg Nenashev added a comment -

          > api/json?depth=2 to find all of the nodes that have ran in each build

          Well, this core's logic will not work for Pipeline now. Generally there should be some node tracking API, e.g. as requested in JENKINS-32225 and JENKINS-33995

          So, routing to the Pipeline team

           

          Oleg Nenashev added a comment - > api/json?depth=2 to find all of the nodes that have ran in each build Well, this core's logic will not work for Pipeline now. Generally there should be some node tracking API, e.g. as requested in  JENKINS-32225 and  JENKINS-33995 So, routing to the Pipeline team  

          Sam Van Oort added a comment -

          > Our goal is to find the total time it takes for all the VMs to run all of our tests - this is all available through the flowGraphTable

          Yep, I added that last year - glad it's handy.

          Anyway, the wfapi API described here is provided by the Pipeline Stage View plugin which isn't getting a lot development priority at the moment.  The depth parameter isn't – and won't – be supported, because it's based on the standard Jenkins API mechanism. 

          However, what I do hear here is "we need a way to get detailed information about node uses" and I think that's a reasonable request. 

          Sam Van Oort added a comment - > Our goal is to find the total time it takes for all the VMs to run all of our tests - this is all available through the flowGraphTable Yep, I added that last year - glad it's handy. Anyway, the wfapi API described here is provided by the Pipeline Stage View plugin which isn't getting a lot development priority at the moment.  The depth parameter isn't – and won't – be supported, because it's based on the standard Jenkins API mechanism.  However, what I do hear here is "we need a way to get detailed information about node uses" and I think that's a reasonable request. 

          Sam Van Oort added a comment -

          What's being asked here won't be done – but I think there's a really good argument for JENKINS-32225 in a more general form (provide an API to get node usage information out of Pipeline more directly).

          nateshkukreja What I'd appreciate if you could do is add a short comment to that JIRA that describes the information you'd want out of a Pipeline about nodes and how you'd use it.

          Sam Van Oort added a comment - What's being asked here won't be done – but I think there's a really good argument for JENKINS-32225 in a more general form (provide an API to get node usage information out of Pipeline more directly). nateshkukreja What I'd appreciate if you could do is add a short comment to that JIRA that describes the information you'd want out of a Pipeline about nodes and how you'd use it.

          For sure, thanks!

          Natesh Kukreja added a comment - For sure, thanks!

            svanoort Sam Van Oort
            nateshkukreja Natesh Kukreja
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: