Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-52529

node() step shall try to reuse node from last run

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • durable-task-plugin
    • None
    • Jenkins 1.121.1
      Pipeline Nodes and Processes 2.19

      Currently when running a simple pipeline like:

      node ('label') {
          echo 'Hello World'
      } 

      Each time you start the job the node used for the node block will be chosen randomly amongst all nodes supplyimg the 'label' label.

      This is very unfortunately because the involves lot of unneccessary compiles and cost a lot of time. In addition to that overall it'll need more disk space.

      So far I could not find any related feature request or bug which would cover this request. I tried to hack a workaround using a global shared library but so far I think this is simply not possible to do a stable approach using a global shared library.

      Request is to:

      • Store labels which had been in use by the last successfull job run.
      • For a ne node block, search in the list of stored label to build agent mappings for the build agent which was used in the last run
      • Try to reuse that very build agent. If there's no free executor shall build on another machine matchine the required abel expression elase.
        *

          [JENKINS-52529] node() step shall try to reuse node from last run

          Got it working locally. For it to work I changed the PlaceHolderTask.getFullDisplayName() method in order to not use the display name of the very run but the name of the run's parent job. That way the LoadBalancer would always pick the same node first. However to me it looks like the real issue here is that the load balancer is using the SubTask.getDisplayName() in the first place. In addition to that some tests in the durable-task-plugin are failing now.

          I wonder whether this was all implemented like that on purpose or whether there's some other user of PlaceHolderTask.getFullDisplayName() who actually will need the run's display rather than the jobs display name. But thinking about this I doub it.

          Think for me changing the LoadBalancers algorithm of calculating the hash would be too risky as I would not know what side effects this may cause. I'd propose to introduce a getHash()}}method in SubTask so both FreeStyle and Pipeline jobs can handle the calculation of the hash to their like. This method then would be called in {{LoadBalancer.assignGreedely instead of calling SubTask.getFullDisplayName().

          Maybe somebody could give me a hint on that one and point me into the right direction?

          Thanks.

          Joerg Schwaerzler added a comment - Got it working locally. For it to work I changed the  PlaceHolderTask.getFullDisplayName() method in order to not use the display name of the very run but the name of the run's parent job. That way the  LoadBalancer would always pick the same node first. However to me it looks like the real issue here is that the load balancer is using the SubTask.getDisplayName() in the first place. In addition to that some tests in the durable-task-plugin are failing now. I wonder whether this was all implemented like that on purpose or whether there's some other user of PlaceHolderTask.getFullDisplayName() who actually will need the run's display rather than the jobs display name. But thinking about this I doub it. Think for me changing the LoadBalancers algorithm of calculating the hash would be too risky as I would not know what side effects this may cause. I'd propose to introduce a getHash()}}method in SubTask so both FreeStyle and Pipeline jobs can handle the calculation of the hash to their like. This method then would be called in {{LoadBalancer.assignGreedely instead of calling SubTask.getFullDisplayName (). Maybe somebody could give me a hint on that one and point me into the right direction? Thanks.

          Looking into the source code made me think this is rather a bug that a feature.

           

          Joerg Schwaerzler added a comment - Looking into the source code made me think this is rather a bug that a feature.  

          Opened PR: https://github.com/jenkinsci/workflow-durable-task-step-plugin/pull/72

          Would be great to see this or some alternative solution included soon as the current implementation will almost always choose a different node - which is crucial especially with long-running builds in many-node environments where an incremental build would safe a lot of time.

           

          Joerg Schwaerzler added a comment - Opened PR: https://github.com/jenkinsci/workflow-durable-task-step-plugin/pull/72 Would be great to see this or some alternative solution included soon as the current implementation will almost always choose a different node - which is crucial especially with long-running builds in many-node environments where an incremental build would safe a lot of time.  

            Unassigned Unassigned
            macdrega Joerg Schwaerzler
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: