Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39975

Possibility to specify number of executors a node step will consume

XMLWordPrintable

      Possibility to specify number of executors a node step will consume. The syntax below is probably not correct but I hope it will get the general idea across.

      // List of tuples with (taskName, executorCount)
def heavyTasks = [('task1', 2), ('task2', 3), ('task3', 8)]
def tasks = [:]
for (x in heavyTasks) {
    def task = x
    tasks[task] = {
        node executors: task[1] {
            sh 'heavyThreadedTaskGoesHere'
            }
    }
}
stage("RunTasks") {
    parallel tasks
}

      EDIT: There is a plugin that tries to do this but there is no pipeline support. https://wiki.jenkins-ci.org/display/JENKINS/Heavy+Job+Plugin

          [JENKINS-39975] Possibility to specify number of executors a node step will consume

          Robin Björklin created issue -
          Robin Björklin made changes -
          Description Original: Possibility to specify number of executors a node step will consume. The syntax below is probably not correct but I hope it will get the general idea across.

          {code:groovy}
          // List of tuples with (taskName, executorCount)
          def heavyTasks = [('task1', 2), ('task2', 3), ('task3', 8)]
          def tasks = [:]
          for (x in heavyTasks) {
              def task = x
              tasks[task] = {
                  node executors: task[1] {
                      sh 'heavyThreadedTaskGoesHere'
                      }
              }
          }
          stage("RunTasks") {
              parallel tasks
          }
          {code}           		New: Possibility to specify number of executors a node step will consume. The syntax below is probably not correct but I hope it will get the general idea across.

          {code:java}
          // List of tuples with (taskName, executorCount)
          def heavyTasks = [('task1', 2), ('task2', 3), ('task3', 8)]
          def tasks = [:]
          for (x in heavyTasks) {
              def task = x
              tasks[task] = {
                  node executors: task[1] {
                      sh 'heavyThreadedTaskGoesHere'
                      }
              }
          }
          stage("RunTasks") {
              parallel tasks
          }
          {code}
          Robin Björklin made changes -
          Description Original: Possibility to specify number of executors a node step will consume. The syntax below is probably not correct but I hope it will get the general idea across.

          {code:java}
          // List of tuples with (taskName, executorCount)
          def heavyTasks = [('task1', 2), ('task2', 3), ('task3', 8)]
          def tasks = [:]
          for (x in heavyTasks) {
              def task = x
              tasks[task] = {
                  node executors: task[1] {
                      sh 'heavyThreadedTaskGoesHere'
                      }
              }
          }
          stage("RunTasks") {
              parallel tasks
          }
          {code}           		New: Possibility to specify number of executors a node step will consume. The syntax below is probably not correct but I hope it will get the general idea across.

          {code:java}
          // List of tuples with (taskName, executorCount)
          def heavyTasks = [('task1', 2), ('task2', 3), ('task3', 8)]
          def tasks = [:]
          for (x in heavyTasks) {
              def task = x
              tasks[task] = {
                  node executors: task[1] {
                      sh 'heavyThreadedTaskGoesHere'
                      }
              }
          }
          stage("RunTasks") {
              parallel tasks
          }
          {code}

          EDIT: There is a plugin that tries to do this but there is no pipeline support. https://wiki.jenkins-ci.org/display/JENKINS/Heavy+Job+Plugin

          Jesse Glick added a comment - 
          node {
  node(NODE_NAME) {
    // …
  }
}

          would do the trick for n=2, though this is probably deadlock-prone—not sure how the plugin handles that.

          Jesse Glick added a comment - node { node(NODE_NAME) { // … } } would do the trick for n=2, though this is probably deadlock-prone—not sure how the plugin handles that.
          Jesse Glick made changes -
          Component/s New: heavy-job-plugin [ 16006 ]
          Component/s Original: workflow-durable-task-step-plugin [ 21715 ]
          Labels New: pipeline
          Jesse Glick made changes -
          Link New: This issue duplicates JENKINS-41940 [ JENKINS-41940 ]
          Jesse Glick made changes -
          Resolution New: Duplicate [ 3 ]
          Status Original: Open [ 1 ] New: Resolved [ 5 ]

            Unassigned Unassigned
            rbjorklin Robin Björklin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: