Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49658

One JCloud node type can starve another in Jenkins

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open (View Workflow)
    • Priority: Major
    • Resolution: Unresolved
    • Component/s: jclouds-plugin
    • Labels:
      None
    • Environment:
      Jenkins 2.73.2
      JCloud 2.14
    • Similar Issues:

      Description

      Problem Statement

      On a given Jenkins instance, a desired cloud-node type (e.g. Ubuntu) may not spin up for hours because another cloud-node type (e.g. Win7) has saturated the cloud's quota. Due to node retention times, this can happen even if the existing cloud nodes are idle, resulting in wasted computing power!

      Steps to Reproduce
      • Start with an idle Jenkins and idle cloud quota of 10 nodes
      • Start 10 builds that require Win7 images
      • Allow each build to run and complete, e.g. 1 hour
      • Start 1 build that requires Ubuntu
      Expected Results
      • The Ubuntu build enters the build queue
      • An idle Win7 node should be killed and spin up with Ubuntu
      • Now the cloud quota of 10 is maxed at 9 idle Win7 nodes and 1 running Ubuntu node
      Actual Results
      • The Ubuntu build enters the build queue
      • All 10 Win7 nodes continue to stay idle
      • No Ubuntu node is spun up because the cloud quota is maxed out at 10
      • The retention time expires after (default) 30 minutes, and the Win7 nodes are killed
      • An Ubuntu image spins up and starts building the build requested 30 minutes ago

      We prefer a long retention time because we're using our private cloud instead of paying for usage from a 3rd party. We want cloud nodes to live a while so we can avoid spin-up time when reused. But we don't want that behavior at the expense of queuing other builds while cloud nodes are idle.

        Attachments

          Activity

          darrelvun Darrel Vuncannon created issue -
          darrelvun Darrel Vuncannon made changes -
          Field Original Value New Value
          Description h5. Problem Statement
          On a given Jenkins instance, a desired cloud-node type (e.g. Ubuntu) may not spin up for hours because another image type (e.g. Win7) has saturated the cloud's quota. {color:darkred}_Due to node retention times, this can happen even if the existing cloud nodes are idle, resulting in wasted computing power!_{color}

          h5. Steps to Reproduce
          * Start with an idle Jenkins and idle cloud quota of 10 nodes
          * Start 10 builds that require Win7 images
          * Allow each build to run and complete, e.g. 1 hour
          * Start 1 build that requires Ubuntu

          h5. Expected Results
          * The Ubuntu build enters the build queue
          * An idle Win7 node should be killed and spin up with Ubuntu
          * Now the cloud quota of 10 is maxed at 9 idle Win7 nodes and 1 running Ubuntu node

          h5. Actual Results
          * The Ubuntu build enters the build queue
          * All 10 Win7 nodes continue to stay idle
          * No Ubuntu node is spun up because the cloud quota is maxed out at 10
          * The retention time expires after (default) 30 minutes, and the Win7 nodes are killed
          * An Ubuntu image spins up and starts building the build requested 30 minutes ago

          We prefer a long retention time because we're using our private cloud instead of paying for usage from a 3rd party. We want cloud nodes to live a while so we can avoid spin-up time when reused. But we don't want that behavior at the expense of queuing other builds.
          h5. Problem Statement
          On a given Jenkins instance, a desired cloud-node type (e.g. Ubuntu) may not spin up for hours because another cloud-node type (e.g. Win7) has saturated the cloud's quota. {color:darkred}_Due to node retention times, this can happen even if the existing cloud nodes are idle, resulting in wasted computing power!_{color}

          h5. Steps to Reproduce
          * Start with an idle Jenkins and idle cloud quota of 10 nodes
          * Start 10 builds that require Win7 images
          * Allow each build to run and complete, e.g. 1 hour
          * Start 1 build that requires Ubuntu

          h5. Expected Results
          * The Ubuntu build enters the build queue
          * An idle Win7 node should be killed and spin up with Ubuntu
          * Now the cloud quota of 10 is maxed at 9 idle Win7 nodes and 1 running Ubuntu node

          h5. Actual Results
          * The Ubuntu build enters the build queue
          * All 10 Win7 nodes continue to stay idle
          * No Ubuntu node is spun up because the cloud quota is maxed out at 10
          * The retention time expires after (default) 30 minutes, and the Win7 nodes are killed
          * An Ubuntu image spins up and starts building the build requested 30 minutes ago

          We prefer a long retention time because we're using our private cloud instead of paying for usage from a 3rd party. We want cloud nodes to live a while so we can avoid spin-up time when reused. But we don't want that behavior at the expense of queuing other builds while cloud nodes are idle.
          darrelvun Darrel Vuncannon made changes -
          Summary One JCloud image type can starve another in Jenkins One JCloud node type can starve another in Jenkins

            People

            Assignee:
            felfert Fritz Elfert
            Reporter:
            darrelvun Darrel Vuncannon
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: