Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7291

Flyweight jobs and zero executors

    XMLWordPrintable

Details

    Description

      In a cloud-enabled environment where Hudson has zero static executors, there will be no Computer object until a node is provisioned, so even a fly-weight job would wait for an allocation of a node.

      (And IIRC, I also saw that the flyweight job still used up a regular executor.)

      Needs more analysis.

      Attachments

        Issue Links

          Activity

            Code changed in jenkins
            User: Jesse Glick
            Path:
            test/src/test/java/hudson/model/QueueTest.java
            http://jenkins-ci.org/commit/jenkins/4089c79a68cf56ae57b79431bd795e5d5e5f4e23
            Log:
            JENKINS-7291 Accidental Java 7 dependency in test.
            (cherry picked from commit cedf17e00e1dc60ac4c8f0561e7975b146c9ffd8)

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: test/src/test/java/hudson/model/QueueTest.java http://jenkins-ci.org/commit/jenkins/4089c79a68cf56ae57b79431bd795e5d5e5f4e23 Log: JENKINS-7291 Accidental Java 7 dependency in test. (cherry picked from commit cedf17e00e1dc60ac4c8f0561e7975b146c9ffd8)

            Code changed in jenkins
            User: Kohsuke Kawaguchi
            Path:
            core/src/main/java/hudson/model/AbstractCIBase.java
            core/src/main/java/hudson/model/Computer.java
            core/src/main/java/hudson/model/Queue.java
            core/src/main/java/hudson/slaves/SlaveComputer.java
            core/src/main/java/jenkins/model/Jenkins.java
            core/src/main/resources/lib/hudson/executors.jelly
            test/src/test/java/hudson/model/QueueTest.java
            http://jenkins-ci.org/commit/jenkins/c5f2d2df58c7781c8003e11b6d389352f4e39adc
            Log:
            [FIXED JENKINS-7291] Permit flyweight tasks to run on master even when it has zero configured executors.

            Always adding Computer for master as a fallback

            The original proposed fix for JENKINS-7291 creates a Computer object
            transitively. This seems unwise as it violates the design of Computer
            as stated in the javadoc, and for example we can end up creating two
            Computers for the master.

            I think a better fix is to create a Computer for the master all the
            time, even if there's no executors configured. The discrimination in
            Queue.makeBuildable would ensure that such phantom Computer is only used
            as a last resort (statistically speaking).

            I've also tweaked executors.jelly a bit. I simplified it somewhat
            based on the idea that "if there's only one computer to show, the
            context is likely making it obvious".

            (I must be missing the intricacy in the current code.)

            Originally developed in a branch at
            2c5b57fcc1f39ed39057254e802f4183db5aa0dc then squashed for clarity.

            (cherry picked from commit a114c693bc61c564232e78c9fffb5de1ef946ea8)

            Conflicts:
            changelog.html

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/model/AbstractCIBase.java core/src/main/java/hudson/model/Computer.java core/src/main/java/hudson/model/Queue.java core/src/main/java/hudson/slaves/SlaveComputer.java core/src/main/java/jenkins/model/Jenkins.java core/src/main/resources/lib/hudson/executors.jelly test/src/test/java/hudson/model/QueueTest.java http://jenkins-ci.org/commit/jenkins/c5f2d2df58c7781c8003e11b6d389352f4e39adc Log: [FIXED JENKINS-7291] Permit flyweight tasks to run on master even when it has zero configured executors. Always adding Computer for master as a fallback The original proposed fix for JENKINS-7291 creates a Computer object transitively. This seems unwise as it violates the design of Computer as stated in the javadoc, and for example we can end up creating two Computers for the master. I think a better fix is to create a Computer for the master all the time, even if there's no executors configured. The discrimination in Queue.makeBuildable would ensure that such phantom Computer is only used as a last resort (statistically speaking). I've also tweaked executors.jelly a bit. I simplified it somewhat based on the idea that "if there's only one computer to show, the context is likely making it obvious". (I must be missing the intricacy in the current code.) Originally developed in a branch at 2c5b57fcc1f39ed39057254e802f4183db5aa0dc then squashed for clarity. (cherry picked from commit a114c693bc61c564232e78c9fffb5de1ef946ea8) Conflicts: changelog.html
            recampbell Ryan Campbell added a comment -

            Sorry, my mistake in backporting.

            recampbell Ryan Campbell added a comment - Sorry, my mistake in backporting.
            jglick Jesse Glick added a comment - - edited

            @recampbell: you backported https://github.com/jenkinsci/jenkins/commit/a114c693bc61c564232e78c9fffb5de1ef946ea8, or are testing a 1.510+ build with the original fix, and still see the issue?

            And did you verify that it is the matrix run which waits for, and runs on, the cloud slave instead of master, as opposed to the matrix configuration run which naturally has to run on the cloud slave? (Does testFlyweightTasksWithoutMasterExecutors pass?)

            jglick Jesse Glick added a comment - - edited @recampbell: you backported https://github.com/jenkinsci/jenkins/commit/a114c693bc61c564232e78c9fffb5de1ef946ea8 , or are testing a 1.510+ build with the original fix, and still see the issue? And did you verify that it is the matrix run which waits for, and runs on, the cloud slave instead of master, as opposed to the matrix configuration run which naturally has to run on the cloud slave? (Does testFlyweightTasksWithoutMasterExecutors pass?)
            recampbell Ryan Campbell added a comment -

            This doesn't resolve the issue for me. I see the master computer, but it is not used for the main matrix job. Instead, a cloud slave is provisioned. Steps to repro:

            1. Create a matrix job with a single configuration, no labels
            2. Ensure that Jenkins.numExecutors = 0
            3. Ensure that a cloud plugin can provision slaves
            4. Run the matrix job

            The matrix job won't run on the master, but will result in a slave being provisioned.

            recampbell Ryan Campbell added a comment - This doesn't resolve the issue for me. I see the master computer, but it is not used for the main matrix job. Instead, a cloud slave is provisioned. Steps to repro: 1. Create a matrix job with a single configuration, no labels 2. Ensure that Jenkins.numExecutors = 0 3. Ensure that a cloud plugin can provision slaves 4. Run the matrix job The matrix job won't run on the master, but will result in a slave being provisioned.

            People

              jglick Jesse Glick
              kohsuke Kohsuke Kawaguchi
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: