Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24095

When node is active, reducing number of executors does not reflect reality

    XMLWordPrintable

Details

    • Bug
    • Status: Resolved (View Workflow)
    • Minor
    • Resolution: Fixed
    • core
    • None

    Description

      I have a node with 3 executors.
      There is currently a job running against that node.
      I go to the configuration page for that node and change the number of executors to 1.
      The node now has 2 executors, instead of 1.

      If I do the same procedure on an idle node, it reduces the number of executors to 1 correctly.

      Attachments

        Issue Links

          links to

          Web Link PR 1353

          Activity

            Ascending order - Click to sort in descending order
            View 4 older comments
            inquisitivecrow Marco Zanchi added a comment -

            Executor doing the current job is 1, UI (1-based).

            I could not run your script, so just did a dump of the node:

            <?xml version="1.0" encoding="UTF-8"?>
            <slave>
            <name>Automated Upgrade</name>
            <description>Buka's computer - UKDT1800</description>
            <remoteFS>c:\JenkinsSlave</remoteFS>
            <numExecutors>1</numExecutors>
            <mode>EXCLUSIVE</mode>
            <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/>
            <launcher class="hudson.slaves.JNLPLauncher"/>
            <label></label>
            <nodeProperties>
            <hudson.tools.ToolLocationNodeProperty>
            <locations>
            <hudson.tools.ToolLocationNodeProperty_-ToolLocation>
            <type>hudson.tasks.Ant$AntInstallation$DescriptorImpl</type>
            <name>ANT1.7.1</name>
            <home>C:\apache-ant-1.7.1</home>
            </hudson.tools.ToolLocationNodeProperty_-ToolLocation>
            <hudson.tools.ToolLocationNodeProperty_-ToolLocation>
            <type>hudson.tasks.Ant$AntInstallation$DescriptorImpl</type>
            <name>ANT1.7</name>
            <home>C:\jdk1.7.0_25</home>
            </hudson.tools.ToolLocationNodeProperty_-ToolLocation>
            </locations>
            </hudson.tools.ToolLocationNodeProperty>
            </nodeProperties>
            <userId>anonymous</userId>
            </slave>

            Seems to be just a cosmetic issue then. Shall we downgrade this to trivial?

            inquisitivecrow Marco Zanchi added a comment - Executor doing the current job is 1, UI (1-based). I could not run your script, so just did a dump of the node: <?xml version="1.0" encoding="UTF-8"?> <slave> <name>Automated Upgrade</name> <description>Buka's computer - UKDT1800</description> <remoteFS>c:\JenkinsSlave</remoteFS> <numExecutors>1</numExecutors> <mode>EXCLUSIVE</mode> <retentionStrategy class="hudson.slaves.RetentionStrategy$Always"/> <launcher class="hudson.slaves.JNLPLauncher"/> <label></label> <nodeProperties> <hudson.tools.ToolLocationNodeProperty> <locations> <hudson.tools.ToolLocationNodeProperty_-ToolLocation> <type>hudson.tasks.Ant$AntInstallation$DescriptorImpl</type> <name>ANT1.7.1</name> <home>C:\apache-ant-1.7.1</home> </hudson.tools.ToolLocationNodeProperty_-ToolLocation> <hudson.tools.ToolLocationNodeProperty_-ToolLocation> <type>hudson.tasks.Ant$AntInstallation$DescriptorImpl</type> <name>ANT1.7</name> <home>C:\jdk1.7.0_25</home> </hudson.tools.ToolLocationNodeProperty_-ToolLocation> </locations> </hudson.tools.ToolLocationNodeProperty> </nodeProperties> <userId>anonymous</userId> </slave> Seems to be just a cosmetic issue then. Shall we downgrade this to trivial?
            danielbeck Daniel Beck added a comment - - edited

            Actually it's both not cosmetic (the node will accept a second task) and not a bug (documented to behave this way).

            Two issues:

            • Your API doesn't show anything relevant. The second executor will be removed once the task is finished, numExecutors is already 1 before.
            • I've been unable to reproduce the issue if the task executes on the first executor 'executor #0' (see also /threadDump), only if it's a higher index. Hence the recent questions.

            To clarify, what happens is that every executor has an index from 0...N-1 with N being numExecutors. When reducing the number to M, Jenkins fills up executors 0...M-1, killing all idle ones, and leaving building ones alone. On a five executor node (0...4) with #3 building, setting numExecutors to 2 will result in #0, #1, and #3 existing at first, and #3 removed once the task is finished. This is not reflected on the UI, those indexes are different from the internal ones.

            I'm investigating whether the number of busy executors could be considered when reducing, e.g. reusing the example above, set numExecutors to 2 will result in #0 and #3, and once #3 is finished it'll be killed but a new #1 created.

            danielbeck Daniel Beck added a comment - - edited Actually it's both not cosmetic (the node will accept a second task) and not a bug (documented to behave this way). Two issues: Your API doesn't show anything relevant. The second executor will be removed once the task is finished, numExecutors is already 1 before. I've been unable to reproduce the issue if the task executes on the first executor 'executor #0' (see also /threadDump), only if it's a higher index. Hence the recent questions. To clarify, what happens is that every executor has an index from 0...N-1 with N being numExecutors. When reducing the number to M, Jenkins fills up executors 0...M-1, killing all idle ones, and leaving building ones alone. On a five executor node (0...4) with #3 building, setting numExecutors to 2 will result in #0, #1, and #3 existing at first, and #3 removed once the task is finished. This is not reflected on the UI, those indexes are different from the internal ones. I'm investigating whether the number of busy executors could be considered when reducing, e.g. reusing the example above, set numExecutors to 2 will result in #0 and #3, and once #3 is finished it'll be killed but a new #1 created.
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Daniel Beck
            Path:
            core/src/main/java/hudson/model/Computer.java
            http://jenkins-ci.org/commit/jenkins/05a26da704b9fce80527f5dd6fc32bf57bb2089c
            Log:
            [FIXED JENKINS-24095] Ensure only numExecutors exist when reducing

            When reducing the number of executors from M to N, previously it'd
            kill all idle executors, and then create executors 0...N-1 unless
            they already exist (busy executors).

            This can lead to the situation where e.g. executor #5 is busy and
            retained, and when setting the number of executors to 3, 0...2 are
            created, resulting temporarily having 4 executors.

            This change prevents #2 from being created in the example. When #4
            is finished, it'll be killed anyway and a new executor #2 created,
            so after some time the executor indexes are cleaned up matching
            expectations.

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/hudson/model/Computer.java http://jenkins-ci.org/commit/jenkins/05a26da704b9fce80527f5dd6fc32bf57bb2089c Log: [FIXED JENKINS-24095] Ensure only numExecutors exist when reducing When reducing the number of executors from M to N, previously it'd kill all idle executors, and then create executors 0...N-1 unless they already exist (busy executors). This can lead to the situation where e.g. executor #5 is busy and retained, and when setting the number of executors to 3, 0...2 are created, resulting temporarily having 4 executors. This change prevents #2 from being created in the example. When #4 is finished, it'll be killed anyway and a new executor #2 created, so after some time the executor indexes are cleaned up matching expectations.
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Daniel Beck
            Path:
            core/src/main/java/hudson/model/Computer.java
            http://jenkins-ci.org/commit/jenkins/7fb4591bdcc179f6eddad34175f2e2afe52788aa
            Log:
            Merge pull request #1353 from daniel-beck/JENKINS-24095

            [FIXED JENKINS-24095] Ensure only numExecutors exist when reducing

            Compare: https://github.com/jenkinsci/jenkins/compare/2ea7e628c739...7fb4591bdcc1

            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Beck Path: core/src/main/java/hudson/model/Computer.java http://jenkins-ci.org/commit/jenkins/7fb4591bdcc179f6eddad34175f2e2afe52788aa Log: Merge pull request #1353 from daniel-beck/ JENKINS-24095 [FIXED JENKINS-24095] Ensure only numExecutors exist when reducing Compare: https://github.com/jenkinsci/jenkins/compare/2ea7e628c739...7fb4591bdcc1
            dogfood dogfood added a comment -

            Integrated in jenkins_main_trunk #3618
            [FIXED JENKINS-24095] Ensure only numExecutors exist when reducing (Revision 05a26da704b9fce80527f5dd6fc32bf57bb2089c)

            Result = SUCCESS
            daniel-beck : 05a26da704b9fce80527f5dd6fc32bf57bb2089c
            Files :

            • core/src/main/java/hudson/model/Computer.java
            dogfood dogfood added a comment - Integrated in jenkins_main_trunk #3618 [FIXED JENKINS-24095] Ensure only numExecutors exist when reducing (Revision 05a26da704b9fce80527f5dd6fc32bf57bb2089c) Result = SUCCESS daniel-beck : 05a26da704b9fce80527f5dd6fc32bf57bb2089c Files : core/src/main/java/hudson/model/Computer.java

            People

              danielbeck Daniel Beck
              inquisitivecrow Marco Zanchi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: