Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68944

Matrix jobs do not start with custom label/expression for the built-in node

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • matrix-project-plugin
    • None
    • Jenkins 2.346.1
      matrix-project-plugin 771.v574584b_39e60 at least

      Matrix jobs only start when specifying exactly built-in as label to select the built-in node.

      When using a label expression like "built-in || master" or a different label that is configured for the built-in node, the matrix job will stay in the queue forever.

      This used to work fine with Jenkins 2.264.3.

      Note: the matrix jobs WILL start fine with a label or label expression that resolves e.g. to a docker cloud instead of the built-in node.

          [JENKINS-68944] Matrix jobs do not start with custom label/expression for the built-in node

          Volodja added a comment -

          FYI: A matrix parent job is handled as flyweight task. The labels of the built-in node are ignored during the check whether the flyweight task is bound to the built-in node. Only self-label will be respected. See https://github.com/jenkinsci/jenkins/blob/jenkins-2.346.1/core/src/main/java/hudson/model/Queue.java#L1740

          Volodja added a comment - FYI: A matrix parent job is handled as flyweight task. The labels of the built-in node are ignored during the check whether the flyweight task is bound to the built-in node. Only self-label will be respected. See https://github.com/jenkinsci/jenkins/blob/jenkins-2.346.1/core/src/main/java/hudson/model/Queue.java#L1740

          Mark Waite added a comment -

          I can't duplicate the problem as reported. I've been using built-in || master as a job label for quite a while with good results. Can you provide more details that will help others understand how you are seeing the issue?

          Steps I took to try to duplicate the issue:

          1. Run a Jenkins 2.346.1 controller with matrix project plugin 772.v494f19991984 as described in my lts-with-plugins branch
          2. Add agents to the controller, set the agent executor count on the controller to zero
          3. Define a matrix project job with the label built-in || master and confirm that it starts but does not complete so long as executor count on the controller is zero
          4. Change controller executor count to 1, confirm that the job runs on the controller

          Mark Waite added a comment - I can't duplicate the problem as reported. I've been using built-in || master as a job label for quite a while with good results. Can you provide more details that will help others understand how you are seeing the issue? Steps I took to try to duplicate the issue: Run a Jenkins 2.346.1 controller with matrix project plugin 772.v494f19991984 as described in my lts-with-plugins branch Add agents to the controller, set the agent executor count on the controller to zero Define a matrix project job with the label built-in || master and confirm that it starts but does not complete so long as executor count on the controller is zero Change controller executor count to 1, confirm that the job runs on the controller

          Volodja added a comment - - edited

          The problem occurs when you configure label built-in || master for matrix project (parent job) itself and not for axis.

          <?xml version="1.1" encoding="UTF-8" standalone="no"?>
          <matrix-project plugin="matrix-project@771.v574584b_39e60">
            ...
            <assignedNode>built-in || master</assignedNode>
            <canRoam>false</canRoam>
            ...
          </matrix-project>
          

          Volodja added a comment - - edited The problem occurs when you configure label built-in || master for matrix project (parent job) itself and not for axis. <?xml version= "1.1" encoding= "UTF-8" standalone= "no" ?> <matrix-project plugin= "matrix-project@771.v574584b_39e60" > ... <assignedNode> built-in || master </assignedNode> <canRoam> false </canRoam> ... </matrix-project>

          Mark Waite added a comment -

          Thanks for the clarification volodja. I thought that flyweight tasks were strictly limited to the controller. Isn't it creating an opportunity for confusion to allow a label on the "Restrict where this project can be run" setting of a matrix project? That doesn't control the location where the matrix jobs run, it only controls where the flyweight task runs, even though the flyweight task can only run on the controller.

          Mark Waite added a comment - Thanks for the clarification volodja . I thought that flyweight tasks were strictly limited to the controller. Isn't it creating an opportunity for confusion to allow a label on the "Restrict where this project can be run" setting of a matrix project? That doesn't control the location where the matrix jobs run, it only controls where the flyweight task runs, even though the flyweight task can only run on the controller.

          If that's the case, it appears sensible to me to remove that option altogether. We only set that option to ensure that the parent job runs on built-in.

          Carsten Pfeiffer added a comment - If that's the case, it appears sensible to me to remove that option altogether. We only set that option to ensure that the parent job runs on built-in.

          Volodja added a comment -

          Jenkins allows for "flyweight" tasks to run on a slave node, which does not occupy an executor and does not affect the slave node's lifecycle, so if Jenkins was executed some flyweight tasks on a container that was busy running a build, and then the build finishes, the container would be terminated without regard to the flyweight tasks.

          See https://github.com/jenkinsci/docker-plugin/issues/653#issuecomment-391649177

          This was the reason to force matrix project flyweight task to run on controller using explicitly configured master or built-in label for the job.
          Since PR #3983: Try to execute flyweight tasks on the master the probability that flyweigh task will be executed not on controller is very low but not excluded.

          Volodja added a comment - Jenkins allows for "flyweight" tasks to run on a slave node, which does not occupy an executor and does not affect the slave node's lifecycle, so if Jenkins was executed some flyweight tasks on a container that was busy running a build, and then the build finishes, the container would be terminated without regard to the flyweight tasks. See https://github.com/jenkinsci/docker-plugin/issues/653#issuecomment-391649177 This was the reason to force matrix project flyweight task to run on controller using explicitly configured master or built-in label for the job. Since PR #3983: Try to execute flyweight tasks on the master the probability that flyweigh task will be executed not on controller is very low but not excluded.

          Mark Waite added a comment - - edited

          Thanks for the education volodja. Much appreciated. Your comments validate that the issue as reported by capf is a real issue. Since flyweight execution is allowed on agents, it seems to me that the label expression should be evaluated. Work around is to use the exact label expression ({built-in}} or master

          Mark Waite added a comment - - edited Thanks for the education volodja . Much appreciated. Your comments validate that the issue as reported by capf is a real issue. Since flyweight execution is allowed on agents, it seems to me that the label expression should be evaluated. Work around is to use the exact label expression ({built-in}} or master

            Unassigned Unassigned
            capf Carsten Pfeiffer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: