Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-16679

project run gets hung up, when run simultaneously

      We have a project (parent) with multiple steps, one of the step being configured as 'trigger/call a build on other projects' (i.e, child job)
      We have selected the option 'block until the triggered projects finish their builds', so that Parent project will be on hold until the child project gets completed.

      It works fine if we run the parent project one time.
      But, if we run another instance of the same Parent project, the child project run (triggered from first run), goes into 'pending forever' state.
      Similarly, the second Parent project run also goes to 'pending forever'

      Please let us know if there is any fix/workaround.

      ===========
      The projects are configured as:

      Parent Project:-

      -Step 1 (Some commands)

      -Step 2 (Invoke a “Child” project) – parent has to be in wait state until the child completes

      -Step 3 (Some commands)

      Child Project:-

      -Step 1 (Some Commands)

      ================

      Regards
      Satya

          [JENKINS-16679] project run gets hung up, when run simultaneously

          cjo9900 added a comment -

          There is a known issue if the child jobs have "block if upstream is running" enabled as they see the triggering job as running, which might be the issue that you are seeing.

          You should check, that there is a free executor for the child project to run on matching its labels, etc. If this is not the case, can you actually post the reasons that the pending jobs are not running (from the tooltip when hovering over the items in the Queue), as this will normally indicate the reason for the builds not running.
          i.e. waiting for an Executor on slave1 or Waiting for Up/Downstream project to finish

          Can you provide the following:
          Plugins(+versions) that are used in the parent and child jobs including the parameterized-trigger.

          and also attach the config.xml files to the issue for the two jobs. (suitably cleansed od sensitive data)

          And if you are are building on the slaves can you also indicate the master/slave config
          i.e. linux master with 0 executors, slave1 windows with 2 executors that these jobs are tied to.

          So that others can reproduce the issue and allow for it to be debugged.

          cjo9900 added a comment - There is a known issue if the child jobs have "block if upstream is running" enabled as they see the triggering job as running, which might be the issue that you are seeing. You should check, that there is a free executor for the child project to run on matching its labels, etc. If this is not the case, can you actually post the reasons that the pending jobs are not running (from the tooltip when hovering over the items in the Queue), as this will normally indicate the reason for the builds not running. i.e. waiting for an Executor on slave1 or Waiting for Up/Downstream project to finish Can you provide the following: Plugins(+versions) that are used in the parent and child jobs including the parameterized-trigger. and also attach the config.xml files to the issue for the two jobs. (suitably cleansed od sensitive data) And if you are are building on the slaves can you also indicate the master/slave config i.e. linux master with 0 executors, slave1 windows with 2 executors that these jobs are tied to. So that others can reproduce the issue and allow for it to be debugged.

          Oleg Nenashev added a comment -

          JENKINS-19776 seems to be a related issue, but it can be reproduced even without additional projects

          Oleg Nenashev added a comment - JENKINS-19776 seems to be a related issue, but it can be reproduced even without additional projects

          ikedam added a comment -

          No more reports for an year.

          ikedam added a comment - No more reports for an year.

          Oleg Nenashev added a comment - - edited

          The issue frequently appears in 2.21 (core version: 1.509.4)

          Reproduction steps:

          • Create a free-style job with parallel runs enabled
          • This job should trigger multiple runs using ParametersFactory and wait till theirs completion
          • The number of submissions should be greater than executors number

          After that, you just need to submit multiple instances of the free-style project. They should be executed in parallel, so you should configure nodes labeling, prioritization or something like that.

          Oleg Nenashev added a comment - - edited The issue frequently appears in 2.21 (core version: 1.509.4) Reproduction steps: Create a free-style job with parallel runs enabled This job should trigger multiple runs using ParametersFactory and wait till theirs completion The number of submissions should be greater than executors number After that, you just need to submit multiple instances of the free-style project. They should be executed in parallel, so you should configure nodes labeling, prioritization or something like that.

          ikedam added a comment -

          That sounds simply caused by executor shortage.
          Is it same to JENKINS-12290 ?

          ikedam added a comment - That sounds simply caused by executor shortage. Is it same to JENKINS-12290 ?

          Oleg Nenashev added a comment -

          The answer is no. JENKINS-12290 is related to installations with only one executor. In that case the triggered jobs goes to the queue and waits infinitely.

          In this case all submitted jobs will be queued and then executed. BTW, Future::get() hangs and waits for the project, which has been already finished. I suppose it may be caused by merging of submissions in the queue.

          Oleg Nenashev added a comment - The answer is no. JENKINS-12290 is related to installations with only one executor. In that case the triggered jobs goes to the queue and waits infinitely. In this case all submitted jobs will be queued and then executed. BTW, Future::get() hangs and waits for the project, which has been already finished. I suppose it may be caused by merging of submissions in the queue.

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          src/main/java/hudson/remoting/Request.java
          http://jenkins-ci.org/commit/remoting/e8ae81f301d11586726f3fdbbf3d14e6ac08b848
          Log:
          Use notifyAll() instead of notify() inside Request::onCompleted() handler

          FindBugs indicates it as a probable issue, which can be the origin of JENKINS-16679

          Signed-off-by: Oleg Nenashev <o.v.nenashev@gmail.com>

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: src/main/java/hudson/remoting/Request.java http://jenkins-ci.org/commit/remoting/e8ae81f301d11586726f3fdbbf3d14e6ac08b848 Log: Use notifyAll() instead of notify() inside Request::onCompleted() handler FindBugs indicates it as a probable issue, which can be the origin of JENKINS-16679 Signed-off-by: Oleg Nenashev <o.v.nenashev@gmail.com>

          ikedam added a comment -

          ikedam added a comment - This may be fixed in Jenkins-1.558, hudson-remoting 2.36. https://github.com/jenkinsci/remoting/commit/6d0d5dea26c5f619164ade791a26a5c7c8dc50ea https://github.com/jenkinsci/jenkins/commit/75447b57b655e38fdb48f1e854a4b287071342cf#diff-600376dffeb79835ede4a0b285078036 Please report if it reproduces after that version.

          ikedam added a comment -

          Looks fixed in Jenkins 1.558.

          ikedam added a comment - Looks fixed in Jenkins 1.558.

            Unassigned Unassigned
            satya_5078 Satya M
            Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: