Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-8929

Deadlock when upstream and downstream jobs are blocked on each other

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • None

      Job A (deploy to functional test) is upstream of job B (run functional tests).

      We need job A to block while its downstream job builds - otherwise a deployment while the functional tests are running will fail job B.

      We need job B to block while its upstream job builds - otherwise trying to run the tests while the deployment is running will fail.

      Unfortunately, if you configure A to block on downstream builds and B to block on upstream builds and A and B are scheduled at the same time they both block each other and are queued indefinitely, requiring manual intervention to clear the deadlock.

      In this situation job A should build and B should remain blocked until A completes.

          [JENKINS-8929] Deadlock when upstream and downstream jobs are blocked on each other

          Robert Elliot created issue -

          Robert Elliot added a comment -

          (In the example above A and B need to be separate jobs - if a change is made to a test only, only B should run, there is no need to redeploy the application. However, if the application changes it needs to be redeployed and the tests need to be run after that deploy.)

          Robert Elliot added a comment - (In the example above A and B need to be separate jobs - if a change is made to a test only, only B should run, there is no need to redeploy the application. However, if the application changes it needs to be redeployed and the tests need to be run after that deploy.)

          Robert Elliot added a comment -

          For reference, I found this thread describing the same problem with a possible solution:
          http://jenkins.361315.n4.nabble.com/Upstream-Downstream-Deadlock-td3307350.html

          Robert Elliot added a comment - For reference, I found this thread describing the same problem with a possible solution: http://jenkins.361315.n4.nabble.com/Upstream-Downstream-Deadlock-td3307350.html

          Robert Elliot added a comment -

          Fix made on my clone of jenkins and pull request made here:
          https://github.com/jenkinsci/jenkins/pull/69

          Robert Elliot added a comment - Fix made on my clone of jenkins and pull request made here: https://github.com/jenkinsci/jenkins/pull/69
          Robert Elliot made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          Robert Elliot made changes -
          Assignee New: Robert Elliot [ mahoney ]

          dogfood added a comment -

          Integrated in jenkins_main_trunk #565
          [FIXED JENKINS-8929] Changed the definition of a building downstream project to one that is Building, Waiting, Pending or Buildable
          JENKINS-8929 follow up changes
          recording JENKINS-8929 fix

          Kohsuke Kawaguchi : b40e7b4228a6a5979fa484b2d2b15259bfab0ff5
          Files :

          • core/src/main/java/hudson/model/AbstractProject.java
          • core/src/main/java/hudson/model/Queue.java

          Kohsuke Kawaguchi : 86d88e5e7b341c884b156185051945678ce81ae8
          Files :

          • core/src/main/java/hudson/model/AbstractProject.java
          • core/src/main/java/hudson/model/Queue.java

          Kohsuke Kawaguchi : 05b9727edf1089b1c5e6e9461b4f62c71790065e
          Files :

          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #565 [FIXED JENKINS-8929] Changed the definition of a building downstream project to one that is Building, Waiting, Pending or Buildable JENKINS-8929 follow up changes recording JENKINS-8929 fix Kohsuke Kawaguchi : b40e7b4228a6a5979fa484b2d2b15259bfab0ff5 Files : core/src/main/java/hudson/model/AbstractProject.java core/src/main/java/hudson/model/Queue.java Kohsuke Kawaguchi : 86d88e5e7b341c884b156185051945678ce81ae8 Files : core/src/main/java/hudson/model/AbstractProject.java core/src/main/java/hudson/model/Queue.java Kohsuke Kawaguchi : 05b9727edf1089b1c5e6e9461b4f62c71790065e Files : changelog.html

          Robert Elliot added a comment -

          Thanks - though I'm not totally convinced the change should be symmetric; I think there may be a case for blocking downstream builds when upstream builds are blocked, so that it is certain that if both are scheduled at the same time then it is definitely the upstream one that goes first.

          Robert Elliot added a comment - Thanks - though I'm not totally convinced the change should be symmetric; I think there may be a case for blocking downstream builds when upstream builds are blocked, so that it is certain that if both are scheduled at the same time then it is definitely the upstream one that goes first.

          Cees Bos added a comment -

          We upgrade to version 1.403, since we also faced this issue.
          But now the block does not work anymore in all situations.

          We see situations were this fails.
          If both upstream and downstream job are queue and the downstream project finishes, both project get started.

          Do I need to log a new issue for that?

          As soon as a upstream project is running the downstream gets waiting in the queue.

          Cees Bos added a comment - We upgrade to version 1.403, since we also faced this issue. But now the block does not work anymore in all situations. We see situations were this fails. If both upstream and downstream job are queue and the downstream project finishes, both project get started. Do I need to log a new issue for that? As soon as a upstream project is running the downstream gets waiting in the queue.

          Robert Elliot added a comment -

          My fault for not writing the test cases that would have shown Kohsuke why I did not make the change symmetrical. I wanted to but I haven't managed to get my head round the testing strategy for Jenkins yet.

          Robert Elliot added a comment - My fault for not writing the test cases that would have shown Kohsuke why I did not make the change symmetrical. I wanted to but I haven't managed to get my head round the testing strategy for Jenkins yet.

            mahoney Robert Elliot
            mahoney Robert Elliot
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: