Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-4873

Deadlock between backup plugin and matrix project build

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • other
    • None
    • Platform: All, OS: All

      Sequentially built matrix project can cause backup plugin to never start backup
      procedure and block hudson from building forever.

      This is happened when backup procedure is started between start of project build
      and start of configuration build. This is very likely to take place if matrix
      project codebase is big (so checkout and build takes a long time) and sequential
      compilation is enabled.

      Backup plugin sets shutdown mode so configuration are not started. After that
      plugin waits for running projects to be finished. But 'main' project waits for
      configurations before finish itself. So we have a deadlock.

          [JENKINS-4873] Deadlock between backup plugin and matrix project build

          eliterr added a comment -

          Created an attachment (id=1017)
          Set shutdown mode after wait for running tasks to be finished

          eliterr added a comment - Created an attachment (id=1017) Set shutdown mode after wait for running tasks to be finished

          eliterr added a comment -

          For our particular hudson installation we found it acceptable to delay backup
          until all the tasks are finished as well as all the tasks that are fired during
          this wait.
          We achieve this by enable shutdown mode after ensuring no tasks are running.
          Probably a deadlock is still possible but chances are much less.

          eliterr added a comment - For our particular hudson installation we found it acceptable to delay backup until all the tasks are finished as well as all the tasks that are fired during this wait. We achieve this by enable shutdown mode after ensuring no tasks are running. Probably a deadlock is still possible but chances are much less.

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/main/java/hudson/matrix/MatrixConfiguration.java
          trunk/hudson/main/core/src/main/java/hudson/model/LoadBalancer.java
          trunk/hudson/main/core/src/main/java/hudson/model/Queue.java
          trunk/hudson/main/test/src/test/groovy/hudson/matrix/MatrixProjectTest.groovy
          trunk/www/changelog.html
          http://fisheye4.cenqua.com/changelog/hudson/?cs=23980
          Log:
          [FIXED JENKINS-4873] in 1.336
          Matrix configuration builds should continue even when Hudson is about to shut down.
          (<a href="https://hudson.dev.java.net/issues/show_bug.cgi?id=4873">issue 4873</a>)

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/matrix/MatrixConfiguration.java trunk/hudson/main/core/src/main/java/hudson/model/LoadBalancer.java trunk/hudson/main/core/src/main/java/hudson/model/Queue.java trunk/hudson/main/test/src/test/groovy/hudson/matrix/MatrixProjectTest.groovy trunk/www/changelog.html http://fisheye4.cenqua.com/changelog/hudson/?cs=23980 Log: [FIXED JENKINS-4873] in 1.336 Matrix configuration builds should continue even when Hudson is about to shut down. (<a href="https://hudson.dev.java.net/issues/show_bug.cgi?id=4873">issue 4873</a>)

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/main/java/hudson/model/Queue.java
          http://fisheye4.cenqua.com/changelog/hudson/?cs=24492
          Log:
          JENKINS-4873 follow up change to rev.23980. Since flyweight support ignores the node availability, it bypasses the buildable state completely, which in turn means it now ignores other checks that buildable state did, namely to see if Hudson was quietting down.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/model/Queue.java http://fisheye4.cenqua.com/changelog/hudson/?cs=24492 Log: JENKINS-4873 follow up change to rev.23980. Since flyweight support ignores the node availability, it bypasses the buildable state completely, which in turn means it now ignores other checks that buildable state did, namely to see if Hudson was quietting down.

            Unassigned Unassigned
            eliterr eliterr
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: