Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-2725

locks and latches shouldnt' be global in multi-config

    • Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Icon: Major Major
    • None
    • Platform: All, OS: All

      The locks and latches plugin stops all instances across a grid from running the
      same build. This is not useful in a multi-config setup where you want to run a
      single build across many machines/os's etc. The default should be to obtain a
      lock per config or slave perhaps with an option to make it global.

          [JENKINS-2725] locks and latches shouldnt' be global in multi-config

          It is not true that this functionality is not useful... Here's a use-case:

          • you might want to run all the builds, but have a shared database and not want
            the integration tests fighting in the shared database.

          However, this is more correctly an enhancement request... probably for a new
          type of lock

          Stephen Connolly added a comment - It is not true that this functionality is not useful... Here's a use-case: you might want to run all the builds, but have a shared database and not want the integration tests fighting in the shared database. However, this is more correctly an enhancement request... probably for a new type of lock

          akostadinov added a comment -

          Yes, the use case described above is very important in our setup so please don't remove it. Having locks per the matrix configuration would be ideal complement for maximum flexibility. Doing the UI right is hard though. I was thinking about a text box where one can set locks per configuration with groovy expressions.

          akostadinov added a comment - Yes, the use case described above is very important in our setup so please don't remove it. Having locks per the matrix configuration would be ideal complement for maximum flexibility. Doing the UI right is hard though. I was thinking about a text box where one can set locks per configuration with groovy expressions.

          emmulator added a comment -

          An expression to lock various Configurations might be good for some cases (I don't have examples), but it would be overkill for my needs and may not even meet my needs or my understanding of brianf's original request. What I need, and what I think the point of this ticket is, is to have the lock apply only to the parent Matrix Job – i.e. to the Matrix as a set, not to the individual configurations at all. That is, if I assign a lock to two Matrix Jobs, then when either of these Jobs runs, all of its Configurations should still run concurrently, but the other Job would not be allowed to start until all of the Configurations of the first one had finished. Then the second one can start up and run all of its Configurations concurrently. I want to distinguish sets of Jobs, each defined by a Matrix Job, and exclude those sets from running concurrently with other sets, but still allow all of the Jobs (Configurations) within a set to run concurrently.

          It seems to me all that would be needed for this is a simple checkbox that, when assigning a Lock to a Matrix Job, indicates whether the Lock applies to the parent or the children. Then, if there is a need, that could be enhanced to allow some process of choosing to which children it applies. But I'm not sure whether anyone has expressed a need for that yet.

          emmulator added a comment - An expression to lock various Configurations might be good for some cases (I don't have examples), but it would be overkill for my needs and may not even meet my needs or my understanding of brianf's original request. What I need, and what I think the point of this ticket is, is to have the lock apply only to the parent Matrix Job – i.e. to the Matrix as a set, not to the individual configurations at all. That is, if I assign a lock to two Matrix Jobs, then when either of these Jobs runs, all of its Configurations should still run concurrently, but the other Job would not be allowed to start until all of the Configurations of the first one had finished. Then the second one can start up and run all of its Configurations concurrently. I want to distinguish sets of Jobs, each defined by a Matrix Job, and exclude those sets from running concurrently with other sets, but still allow all of the Jobs (Configurations) within a set to run concurrently. It seems to me all that would be needed for this is a simple checkbox that, when assigning a Lock to a Matrix Job, indicates whether the Lock applies to the parent or the children. Then, if there is a need, that could be enhanced to allow some process of choosing to which children it applies. But I'm not sure whether anyone has expressed a need for that yet.

          mdonohue added a comment -

          What if locks and latches did variable expansion in the lock name? Then you could set the lock to 'matrixlock-%

          {jobname}

          ' or 'matrixlock-%

          {nodename}

          ' or some such. If you only want one matrix job on a node at a time, the latter would do the trick.

          mdonohue added a comment - What if locks and latches did variable expansion in the lock name? Then you could set the lock to 'matrixlock-% {jobname} ' or 'matrixlock-% {nodename} ' or some such. If you only want one matrix job on a node at a time, the latter would do the trick.

          emmulator added a comment -

          Yes, matrixlock-${NODE_NAME} would work for me.

          emmulator added a comment - Yes, matrixlock-${NODE_NAME} would work for me.

          abloom added a comment -

          Regarding locking each configuration separately, isn't that functionality already implemented for Matrix builds via the "Run each configuration sequentially" checkbox? Since that functionality is already supported it does it really make sense to have Locks and Latches duplicate it? Specially at the expense of being able to lock the 'parent' job.

          In our setup we have a need for an exclusive lock for jobs A, B, C and D, where C and D are matrix configurations with dozens of individual, configurations (which need to run in simultaneously). We need to prevent job B from starting while C or D are running. At this time we haven't found a way to accomplish this, but Locks and Latches would seem ideal for the task - if it didn't lock the individual configurations.

          Please reconsider or add an option to provide this support.

          abloom added a comment - Regarding locking each configuration separately, isn't that functionality already implemented for Matrix builds via the "Run each configuration sequentially" checkbox? Since that functionality is already supported it does it really make sense to have Locks and Latches duplicate it? Specially at the expense of being able to lock the 'parent' job. In our setup we have a need for an exclusive lock for jobs A, B, C and D, where C and D are matrix configurations with dozens of individual, configurations (which need to run in simultaneously). We need to prevent job B from starting while C or D are running. At this time we haven't found a way to accomplish this, but Locks and Latches would seem ideal for the task - if it didn't lock the individual configurations. Please reconsider or add an option to provide this support.

          aggamemnon added a comment -

          This issue is also affecting my build environment, as each of our projects in Hudson have around 4 different build configurations, and obviously running them in parallel would be MUCH quicker.

          aggamemnon added a comment - This issue is also affecting my build environment, as each of our projects in Hudson have around 4 different build configurations, and obviously running them in parallel would be MUCH quicker.

          Andrew Bayer added a comment -

          Try using the Throttle Concurrent Builds plugin instead - it properly handles matrix builds in this scenario. https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

          Andrew Bayer added a comment - Try using the Throttle Concurrent Builds plugin instead - it properly handles matrix builds in this scenario. https://wiki.jenkins-ci.org/display/JENKINS/Throttle+Concurrent+Builds+Plugin

          emmulator added a comment -

          abayer, I don't understand. How does the Throttle Concurrent Builds plugin address the usecase where one wants an exclusive lock between a matrix and a non-matrix build?

          emmulator added a comment - abayer, I don't understand. How does the Throttle Concurrent Builds plugin address the usecase where one wants an exclusive lock between a matrix and a non-matrix build?

            stephenconnolly Stephen Connolly
            brianf brianf
            Votes:
            8 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: