• Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • plugin-proposals
    • None
    • Platform: All, OS: All

      Hi guys,

      I would like to create a "build priority plugin" so, depending on the build
      priority number, the build queue could reorder the applications, API to be
      built. I read the plugin section on the Wiki, however I cannot find the good
      extension point. Perhaps you could give me some tips and advices for starting
      this plugin development.

      Thanks

      David

          [JENKINS-833] Build Priority

          davyboyhayes added a comment -

          Scrap my last comment, this doesn't appear to work. Maybe if you had the same
          number of Build Executors as you had projects, and you used this mentality, it
          might work. But after running a test, even if one of the projects takes too long
          to pull from SCM, it throws the whole thing off and everything build breaks
          again in a big steaming pile .

          I can second the need to have build priority! The build priority should take
          over from the Quiet Period, i.e. if Project B (that depends upon Project A) is
          in the Quiet Period for 1 minute, and Project A is in the quiet period for 10
          minutes, Project B should be queued until after Project A has completed (i.e. at
          least 10 minutes away).

          In my case we have 7 projects (named A-G for simplicity).
          Project A is a root library
          Project B depends upon A
          Project C depends upon A and B (and is an end application)
          Project D depends upon A and B (library)
          Project E depends upon A and B (library)
          Project F depends upon A and B (library)
          Project G depends upon A, B, D, E and F.

          Thus if I make changes to say Projects B, C and G, I would expect the build to
          actually build B, then C, then D (it depends upon B), then E (it depends upon
          B), then F (it depends upon B), then G.

          Many thanks,

          David (a different David to the original)

          davyboyhayes added a comment - Scrap my last comment, this doesn't appear to work. Maybe if you had the same number of Build Executors as you had projects, and you used this mentality, it might work. But after running a test, even if one of the projects takes too long to pull from SCM, it throws the whole thing off and everything build breaks again in a big steaming pile . I can second the need to have build priority! The build priority should take over from the Quiet Period, i.e. if Project B (that depends upon Project A) is in the Quiet Period for 1 minute, and Project A is in the quiet period for 10 minutes, Project B should be queued until after Project A has completed (i.e. at least 10 minutes away). In my case we have 7 projects (named A-G for simplicity). Project A is a root library Project B depends upon A Project C depends upon A and B (and is an end application) Project D depends upon A and B (library) Project E depends upon A and B (library) Project F depends upon A and B (library) Project G depends upon A, B, D, E and F. Thus if I make changes to say Projects B, C and G, I would expect the build to actually build B, then C, then D (it depends upon B), then E (it depends upon B), then F (it depends upon B), then G. Many thanks, David (a different David to the original)

          Recent activity about this in the users list:
          http://www.nabble.com/Build-Priority-Status-tt21441976.html

          Kohsuke Kawaguchi added a comment - Recent activity about this in the users list: http://www.nabble.com/Build-Priority-Status-tt21441976.html

          mdonohue added a comment -

          Adding dependency on replaceable queue

          mdonohue added a comment - Adding dependency on replaceable queue

          I think we can do this by introducing a new singleton extension point (like LoadBalancer) that induces the order between items in the Queue. Plugins can then define various ordering strategy, such as:

          • introduce a JobProperty that remembers priority as int or float, then use that for sorting.
          • introduce a JobProperty that remembers a boolean field for "high priority project", then use that for sorting.
          • look at "Cause" objects and determine which causes are more important (for example, manually triggered builds might be considered more important than cron triggered builds)
          • look at the owner of the job and determine the order (builds by the release engineer might be more important than builds by developers.)

          A related improvement to allow a badge next to queue items might be good (just like how builds can get badges), so that such plugins can show some visual clues to the user.

          Back then I was thinking of this in the context of JENKINS-2072 to make the entire Queue pluggable, but since then my thinking changed a bit, and now I think splitting off various strategies around the default Queue implementation is useful on its own (possibly in addition to JENKINS-2072.)

          Kohsuke Kawaguchi added a comment - I think we can do this by introducing a new singleton extension point (like LoadBalancer) that induces the order between items in the Queue. Plugins can then define various ordering strategy, such as: introduce a JobProperty that remembers priority as int or float, then use that for sorting. introduce a JobProperty that remembers a boolean field for "high priority project", then use that for sorting. look at "Cause" objects and determine which causes are more important (for example, manually triggered builds might be considered more important than cron triggered builds) look at the owner of the job and determine the order (builds by the release engineer might be more important than builds by developers.) A related improvement to allow a badge next to queue items might be good (just like how builds can get badges), so that such plugins can show some visual clues to the user. Back then I was thinking of this in the context of JENKINS-2072 to make the entire Queue pluggable, but since then my thinking changed a bit, and now I think splitting off various strategies around the default Queue implementation is useful on its own (possibly in addition to JENKINS-2072 .)

          Code changed in hudson
          User: : kohsuke
          Path:
          trunk/hudson/main/core/src/main/java/hudson/model/Cause.java
          trunk/hudson/main/core/src/main/java/hudson/model/Queue.java
          trunk/hudson/main/core/src/main/java/hudson/model/queue/AbstractQueueSorterImpl.java
          trunk/hudson/main/core/src/main/java/hudson/model/queue/QueueSorter.java
          trunk/www/changelog.html
          http://jenkins-ci.org/commit/26610
          Log:
          JENKINS-833 formalized the QueueSorter.

          SCM/JIRA link daemon added a comment - Code changed in hudson User: : kohsuke Path: trunk/hudson/main/core/src/main/java/hudson/model/Cause.java trunk/hudson/main/core/src/main/java/hudson/model/Queue.java trunk/hudson/main/core/src/main/java/hudson/model/queue/AbstractQueueSorterImpl.java trunk/hudson/main/core/src/main/java/hudson/model/queue/QueueSorter.java trunk/www/changelog.html http://jenkins-ci.org/commit/26610 Log: JENKINS-833 formalized the QueueSorter.

          radaczynski added a comment -

          Is there any work going on this issue? I am also interested in this happening and I can actually put some time into implementing it.

          radaczynski added a comment - Is there any work going on this issue? I am also interested in this happening and I can actually put some time into implementing it.

          bklarson added a comment -

          I've made a plugin which supports job priority, but it is very basic/simple.

          Each job gets a static priority, and builds in the queue are sorted based on the job priority. Jobs with the same priority are still ordered FIFO. If anyone is interested, let me know and I'll upload it to the plugin server.

          bklarson added a comment - I've made a plugin which supports job priority, but it is very basic/simple. Each job gets a static priority, and builds in the queue are sorted based on the job priority. Jobs with the same priority are still ordered FIFO. If anyone is interested, let me know and I'll upload it to the plugin server.

          deccico added a comment - - edited

          Hi, I wrote a simple Groovy script to avoid this problem. It is published here: http://adrian.org.ar/ci/concurrency-in-hudson

          deccico added a comment - - edited Hi, I wrote a simple Groovy script to avoid this problem. It is published here: http://adrian.org.ar/ci/concurrency-in-hudson

          alexku added a comment -

          bklarson's Priority Sorter plugin is on the plugin server, and it works fine:
          http://wiki.jenkins-ci.org/display/JENKINS/Priority+Sorter+Plugin

          Just an FYI for all those watching this issue.

          alexku added a comment - bklarson's Priority Sorter plugin is on the plugin server, and it works fine: http://wiki.jenkins-ci.org/display/JENKINS/Priority+Sorter+Plugin Just an FYI for all those watching this issue.

          jieryn added a comment -

          jieryn added a comment - The http://wiki.jenkins-ci.org/display/JENKINS/Priority+Sorter+Plugin solves this issue. Closing.

            kohsuke Kohsuke Kawaguchi
            dboissier dboissier
            Votes:
            10 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: