Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-41205

Stage graph unsuitable for large and/or complex pipelines

XMLWordPrintable

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major Major
    • blueocean-plugin
    • None
    • Jenkins 2.40
      Blue Ocean 1.0.0-b17

      Improvement on roadmap

      This improvement is on the Blue Ocean project roadmap. Check the roadmap page for updates.

      The Blue Ocean stage graph is great for small, simple pipelines however it breaks down with many parallel builds. See attached screenshot for an example.

      Because 'stage' can no longer be nested within 'parallel', all of our steps must belong under a single 'Test' stage. We have 19 parallel jobs, which is not an uncommon number for iOS/Android development where many combinations of app, device and OS version need to be tested. We'd actually like to split some of the jobs into smaller chunks to take advantage of idle build agents, but this would greatly exacerbate the problem.

      Grouping jobs under multiple stages would improve the UI experience, but also drastically increase the runtime of our integration runs as stages are executed serially.

      I envision two possible solutions:

      1. Stages have a 'parallel' option that allows them to run at the same time as other parallel stages.
      2. A step is introduced that is used purely as an annotation for the purposes of rendering a more appropriate graph. Ideally the step would be deeply nestable allowing for complex graph hierarchies.

      Thanks for all the hard work on Blue Ocean, it's really shaping up nicely and I eagerly await each new release.

        1. too-much-pipeline.png
          too-much-pipeline.png
          68 kB
        2. matrix-test-parallel.jpg
          matrix-test-parallel.jpg
          144 kB
        3. Screen Shot 2017-01-19 at 7.20.51 PM.png
          Screen Shot 2017-01-19 at 7.20.51 PM.png
          101 kB

          [JENKINS-41205] Stage graph unsuitable for large and/or complex pipelines

          Ian Leitch created issue -
          James Dumay made changes -
          Sprint New: post-release [ 181 ]
          James Dumay made changes -
          Rank New: Ranked lower
          James Dumay made changes -
          Epic Link New: JENKINS-35755 [ 171716 ]

          James Dumay added a comment -

          Thanks ileitch! For that second point would you be interested in something like JENKINS-38442 ?

          James Dumay added a comment - Thanks ileitch ! For that second point would you be interested in something like JENKINS-38442 ?

          Ian Leitch added a comment -

          jamesdumay It's not clear in JENKINS-38442 how the graph would be rendered. Nested stages would solve the issue for me provided they do alter the graph, however ideally both nested stages and parallel stages would provide the most flexibility.

          Ian Leitch added a comment - jamesdumay It's not clear in JENKINS-38442 how the graph would be rendered. Nested stages would solve the issue for me provided they do alter the graph, however ideally both nested stages and parallel stages would provide the most flexibility.

          James Dumay added a comment -

          ileitch we are not even clear how things should be rendered because users ideas of how that should look are so diverse

          I think there are some things we can do here to make the existing graph more acceptable in your case. We will also be looking at how to solve the nesting problem this year but I can't give a good ETA on it yet.

          James Dumay added a comment - ileitch we are not even clear how things should be rendered because users ideas of how that should look are so diverse I think there are some things we can do here to make the existing graph more acceptable in your case. We will also be looking at how to solve the nesting problem this year but I can't give a good ETA on it yet.

          Ian Leitch added a comment -

          The sketch you provided: https://issues.jenkins-ci.org/secure/attachment/34100/blueocean.sketch%202016-09-28%2015-03-57.png
          would not be ideal as it only alters the steps table, not the graph. If we applied this approach to our pipeline, the graph would only display a single "Test" stage, with all of the meaningful stages being shown in the list below. This would make the graph linear, and somewhat redundant.

          Ian Leitch added a comment - The sketch you provided: https://issues.jenkins-ci.org/secure/attachment/34100/blueocean.sketch%202016-09-28%2015-03-57.png would not be ideal as it only alters the steps table, not the graph. If we applied this approach to our pipeline, the graph would only display a single "Test" stage, with all of the meaningful stages being shown in the list below. This would make the graph linear, and somewhat redundant.

          James Dumay added a comment -

          ileitch yeah I think that design is totally out for the reason you provided and because two steps can appear on the screen it makes the running behaviour really complex.

          James Dumay added a comment - ileitch yeah I think that design is totally out for the reason you provided and because two steps can appear on the screen it makes the running behaviour really complex.

          James Dumay added a comment -

          ileitch actually would you be up for a Google Hangout in the next few weeks? We can discuss this in greater detail. Would love to see some of the Pipelines you have built and how they show up in Blue Ocean.

          James Dumay added a comment - ileitch actually would you be up for a Google Hangout in the next few weeks? We can discuss this in greater detail. Would love to see some of the Pipelines you have built and how they show up in Blue Ocean.

            benlangfeld Ben Langfeld
            ileitch Ian Leitch
            Votes:
            19 Vote for this issue
            Watchers:
            36 Start watching this issue

              Created:
              Updated:
              Resolved: