Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49050

I can view my sequential declarative pipeline stages in the pipeline visualization plugin

    • Icon: New Feature New Feature
    • Resolution: Fixed
    • Icon: Major Major
    • blueocean-plugin
    • None
    • Blue Ocean 1.8.0

      For Declarative 1.3 (JENKINS-46809), we're adding support for sequential groups of stages, either as part of a set of parallel stages or on its own in a top-level stage. I won't bother going into all the rationale behind it. =) See the Declarative PR for the relevant changes, examples, discussion, etc.

      This will require changes for both visualization and the editor:

      • Visualization - see my pathetic attempts at mockups attached here. jamesdumay probably will come up with something better in like five seconds. =)
      • Editor - Should be something like we have for parallel stages - reordering would be good too.

          [JENKINS-49050] I can view my sequential declarative pipeline stages in the pipeline visualization plugin

          Michael Neale added a comment -

          hey sophistifunk any guesses on how much work this would be for both vis and editor? 

          Michael Neale added a comment - hey sophistifunk any guesses on how much work this would be for both vis and editor? 

          Josh McDonald added a comment -

          Renderer: zero for multi-stage-parallels, as that's already done. As for "grouping" at the top level, it's mostly about cooking up a sensible visual language rather than layout changes.

           

          Editor-wise, I'm not sure. We need to replace the editor's mixed-in layout code with the renderer layout class, which was extracted from the renderer for this purpose and should be done either way.

           

          Also, we would need changes to the `PipelineNodeGraphVisitor` to support it. I have ideas about refactoring that to make it less... intimidating... to change, but I'll need to run them by Vivek and probably a few people on pipeline to make sure I'm not talking out of my ass

          Josh McDonald added a comment - Renderer: zero for multi-stage-parallels, as that's already done. As for "grouping" at the top level, it's mostly about cooking up a sensible visual language rather than layout changes.   Editor-wise, I'm not sure. We need to replace the editor's mixed-in layout code with the renderer layout class, which was extracted from the renderer for this purpose and should be done either way.   Also, we would need changes to the `PipelineNodeGraphVisitor` to support it. I have ideas about refactoring that to make it less... intimidating... to change, but I'll need to run them by Vivek and probably a few people on pipeline to make sure I'm not talking out of my ass

          Michael Neale added a comment -

          thanks sophistifunk - so it sounds like "not a small thing" 

          Michael Neale added a comment - thanks sophistifunk - so it sounds like "not a small thing" 

          Vivek Pandey added a comment -

          Yeah we need changes in nodes  API, opened this ticket: https://issues.jenkins-ci.org/browse/JENKINS-49779.

          Vivek Pandey added a comment - Yeah we need changes in nodes  API, opened this ticket:  https://issues.jenkins-ci.org/browse/JENKINS-49779 .

          Hokwang Lee added a comment -

          Hello.
          I was looking forward to implementing this sequential in parallel feature. Yesterday finally I can use this in Jenkinsfile as code. But I didn't know visualization does not support yet!

          Please assign right person and consider very critical thing to do.

          Hokwang Lee added a comment - Hello. I was looking forward to implementing this sequential in parallel feature. Yesterday finally I can use this in Jenkinsfile as code. But I didn't know visualization does not support yet! Please assign right person and consider very critical thing to do.

          Jenn Briden added a comment -

          We will be focusing first on visualization of sequential stages within the UI. We've begun that work. Editing needs significant review so that we can create an intuitive user experience. In the sake of hyper focusing on delivering high quality experiences for the community very quickly, we are taking one UX task at a time. Stay tuned!

          Jenn Briden added a comment - We will be focusing first on visualization of sequential stages within the UI. We've begun that work. Editing needs significant review so that we can create an intuitive user experience. In the sake of hyper focusing on delivering high quality experiences for the community very quickly, we are taking one UX task at a time. Stay tuned!

          Olivier Lamy added a comment -

          first we need to improve backend rest api.

          Olivier Lamy added a comment - first we need to improve backend rest api.

          Olivier Lamy added a comment -

          Olivier Lamy added a comment - pr https://github.com/jenkinsci/blueocean-plugin/pull/1778

          Josh McDonald added a comment -

          vivek I can take this over at the end of the week when I'm done with these last few PoC tasks if you like, let Nicu enjoy his time seeing the sights. The code's reasonably fresh in my mind because I've been using parts of it for the demo.

          Josh McDonald added a comment - vivek I can take this over at the end of the week when I'm done with these last few PoC tasks if you like, let Nicu enjoy his time seeing the sights. The code's reasonably fresh in my mind because I've been using parts of it for the demo.

          Vivek Pandey added a comment -

          sophistifunk Fine with me, nicu ok with you? We would rather have you enjoy your vacation

          Vivek Pandey added a comment - sophistifunk Fine with me, nicu ok with you? We would rather have you enjoy your vacation

          Nicolae Pascu added a comment -

          Thanks sophistifunk. I have all the debugging code already written locally, and I think I have an easy fix for this. I just talked with olamy about a small bug that prevents me from testing my fix, once that is done, I think I can open a PR without much trouble. If I can't finish by the end of the week or it requires too much time I will contact you and hand over what I already have. I will keep everyone updated here with the progress.

          Nicolae Pascu added a comment - Thanks sophistifunk . I have all the debugging code already written locally, and I think I have an easy fix for this. I just talked with olamy about a small bug that prevents me from testing my fix, once that is done, I think I can open a PR without much trouble. If I can't finish by the end of the week or it requires too much time I will contact you and hand over what I already have. I will keep everyone updated here with the progress.

          Nicolae Pascu added a comment -

          cbercea here is the preview of this Jenkinsfile that we talked about in yesterdays meeting. Let me know what you think.

          Nicolae Pascu added a comment - cbercea here is the preview of this Jenkinsfile that we talked about in yesterdays meeting. Let me know what you think.

          Steven Foster added a comment -

          From the screenshot, I guess that the stage group name ('multiple-stages' in the example Jenkinsfile) won't be displayed anywhere?

          Steven Foster added a comment - From the screenshot, I guess that the stage group name ('multiple-stages' in the example Jenkinsfile) won't be displayed anywhere?

          Chad Bercea added a comment -

          nicu I really like the idea of no longer combining the top most stage name with the name of the "stack". In cases where this happens, what would we name that "parent"? Just curious. Otherwise this looks exactly like what we talked about. Nice!

          Chad Bercea added a comment - nicu I really like the idea of no longer combining the top most stage name with the name of the "stack". In cases where this happens, what would we name that "parent"? Just curious. Otherwise this looks exactly like what we talked about. Nice!

          Nicolae Pascu added a comment -

          stevenfoster that is correct, the container stage ('multiple-stages') can't have any steps so that is why we decided to not include it anywhere.

          Nicolae Pascu added a comment - stevenfoster that is correct, the container stage ('multiple-stages') can't have any steps so that is why we decided to not include it anywhere.

          Nicolae Pascu added a comment -

          Thanks cbercea, as for the edge case that you mention, I don't really understand it, I will PM you later so we can clarify.

          Nicolae Pascu added a comment - Thanks  cbercea , as for the edge case that you mention, I don't really understand it, I will PM you later so we can clarify.

          Nicolae Pascu added a comment -

          Opened PR#1784

          Nicolae Pascu added a comment - Opened  PR#1784

          Marcin Grinberg added a comment - - edited

          Hi nicu,

           

          Are you aware in which version of blue-ocean can we expect this feature?

           

          Thanks for your work on this feature btw.

          Marcin Grinberg added a comment - - edited Hi nicu ,   Are you aware in which version of blue-ocean can we expect this feature?   Thanks for your work on this feature btw.

          Nicolae Pascu added a comment -

          Hi mgrinberg, this feature will land in the next version of blueocean, which will be 1.8

          Nicolae Pascu added a comment - Hi  mgrinberg , this feature will land in the next version of blueocean, which will be 1.8

          Stephan Vedder added a comment - - edited

          This new feature in BlueOcean 1.8 did definetly not fix visualization of parallel sequential stages, but instead break it completly:

          Each of Windows, Linux etc. is one sequential stage. The corresponding Jenkinsfile looks like this:
          https://gist.github.com/feliwir/75864cb641c448446f836056fc698d8a

          While the build is running everything is bugging out, things are moving all over the screen, but not where they're supposed to be. Any advice why the visualization is completly bugged now is welcome!

          Stephan Vedder added a comment - - edited This new feature in BlueOcean 1.8 did definetly not fix visualization of parallel sequential stages, but instead break it completly: Each of Windows, Linux etc. is one sequential stage. The corresponding Jenkinsfile looks like this: https://gist.github.com/feliwir/75864cb641c448446f836056fc698d8a While the build is running everything is bugging out, things are moving all over the screen, but not where they're supposed to be. Any advice why the visualization is completly bugged now is welcome!

          Nicolae Pascu added a comment - - edited

          Hi feliwir, I looked over your Jenkinsfile and saw a problem. Per the sequential stages documentation each sequential parallel group needs to be contained by a named Stage that doesn't have any steps.

          As an example, your sequential stages called 'Build' and 'Test' need to have another container stage with a name that is the immediate child of the 'stages' container.

          Please take a look at this Jenkinsfile for an example.

          Nicolae Pascu added a comment - - edited Hi feliwir , I looked over your Jenkinsfile and saw a problem. Per the  sequential stages documentation  each sequential parallel group needs to be contained by a named Stage that doesn't have any steps. As an example, your sequential stages called 'Build' and 'Test' need to have another container stage with a name that is the immediate child of the 'stages' container. Please take a look at this Jenkinsfile  for an example.

          Will this also work for "Scripted Pipeline syntax" or do one need to start migrating all "scripted" scripts to "declarative"?

          Carel Combrink added a comment - Will this also work for "Scripted Pipeline syntax" or do one need to start migrating all "scripted" scripts to "declarative"?

          Olivier Lamy added a comment -

          carelc it's only supported with declarative

          Olivier Lamy added a comment - carelc it's only supported with declarative

          Stephan Vedder added a comment - - edited

          nicu i did look at the jenkinsfile you've linked above very closely, but i can't spot any differences. The "Build" and "Test" stage are directly in a stages block, just like your sequential stages (here). The nesting doesn't seem to be different in any way or did i miss sth?

          Stephan Vedder added a comment - - edited nicu i did look at the jenkinsfile you've linked above very closely, but i can't spot any differences. The "Build" and "Test" stage are directly in a stages block, just like your sequential stages ( here ). The nesting doesn't seem to be different in any way or did i miss sth?

          Andrew Bayer added a comment -

          Yeah, feliwir’s Jenkinsfile looks right to me - here’s how the stages are laid out:

          Continuous (children are parallel) ->
            Windows (children are sequential) ->
              Build
              Test
            Linux (children are sequential) ->
              Build
              Test
              Coverage
            MemCheck (children are sequential) ->
              Build
              Memcheck 
            Android (children are sequential) ->
              Build
            iOS (children are sequential) ->
              Build
          

          It’s a valid Jenkinsfile and should render correctly. feliwir, could you please open a new JIRA for this and include your Jenkinsfile there? Thanks.

          Andrew Bayer added a comment - Yeah, feliwir ’s Jenkinsfile looks right to me - here’s how the stages are laid out: Continuous (children are parallel) -> Windows (children are sequential) -> Build Test Linux (children are sequential) -> Build Test Coverage MemCheck (children are sequential) -> Build Memcheck Android (children are sequential) -> Build iOS (children are sequential) -> Build It’s a valid Jenkinsfile and should render correctly. feliwir , could you please open a new JIRA for this and include your Jenkinsfile there? Thanks.

          Stephan Vedder added a comment - I've create the new issue: abayer nicu See https://issues.jenkins-ci.org/browse/JENKINS-53048

          Looks awesome! Thanks a lot!

          Mathias Hasselmann added a comment - Looks awesome! Thanks a lot!

          Olivier Lamy added a comment -

          horangs any Jenkinsfile to reproduce that?

          Olivier Lamy added a comment - horangs any Jenkinsfile to reproduce that?

          Hokwang Lee added a comment - - edited

          nicu and abayer

           

          Today I can test this visualization, and it shows me like below. Nice works.

          But there's one big concern.

          In my pipeline code,

          stage('Parallel') : it really does not have meaning.

          stage('Feature A') and stage('Feature B') and stage('Feature C') : now there's no way to see, (it means sequential actions group(type) name)

          but this is more important and user should be aware of it.

           

          How do you think of this? And please let me know related issue number if exists. Or not, can I create new issue for this?

           

          Thank you.

          Hokwang Lee added a comment - - edited nicu and abayer   Today I can test this visualization, and it shows me like below. Nice works. But there's one big concern. In my pipeline code, stage('Parallel') : it really does not have meaning. stage('Feature A') and stage('Feature B') and stage('Feature C') : now there's no way to see, (it means sequential actions group(type) name) but this is more important and user should be aware of it.   How do you think of this? And please let me know related issue number if exists. Or not, can I create new issue for this?   Thank you.

          Hokwang Lee added a comment -

          olamy Oh, please forget it, now I guess there's no problem. Please look at my comment anyway upside.

          Hokwang Lee added a comment - olamy Oh, please forget it, now I guess there's no problem. Please look at my comment anyway upside.

          horangs see JENKINS-53127. Unfortunately nobody triaged that issue yet.

          Also much more important: JENKINS-53547

          Mathias Hasselmann added a comment - horangs see  JENKINS-53127 . Unfortunately nobody triaged that issue yet. Also much more important:  JENKINS-53547

            nicu Nicolae Pascu
            abayer Andrew Bayer
            Votes:
            18 Vote for this issue
            Watchers:
            40 Start watching this issue

              Created:
              Updated:
              Resolved: