Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49050

I can view my sequential declarative pipeline stages in the pipeline visualization plugin

    XMLWordPrintable

Details

    • Blue Ocean 1.8.0

    Description

      For Declarative 1.3 (JENKINS-46809), we're adding support for sequential groups of stages, either as part of a set of parallel stages or on its own in a top-level stage. I won't bother going into all the rationale behind it. =) See the Declarative PR for the relevant changes, examples, discussion, etc.

      This will require changes for both visualization and the editor:

      • Visualization - see my pathetic attempts at mockups attached here. jamesdumay probably will come up with something better in like five seconds. =)
      • Editor - Should be something like we have for parallel stages - reordering would be good too.

      Attachments

        Issue Links

          Activity

            michaelneale Michael Neale added a comment -

            hey sophistifunk any guesses on how much work this would be for both vis and editor? 

            michaelneale Michael Neale added a comment - hey sophistifunk any guesses on how much work this would be for both vis and editor? 
            sophistifunk Josh McDonald added a comment -

            Renderer: zero for multi-stage-parallels, as that's already done. As for "grouping" at the top level, it's mostly about cooking up a sensible visual language rather than layout changes.

             

            Editor-wise, I'm not sure. We need to replace the editor's mixed-in layout code with the renderer layout class, which was extracted from the renderer for this purpose and should be done either way.

             

            Also, we would need changes to the `PipelineNodeGraphVisitor` to support it. I have ideas about refactoring that to make it less... intimidating... to change, but I'll need to run them by Vivek and probably a few people on pipeline to make sure I'm not talking out of my ass

            sophistifunk Josh McDonald added a comment - Renderer: zero for multi-stage-parallels, as that's already done. As for "grouping" at the top level, it's mostly about cooking up a sensible visual language rather than layout changes.   Editor-wise, I'm not sure. We need to replace the editor's mixed-in layout code with the renderer layout class, which was extracted from the renderer for this purpose and should be done either way.   Also, we would need changes to the `PipelineNodeGraphVisitor` to support it. I have ideas about refactoring that to make it less... intimidating... to change, but I'll need to run them by Vivek and probably a few people on pipeline to make sure I'm not talking out of my ass
            michaelneale Michael Neale added a comment -

            thanks sophistifunk - so it sounds like "not a small thing" 

            michaelneale Michael Neale added a comment - thanks sophistifunk - so it sounds like "not a small thing" 
            vivek Vivek Pandey added a comment -

            Yeah we need changes in nodes  API, opened this ticket: https://issues.jenkins-ci.org/browse/JENKINS-49779.

            vivek Vivek Pandey added a comment - Yeah we need changes in nodes  API, opened this ticket:  https://issues.jenkins-ci.org/browse/JENKINS-49779 .
            horangs Hokwang Lee added a comment -

            Hello.
            I was looking forward to implementing this sequential in parallel feature. Yesterday finally I can use this in Jenkinsfile as code. But I didn't know visualization does not support yet!

            Please assign right person and consider very critical thing to do.

            horangs Hokwang Lee added a comment - Hello. I was looking forward to implementing this sequential in parallel feature. Yesterday finally I can use this in Jenkinsfile as code. But I didn't know visualization does not support yet! Please assign right person and consider very critical thing to do.
            jbriden Jenn Briden added a comment -

            We will be focusing first on visualization of sequential stages within the UI. We've begun that work. Editing needs significant review so that we can create an intuitive user experience. In the sake of hyper focusing on delivering high quality experiences for the community very quickly, we are taking one UX task at a time. Stay tuned!

            jbriden Jenn Briden added a comment - We will be focusing first on visualization of sequential stages within the UI. We've begun that work. Editing needs significant review so that we can create an intuitive user experience. In the sake of hyper focusing on delivering high quality experiences for the community very quickly, we are taking one UX task at a time. Stay tuned!
            olamy Olivier Lamy added a comment -

            first we need to improve backend rest api.

            olamy Olivier Lamy added a comment - first we need to improve backend rest api.
            olamy Olivier Lamy added a comment - pr https://github.com/jenkinsci/blueocean-plugin/pull/1778
            sophistifunk Josh McDonald added a comment -

            vivek I can take this over at the end of the week when I'm done with these last few PoC tasks if you like, let Nicu enjoy his time seeing the sights. The code's reasonably fresh in my mind because I've been using parts of it for the demo.

            sophistifunk Josh McDonald added a comment - vivek I can take this over at the end of the week when I'm done with these last few PoC tasks if you like, let Nicu enjoy his time seeing the sights. The code's reasonably fresh in my mind because I've been using parts of it for the demo.
            vivek Vivek Pandey added a comment -

            sophistifunk Fine with me, nicu ok with you? We would rather have you enjoy your vacation

            vivek Vivek Pandey added a comment - sophistifunk Fine with me, nicu ok with you? We would rather have you enjoy your vacation
            nicu Nicolae Pascu added a comment -

            Thanks sophistifunk. I have all the debugging code already written locally, and I think I have an easy fix for this. I just talked with olamy about a small bug that prevents me from testing my fix, once that is done, I think I can open a PR without much trouble. If I can't finish by the end of the week or it requires too much time I will contact you and hand over what I already have. I will keep everyone updated here with the progress.

            nicu Nicolae Pascu added a comment - Thanks sophistifunk . I have all the debugging code already written locally, and I think I have an easy fix for this. I just talked with olamy about a small bug that prevents me from testing my fix, once that is done, I think I can open a PR without much trouble. If I can't finish by the end of the week or it requires too much time I will contact you and hand over what I already have. I will keep everyone updated here with the progress.
            nicu Nicolae Pascu added a comment -

            cbercea here is the preview of this Jenkinsfile that we talked about in yesterdays meeting. Let me know what you think.

            nicu Nicolae Pascu added a comment - cbercea here is the preview of this Jenkinsfile that we talked about in yesterdays meeting. Let me know what you think.
            stevenfoster Steven Foster added a comment -

            From the screenshot, I guess that the stage group name ('multiple-stages' in the example Jenkinsfile) won't be displayed anywhere?

            stevenfoster Steven Foster added a comment - From the screenshot, I guess that the stage group name ('multiple-stages' in the example Jenkinsfile) won't be displayed anywhere?
            cbercea Chad Bercea added a comment -

            nicu I really like the idea of no longer combining the top most stage name with the name of the "stack". In cases where this happens, what would we name that "parent"? Just curious. Otherwise this looks exactly like what we talked about. Nice!

            cbercea Chad Bercea added a comment - nicu I really like the idea of no longer combining the top most stage name with the name of the "stack". In cases where this happens, what would we name that "parent"? Just curious. Otherwise this looks exactly like what we talked about. Nice!
            nicu Nicolae Pascu added a comment -

            stevenfoster that is correct, the container stage ('multiple-stages') can't have any steps so that is why we decided to not include it anywhere.

            nicu Nicolae Pascu added a comment - stevenfoster that is correct, the container stage ('multiple-stages') can't have any steps so that is why we decided to not include it anywhere.
            nicu Nicolae Pascu added a comment -

            Thanks cbercea, as for the edge case that you mention, I don't really understand it, I will PM you later so we can clarify.

            nicu Nicolae Pascu added a comment - Thanks  cbercea , as for the edge case that you mention, I don't really understand it, I will PM you later so we can clarify.
            nicu Nicolae Pascu added a comment -

            Opened PR#1784

            nicu Nicolae Pascu added a comment - Opened  PR#1784
            mgrinberg Marcin Grinberg added a comment - - edited

            Hi nicu,

             

            Are you aware in which version of blue-ocean can we expect this feature?

             

            Thanks for your work on this feature btw.

            mgrinberg Marcin Grinberg added a comment - - edited Hi nicu ,   Are you aware in which version of blue-ocean can we expect this feature?   Thanks for your work on this feature btw.
            nicu Nicolae Pascu added a comment -

            Hi mgrinberg, this feature will land in the next version of blueocean, which will be 1.8

            nicu Nicolae Pascu added a comment - Hi  mgrinberg , this feature will land in the next version of blueocean, which will be 1.8
            feliwir Stephan Vedder added a comment - - edited

            This new feature in BlueOcean 1.8 did definetly not fix visualization of parallel sequential stages, but instead break it completly:

            Each of Windows, Linux etc. is one sequential stage. The corresponding Jenkinsfile looks like this:
            https://gist.github.com/feliwir/75864cb641c448446f836056fc698d8a

            While the build is running everything is bugging out, things are moving all over the screen, but not where they're supposed to be. Any advice why the visualization is completly bugged now is welcome!

            feliwir Stephan Vedder added a comment - - edited This new feature in BlueOcean 1.8 did definetly not fix visualization of parallel sequential stages, but instead break it completly: Each of Windows, Linux etc. is one sequential stage. The corresponding Jenkinsfile looks like this: https://gist.github.com/feliwir/75864cb641c448446f836056fc698d8a While the build is running everything is bugging out, things are moving all over the screen, but not where they're supposed to be. Any advice why the visualization is completly bugged now is welcome!
            nicu Nicolae Pascu added a comment - - edited

            Hi feliwir, I looked over your Jenkinsfile and saw a problem. Per the sequential stages documentation each sequential parallel group needs to be contained by a named Stage that doesn't have any steps.

            As an example, your sequential stages called 'Build' and 'Test' need to have another container stage with a name that is the immediate child of the 'stages' container.

            Please take a look at this Jenkinsfile for an example.

            nicu Nicolae Pascu added a comment - - edited Hi feliwir , I looked over your Jenkinsfile and saw a problem. Per the  sequential stages documentation  each sequential parallel group needs to be contained by a named Stage that doesn't have any steps. As an example, your sequential stages called 'Build' and 'Test' need to have another container stage with a name that is the immediate child of the 'stages' container. Please take a look at this Jenkinsfile  for an example.

            Will this also work for "Scripted Pipeline syntax" or do one need to start migrating all "scripted" scripts to "declarative"?

            carelc Carel Combrink added a comment - Will this also work for "Scripted Pipeline syntax" or do one need to start migrating all "scripted" scripts to "declarative"?
            olamy Olivier Lamy added a comment -

            carelc it's only supported with declarative

            olamy Olivier Lamy added a comment - carelc it's only supported with declarative
            feliwir Stephan Vedder added a comment - - edited

            nicu i did look at the jenkinsfile you've linked above very closely, but i can't spot any differences. The "Build" and "Test" stage are directly in a stages block, just like your sequential stages (here). The nesting doesn't seem to be different in any way or did i miss sth?

            feliwir Stephan Vedder added a comment - - edited nicu i did look at the jenkinsfile you've linked above very closely, but i can't spot any differences. The "Build" and "Test" stage are directly in a stages block, just like your sequential stages ( here ). The nesting doesn't seem to be different in any way or did i miss sth?
            abayer Andrew Bayer added a comment -

            Yeah, feliwir’s Jenkinsfile looks right to me - here’s how the stages are laid out:

            Continuous (children are parallel) ->
              Windows (children are sequential) ->
                Build
                Test
              Linux (children are sequential) ->
                Build
                Test
                Coverage
              MemCheck (children are sequential) ->
                Build
                Memcheck 
              Android (children are sequential) ->
                Build
              iOS (children are sequential) ->
                Build
            

            It’s a valid Jenkinsfile and should render correctly. feliwir, could you please open a new JIRA for this and include your Jenkinsfile there? Thanks.

            abayer Andrew Bayer added a comment - Yeah, feliwir ’s Jenkinsfile looks right to me - here’s how the stages are laid out: Continuous (children are parallel) -> Windows (children are sequential) -> Build Test Linux (children are sequential) -> Build Test Coverage MemCheck (children are sequential) -> Build Memcheck Android (children are sequential) -> Build iOS (children are sequential) -> Build It’s a valid Jenkinsfile and should render correctly. feliwir , could you please open a new JIRA for this and include your Jenkinsfile there? Thanks.
            feliwir Stephan Vedder added a comment - I've create the new issue: abayer nicu See https://issues.jenkins-ci.org/browse/JENKINS-53048

            Looks awesome! Thanks a lot!

            hasselmm Mathias Hasselmann added a comment - Looks awesome! Thanks a lot!
            olamy Olivier Lamy added a comment -

            horangs any Jenkinsfile to reproduce that?

            olamy Olivier Lamy added a comment - horangs any Jenkinsfile to reproduce that?
            horangs Hokwang Lee added a comment - - edited

            nicu and abayer

             

            Today I can test this visualization, and it shows me like below. Nice works.

            But there's one big concern.

            In my pipeline code,

            stage('Parallel') : it really does not have meaning.

            stage('Feature A') and stage('Feature B') and stage('Feature C') : now there's no way to see, (it means sequential actions group(type) name)

            but this is more important and user should be aware of it.

             

            How do you think of this? And please let me know related issue number if exists. Or not, can I create new issue for this?

             

            Thank you.

            horangs Hokwang Lee added a comment - - edited nicu and abayer   Today I can test this visualization, and it shows me like below. Nice works. But there's one big concern. In my pipeline code, stage('Parallel') : it really does not have meaning. stage('Feature A') and stage('Feature B') and stage('Feature C') : now there's no way to see, (it means sequential actions group(type) name) but this is more important and user should be aware of it.   How do you think of this? And please let me know related issue number if exists. Or not, can I create new issue for this?   Thank you.
            horangs Hokwang Lee added a comment -

            olamy Oh, please forget it, now I guess there's no problem. Please look at my comment anyway upside.

            horangs Hokwang Lee added a comment - olamy Oh, please forget it, now I guess there's no problem. Please look at my comment anyway upside.

            horangs see JENKINS-53127. Unfortunately nobody triaged that issue yet.

            Also much more important: JENKINS-53547

            hasselmm Mathias Hasselmann added a comment - horangs see  JENKINS-53127 . Unfortunately nobody triaged that issue yet. Also much more important:  JENKINS-53547

            People

              nicu Nicolae Pascu
              abayer Andrew Bayer
              Votes:
              18 Vote for this issue
              Watchers:
              40 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: