Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43679

Support visualization of downstream pipelines in Jenkins pipeline based views

      If I have a pipeline job (formerly workflow) Foo, that then triggers another pipeline job Bar, it does not get reflected in the delivery pipeline view. Job Bar is triggered as a build step in my job, for example:

      Job Foo:

      stage('Building') {
          println 'Triggering Bar'
          build 'Bar'
      }
      

      Job Bar:

      node('master') {
          stage('Child job stage') {
              println 'Doing stuff in child job'
          }
      }
      

      Only the Foo pipeline job and its steps are displayed in the delivery pipeline view:

      In this case, both Foo and Bar should have appeared as separate entities on the delivery pipeline view.

        1. teamcityView.png
          teamcityView.png
          24 kB
        2. pipeline view.png
          pipeline view.png
          12 kB
        3. mtldP.png
          mtldP.png
          6 kB

          [JENKINS-43679] Support visualization of downstream pipelines in Jenkins pipeline based views

          Tommy Tynjä added a comment -

          martinmine this is currently not within scope and I'm unsure if it should be. Do you have any suggestion for how that could be visualized?

          Tommy Tynjä added a comment - martinmine this is currently not within scope and I'm unsure if it should be. Do you have any suggestion for how that could be visualized?

          Martin added a comment -

          tommysdk how is this outside the scope of the plugin? The summary on the wiki page says "Visualisation of Delivery/Build Pipelines, renders pipelines based on upstream/downstream jobs", I don't see why the case my jobs are pipelines makes them fall outside the scope of the plugin. 

          I would suggest to display the pipeline jobs the same way as downstream jobs are visualized. Based on the description on the wiki, I was expecting something more like this: 

          Martin added a comment - tommysdk  how is this outside the scope of the plugin? The summary on the wiki page says "Visualisation of Delivery/Build Pipelines, renders pipelines based on upstream/downstream jobs ", I don't see why the case my jobs are pipelines makes them fall outside the scope of the plugin.  I would suggest to display the pipeline jobs the same way as downstream jobs are visualized. Based on the description on the wiki, I was expecting something more like this: 

          Tommy Tynjä added a comment -

          martinmine Thanks for your thoughts on this. What I meant is that with the current solution, this is not a supported use case and therefore it does not show up as you expect in the pipeline view.

          Are your pipelines always short and only contain one or a few stages only? I think it could make a difference on how you would expect this to work depending on if you have short or long pipelines.

          Tommy Tynjä added a comment - martinmine Thanks for your thoughts on this. What I meant is that with the current solution, this is not a supported use case and therefore it does not show up as you expect in the pipeline view. Are your pipelines always short and only contain one or a few stages only? I think it could make a difference on how you would expect this to work depending on if you have short or long pipelines.

          Martin added a comment -

          tommysdk In that case if my current use-case is not supported, what is the actual use-case that the pipeline (workflow) job support that is currently covered by JENKINS-34040? The issue description says pipelines (note: plural), and the description for adding a new view for pipeline jobs says "Continuous Delivery pipelines, perfect for visualization on information radiators. Shows one or more delivery pipeline instances, based on traditional Jenkins jobs with upstream/downstream dependencies." which gives me the impression I can use this for displaying multiple pipelines together and their relationship. 

          How is the length of my pipeline relevant (Do you also mean pipeline jobs or the concept of pipelines which this plugin utilizes that is again different from pipeline jobs ? To answer your question, my current pipeline jobs contains just 3-4 stages. However, they are then distributed across multiple jobs as I'm doing multi-platform builds. 

          Martin added a comment - tommysdk In that case if my current use-case is not supported, what is the actual use-case that the pipeline (workflow) job support that is currently covered by JENKINS-34040 ? The issue description says pipelines (note: plural), and the description for adding a new view for pipeline jobs says "Continuous Delivery pipelines, perfect for visualization on information radiators. Shows one or more delivery pipeline instances , based on traditional Jenkins jobs with upstream/downstream dependencies." which gives me the impression I can use this for displaying multiple pipelines together and their relationship.  How is the length of my pipeline relevant (Do you also mean pipeline jobs or the concept of pipelines which this plugin utilizes that is again different from pipeline jobs ? To answer your question, my current pipeline jobs contains just 3-4 stages. However, they are then distributed across multiple jobs as I'm doing multi-platform builds. 

          Tommy Tynjä added a comment -

          martinmine JENKINS-34040 covers the support of visualizing a single pipeline script as a delivery pipeline. I think it makes more sense to add features as we go along. If all possible scenarios would have been expected to be supported, development would still be ongoing and for a foreseeable future.

          You can select to show multiple pipelines in the same view, but that does not mean that they have any relation as such. Here's an example of a delivery pipeline view showing multiple pipelines in the same view: https://www.slideshare.net/tommysdk/continuous-delivery-of-microservices/33

          The most feedback I've got from users using Jenkins pipelines is that the entire pipeline is covered in one definition, therefore we started to implement support for that. I understand your use case and it could possibly be solved. However, with regards to how the piplines and cps executions work it is not as trivial to resolve as for upstream/downstream jobs.

          Tommy Tynjä added a comment - martinmine JENKINS-34040 covers the support of visualizing a single pipeline script as a delivery pipeline. I think it makes more sense to add features as we go along. If all possible scenarios would have been expected to be supported, development would still be ongoing and for a foreseeable future. You can select to show multiple pipelines in the same view, but that does not mean that they have any relation as such. Here's an example of a delivery pipeline view showing multiple pipelines in the same view: https://www.slideshare.net/tommysdk/continuous-delivery-of-microservices/33 The most feedback I've got from users using Jenkins pipelines is that the entire pipeline is covered in one definition, therefore we started to implement support for that. I understand your use case and it could possibly be solved. However, with regards to how the piplines and cps executions work it is not as trivial to resolve as for upstream/downstream jobs.

          Federico Naum added a comment -

          Hi, 

          At my workplace, we are switching all our traditional job to Jenkins pipelines (as in scripted/declarative pipelines) and as well as martinmine we have several stages per job and, and usually, the last stage of a job triggers another job (which clones a different git repo and executes several stages, and triggers another dependency job).

          We would like to see the whole pipeline as described by Martin in the second comment. Also talked with some people at the Sydney Jenkins Area Meetup and they were in the same boat, they wanted to see the whole pipeline. Similar to what can be rendered in TeamCity. See attached picture

           

          Int he meantime, I guess we might get away when you release  47529. Do you have an ETA for releasing it? 

          In the 'not so long term', I guess we will like to see this feature. I'll check with one of the Product Manager of Cloudbees to see if this is on the Blue ocean roadmap. 

          I'm pretty busy until Feb, but I guess I could devote some time after that.

           

          Fede

           

          Federico Naum added a comment - Hi,  At my workplace, we are switching all our traditional job to Jenkins pipelines (as in scripted/declarative pipelines) and as well as martinmine we have several stages per job and, and usually, the last stage of a job triggers another job (which clones a different git repo and executes several stages, and triggers another dependency job). We would like to see the whole pipeline as described by Martin in the second comment. Also talked with some people at the  Sydney Jenkins Area Meetup  and they were in the same boat, they wanted to see the whole pipeline. Similar to what can be rendered in TeamCity. See attached picture   Int he meantime, I guess we might get away when you release  47529. Do you have an ETA for releasing it?  In the 'not so long term', I guess we will like to see this feature. I'll check with one of the Product Manager of Cloudbees to see if this is on the Blue ocean roadmap.  I'm pretty busy until Feb, but I guess I could devote some time after that.   Fede  

          Tommy Tynjä added a comment - - edited

          Hi fnaum! Thank you for your comment and the thorough explanation and attached screenshot.

          JENKINS-47529 is actually ready to be released. I had hoped to be able to release it together with JENKINS-45738 which is still in progress though, but they are not dependent on each other so if there is an urgent need we could go ahead and release 47529 prior to that.

          For this particular issue, there is currently no active development ongoing and since plugin development time is limited, any contributions from the community to realize this feature would be much appreciated.

          Tommy Tynjä added a comment - - edited Hi fnaum ! Thank you for your comment and the thorough explanation and attached screenshot. JENKINS-47529 is actually ready to be released. I had hoped to be able to release it together with JENKINS-45738 which is still in progress though, but they are not dependent on each other so if there is an urgent need we could go ahead and release 47529 prior to that. For this particular issue, there is currently no active development ongoing and since plugin development time is limited, any contributions from the community to realize this feature would be much appreciated.

          Federico Naum added a comment -

          Hi tommysdk , Thank you for your quick answer.

          There is no real urgency with -JENKINS-47529- , but on the other hand, it seems like a great feature that is independent of JENKINS-45738  and it deserves its own release.

          Without looking at the actual code changes in JENKINS-45738, it seems quite an involved change that could bring some bugs (hopefully not).

          Personally, I prefer smaller iterations, that are easier to roll back. And in the case, if you need to roll back you only lose one feature, not two. 

          I would say that if you are happy with the tests in JENKINS-47529 , just go ahead and release it.  I'll be happy to test it.

           

          Thank you for your hard work, I totally understand the limited dev time. I will try to do some work for this feature to contribute back.

          Fede

           

           

          Federico Naum added a comment - Hi tommysdk , Thank you for your quick answer. There is no real urgency with - JENKINS-47529 - , but on the other hand, it seems like a great feature that is independent of  JENKINS-45738   and it deserves its own release. Without looking at the actual code changes in  JENKINS-45738 , it seems quite an involved change that could bring some bugs (hopefully not). Personally, I prefer smaller iterations, that are easier to roll back. And in the case, if you need to roll back you only lose one feature, not two.  I would say that if you are happy with the tests in JENKINS-47529  , just go ahead and release it.  I'll be happy to test it.   Thank you for your hard work, I totally understand the limited dev time. I will try to do some work for this feature to contribute back. Fede    

          Federico Naum added a comment -

          Hi,

          I did not get an answer from the Product Manager of Cloudbees, but I can see they have planned "linking downstream jobs"

          Blue ocean roadmap https://jenkins.io/projects/blueocean/roadmap/

          My understanding is that in this Issue (in progress) they will implement the visualization of the downstream jobs stages

          https://issues.jenkins-ci.org/browse/JENKINS-38339

          Federico Naum added a comment - Hi, I did not get an answer from the Product Manager of Cloudbees, but I can see they have planned "linking downstream jobs" Blue ocean roadmap https://jenkins.io/projects/blueocean/roadmap/ My understanding is that in this Issue (in progress) they will implement the visualization of the downstream jobs stages https://issues.jenkins-ci.org/browse/JENKINS-38339

          Tommy Tynjä added a comment -

          fnaum, please note that this issue tracks the visualization in the Delivery Pipeline plugin, which is unrelated to any work done in the Blue Ocean plugin.

          Tommy Tynjä added a comment - fnaum , please note that this issue tracks the visualization in the Delivery Pipeline plugin, which is unrelated to any work done in the Blue Ocean plugin.

          Federico Naum added a comment -

          Hi tommysdk , I'm aware of that. I just posted here because I mentioned that I will find out. 

          Our need (and some other people) is to visualize the full pipeline including the dependant pipeline stages.

          I was about to get some dev time next month to try to tackle that within this plugin, but if that is going to be somehow implemented in blue ocean, then I don't think I'll get that dev time .

          Fede

          Federico Naum added a comment - Hi tommysdk , I'm aware of that. I just posted here because I mentioned that I will find out.  Our need (and some other people) is to visualize the full pipeline including the dependant pipeline stages. I was about to get some dev time next month to try to tackle that within this plugin, but if that is going to be somehow implemented in blue ocean, then I don't think I'll get that dev time . Fede

          Tommy Tynjä added a comment -

          fnaum if you have willingness and get time to implement this in the Delivery Pipeline plugin, I'm more than happy to help you get going on this, just let me know in such case!

          Tommy Tynjä added a comment - fnaum if you have willingness and get time to implement this in the Delivery Pipeline plugin, I'm more than happy to help you get going on this, just let me know in such case!

          Vikram Raina added a comment -

          Hi,

          Do we have this available in BlueOcean?

          I am still seeing this issue on my Jenkins instance via BlueOcean.

          >> If I have a pipeline job (formerly workflow) Foo, that then triggers another pipeline job Bar, it does not get reflected in the delivery pipeline view. 

           

          Thanks,

          Vikram Raina

          Vikram Raina added a comment - Hi, Do we have this available in BlueOcean? I am still seeing this issue on my Jenkins instance via BlueOcean. >> If I have a pipeline job (formerly workflow) Foo, that then triggers another pipeline job Bar, it does not get reflected in the delivery pipeline view.    Thanks, Vikram Raina

          Tommy Tynjä added a comment -

          vikram_raina This ticket is tracking the feature for the delivery pipeline plugin, thus having nothing to do with the Blue Ocean plugin.

          Tommy Tynjä added a comment - vikram_raina This ticket is tracking the feature for the delivery pipeline plugin, thus having nothing to do with the Blue Ocean plugin.

            Unassigned Unassigned
            martinmine Martin
            Votes:
            9 Vote for this issue
            Watchers:
            14 Start watching this issue

              Created:
              Updated: