Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-26210

Configure Jobs to be hidden from the delivery pipeline

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      It would be cool to be able to configure Jobs to be hidden from the delivery pipeline. I have the problem, that I have multiple Deploy Jobs which are all using the same parameterized MultiJob. The Sub-Tasks of the MultiJob are displayed totally out of order and are not important for the high level pipeline view.

        Attachments

          Issue Links

            Activity

            mapo Marc Popp created issue -
            mapo Marc Popp made changes -
            Field Original Value New Value
            Description It would be cool to be able to configure Jobs to be hidden from the delivery pipeline. I have the problem, that I have multiple Deploy Jobs which are all using the same parameterized MultiJob. The Sub-Tasks of the MultiJob are displayed totally out of order. It would be cool to be able to configure Jobs to be hidden from the delivery pipeline. I have the problem, that I have multiple Deploy Jobs which are all using the same parameterized MultiJob. The Sub-Tasks of the MultiJob are displayed totally out of order and not important for the high level pipeline view.
            Hide
            patbos Patrik Boström added a comment -

            It is an interesting idea to have the possibility hide jobs.
            Im also curious to understand the problem you have with the Multijob being displayed in the wrong order. Can you give some more details?

            Show
            patbos Patrik Boström added a comment - It is an interesting idea to have the possibility hide jobs. Im also curious to understand the problem you have with the Multijob being displayed in the wrong order. Can you give some more details?
            mapo Marc Popp made changes -
            Description It would be cool to be able to configure Jobs to be hidden from the delivery pipeline. I have the problem, that I have multiple Deploy Jobs which are all using the same parameterized MultiJob. The Sub-Tasks of the MultiJob are displayed totally out of order and not important for the high level pipeline view. It would be cool to be able to configure Jobs to be hidden from the delivery pipeline. I have the problem, that I have multiple Deploy Jobs which are all using the same parameterized MultiJob. The Sub-Tasks of the MultiJob are displayed totally out of order and are not important for the high level pipeline view.
            Hide
            mapo Marc Popp added a comment -

            I have a multijob 'Deploy' that is called with parameters to deploy a host. It looks like that:

            Name: Deploy
              - Phase 1
                - Job Step 1
              - Phase 2
                - Job Step 2
                - Job Step 3
            

            Then I have one multijob per host, that calls the common multijob 'Deploy' with parameters:

            Deploy_XYZ
              - Phase 1
                - Call Parameterised Job: Deploy
            

            If I create the pipeline for one job everything is fine:

            Deploy_XYZ -> Deploy -+-> Step 1
                                  |
                                  +-> Step 2
                                  |
                                  +-> Step 3
            

            Also, in the the pipeline view I'm not interested in the common multijob 'Deploy' and the Steps, but just something like:

            +-------------+   +-------------+
            | ENV - QS    |-->| ENV - PROD  |
            | Deploy_XYZ1 |   | Deploy_ABC1 |
            | Deploy_XYZ2 |   | Deploy_ABC2 |
            | Deploy_XYZ3 |   | Deploy_ABC3 |
            +-------------+   +-------------+
            

            I've attached a screenshot of how the pipeline looks like, with 2 ENVs and 1 Deploy Job each.

            Show
            mapo Marc Popp added a comment - I have a multijob 'Deploy' that is called with parameters to deploy a host. It looks like that: Name: Deploy - Phase 1 - Job Step 1 - Phase 2 - Job Step 2 - Job Step 3 Then I have one multijob per host, that calls the common multijob 'Deploy' with parameters: Deploy_XYZ - Phase 1 - Call Parameterised Job: Deploy If I create the pipeline for one job everything is fine: Deploy_XYZ -> Deploy -+-> Step 1 | +-> Step 2 | +-> Step 3 Also, in the the pipeline view I'm not interested in the common multijob 'Deploy' and the Steps, but just something like: +-------------+ +-------------+ | ENV - QS |-->| ENV - PROD | | Deploy_XYZ1 | | Deploy_ABC1 | | Deploy_XYZ2 | | Deploy_ABC2 | | Deploy_XYZ3 | | Deploy_ABC3 | +-------------+ +-------------+ I've attached a screenshot of how the pipeline looks like, with 2 ENVs and 1 Deploy Job each.
            Hide
            mapo Marc Popp added a comment -

            Screenshot with 2 ENVs and 1 Deploy Job each.

            Show
            mapo Marc Popp added a comment - Screenshot with 2 ENVs and 1 Deploy Job each.
            mapo Marc Popp made changes -
            Hide
            rg Russell Gallop added a comment -

            This would be useful for me also. I use multijobs to control a large number of subjobs but would like these hidden from the top level pipeline.

            I would also like to hide jobs when one pipeline feeds into another, I'm not sure whether this would be the same issue.
            As a simplified example, I would like 3 separate pipelines for 2 components and an integration of them:
            Build A->Test A
            Build B->Test B
            Integrate A&B->Test A&B

            but because Test A and Test B both trigger Integrate A&B I end up with:
            Build A->Test A->Integrate A&B->Test A&B
            Build B->Test B->Integrate A&B->Test A&B
            Integrate A&B->Test A&B

            Show
            rg Russell Gallop added a comment - This would be useful for me also. I use multijobs to control a large number of subjobs but would like these hidden from the top level pipeline. I would also like to hide jobs when one pipeline feeds into another, I'm not sure whether this would be the same issue. As a simplified example, I would like 3 separate pipelines for 2 components and an integration of them: Build A->Test A Build B->Test B Integrate A&B->Test A&B but because Test A and Test B both trigger Integrate A&B I end up with: Build A->Test A->Integrate A&B->Test A&B Build B->Test B->Integrate A&B->Test A&B Integrate A&B->Test A&B
            Hide
            dawidmalina Dawid Malinowski added a comment - - edited

            I think I've solved this issue with JENKINS-29806

            I had same issue like you and I've ended up with splitted big pipeline in to two piplines (build pipeline and deploy pipeline).

            Show
            dawidmalina Dawid Malinowski added a comment - - edited I think I've solved this issue with JENKINS-29806 I had same issue like you and I've ended up with splitted big pipeline in to two piplines (build pipeline and deploy pipeline).
            dawidmalina Dawid Malinowski made changes -
            Link This issue is related to JENKINS-29806 [ JENKINS-29806 ]
            Hide
            prstell Patrick Ryan added a comment -

            I don't think JENKINS-29806 is the same thing. I, too, would like to able to hide jobs. In my pipeline, I have util jobs that get called multiple times from multiple stages. Those jobs don't need to be shown in the pipeline and when they are, they are put in a stage far to the right with lines cutting thru other stages to get there.

            Show
            prstell Patrick Ryan added a comment - I don't think JENKINS-29806 is the same thing. I, too, would like to able to hide jobs. In my pipeline, I have util jobs that get called multiple times from multiple stages. Those jobs don't need to be shown in the pipeline and when they are, they are put in a stage far to the right with lines cutting thru other stages to get there.
            Hide
            dawidmalina Dawid Malinowski added a comment -

            I get your point, but one question that I have. What if you will have
            A -> B -> C and B will be job that you would like to hide. We will display:
            A -> C but what if B job will fail? You will lose ability to see why you pipeline stuck.

            Show
            dawidmalina Dawid Malinowski added a comment - I get your point, but one question that I have. What if you will have A -> B -> C and B will be job that you would like to hide. We will display: A -> C but what if B job will fail? You will lose ability to see why you pipeline stuck.
            Hide
            prstell Patrick Ryan added a comment -

            That would be in the design. One shouldn't do that but should be able to.
            This might help with another issue as well:
            If Job A calls Job B as a build step and is blocked until B completes, you can see the progress of Job B in Job A (Job A is running while Job B is running). The issue is if Job A takes 5 minutes by itself and Job B takes 20 minutes. Then the pipeline show Job A taking 25 minutes and Job B taking 20 minutes, which is accurate, however it shows Total build time as 45 minutes, even though it only took 25 minutes. It adds A and B even though A includes B.

            Show
            prstell Patrick Ryan added a comment - That would be in the design. One shouldn't do that but should be able to. This might help with another issue as well: If Job A calls Job B as a build step and is blocked until B completes, you can see the progress of Job B in Job A (Job A is running while Job B is running). The issue is if Job A takes 5 minutes by itself and Job B takes 20 minutes. Then the pipeline show Job A taking 25 minutes and Job B taking 20 minutes, which is accurate, however it shows Total build time as 45 minutes, even though it only took 25 minutes. It adds A and B even though A includes B.
            Hide
            amanica amanica added a comment -

            Yea, just add a warning to the description saying that you will obviously loose sight of these jobs even if they are blocked.
            For my use case I trigger some housekeeping jobs all over and they don't affect the pipeline at all except now it messes up the pipeline display
            Another option is to actually show any 'hidden' jobs when they are actually running but hide them otherwise, this would make the pipeline reshuffle all the time which will not be very nice though

            Show
            amanica amanica added a comment - Yea, just add a warning to the description saying that you will obviously loose sight of these jobs even if they are blocked. For my use case I trigger some housekeeping jobs all over and they don't affect the pipeline at all except now it messes up the pipeline display Another option is to actually show any 'hidden' jobs when they are actually running but hide them otherwise, this would make the pipeline reshuffle all the time which will not be very nice though
            Hide
            pmatignon Patrice Matignon added a comment -

            @amanica : regarding your last suggestion "Another option is to actually show any 'hidden' jobs when they are actually running but hide them otherwise, this would make the pipeline reshuffle all the time which will not be very nice though" : I'm not crazy about the idea. I think they should be always hidden, or maybe a compromise is to allow a toggle on the UI "show/hide hidden jobs" (default: hidden").
            If the concern is that the pipeline instance may be "active" because hidden jobs are still running (but the display pipeline instance shows no activity / all jobs complete), then maybe there could be a small blinking indicator of some kind, when (downstream) jobs are executing though not shown. Could be combined with the show/hide button discussed above. But for my use case, it would be superfluous.

            Show
            pmatignon Patrice Matignon added a comment - @amanica : regarding your last suggestion " Another option is to actually show any 'hidden' jobs when they are actually running but hide them otherwise, this would make the pipeline reshuffle all the time which will not be very nice though " : I'm not crazy about the idea. I think they should be always hidden, or maybe a compromise is to allow a toggle on the UI "show/hide hidden jobs" (default: hidden"). If the concern is that the pipeline instance may be "active" because hidden jobs are still running (but the display pipeline instance shows no activity / all jobs complete), then maybe there could be a small blinking indicator of some kind, when (downstream) jobs are executing though not shown. Could be combined with the show/hide button discussed above. But for my use case, it would be superfluous.
            Hide
            pskumar448 Suresh Kumar added a comment -

            Pull Request for this change has submiited

            Show
            pskumar448 Suresh Kumar added a comment - Pull Request for this change has submiited
            Hide
            tommysdk Tommy Tynjä added a comment -

            Suresh Kumar;

            I've now gone through your pull request and found the following problems.

            If I enable delivery pipeline configuration for a job, omit stage and task name but mark the task as hidden, the job will still be visible in the pipeline.

            If the stage and task names are present and the job is marked as hidden, the stage will still be shown even though no jobs for that stage are visible, which is ambiguous and looks weird.

            I'm still not convinced about the feature in general as I don't find the visualisation intuitive, if jobs are hidden. I understand the use cases, but I'm afraid it will cause unnecessary confusion if a hidden job fails or if a hidden job has additional logic to it which should be considered from a pipeline point of view.

            Show
            tommysdk Tommy Tynjä added a comment - Suresh Kumar ; I've now gone through your pull request and found the following problems. If I enable delivery pipeline configuration for a job, omit stage and task name but mark the task as hidden, the job will still be visible in the pipeline. If the stage and task names are present and the job is marked as hidden, the stage will still be shown even though no jobs for that stage are visible, which is ambiguous and looks weird. I'm still not convinced about the feature in general as I don't find the visualisation intuitive, if jobs are hidden. I understand the use cases, but I'm afraid it will cause unnecessary confusion if a hidden job fails or if a hidden job has additional logic to it which should be considered from a pipeline point of view.
            Hide
            pskumar448 Suresh Kumar added a comment -

            Tommy Tynjä
            I know the 2 issues which you have mentioned. I thought I may get some more ideas how to handle this feature in more appropriate way.
            If this is ok means I will fix these issues and move forward adding anything needed.

            Thanks for reviewing and giving feedback. Let me know how to handle this.

            Show
            pskumar448 Suresh Kumar added a comment - Tommy Tynjä I know the 2 issues which you have mentioned. I thought I may get some more ideas how to handle this feature in more appropriate way. If this is ok means I will fix these issues and move forward adding anything needed. Thanks for reviewing and giving feedback. Let me know how to handle this.
            Hide
            pskumar448 Suresh Kumar added a comment -

            Hi Tommy Tynjä
            Fixed the above 2 problems.
            Merged updates from master, made this up-to date, increased code coverage based on the added logic.

            Review it and let me know feedback.

            Show
            pskumar448 Suresh Kumar added a comment - Hi Tommy Tynjä Fixed the above 2 problems. Merged updates from master, made this up-to date, increased code coverage based on the added logic. Review it and let me know feedback.
            Hide
            tommysdk Tommy Tynjä added a comment -

            Thank you for the feature request.

            After trying out the proposed solutions and discussing this among the maintainers, we have decided to decline this feature request. The reason for this is manily because the feature doesn't provide an intuitive visualisation which will cause unnecessary confusion if hidden jobs fail. An intuitive pipeline visualisation is key for this plugin. We also don't think the reasoning for hiding jobs that are generic jobs that are used multiple times from multiple stages is reason enough for this feature. It rather feels like an anti-pattern, that the generic jobs are manually created and / or maintained and for that reason job duplication is desired. If jobs are created with Jenkins Job Builder or JobDSL, that problem should be solved, as generic jobs can be avoided by generating unique jobs from a generic template.

            We are of course open to further discussions if anyone has a good way to address the concerns mentioned.

            Show
            tommysdk Tommy Tynjä added a comment - Thank you for the feature request. After trying out the proposed solutions and discussing this among the maintainers, we have decided to decline this feature request. The reason for this is manily because the feature doesn't provide an intuitive visualisation which will cause unnecessary confusion if hidden jobs fail. An intuitive pipeline visualisation is key for this plugin. We also don't think the reasoning for hiding jobs that are generic jobs that are used multiple times from multiple stages is reason enough for this feature. It rather feels like an anti-pattern, that the generic jobs are manually created and / or maintained and for that reason job duplication is desired. If jobs are created with Jenkins Job Builder or JobDSL, that problem should be solved, as generic jobs can be avoided by generating unique jobs from a generic template. We are of course open to further discussions if anyone has a good way to address the concerns mentioned.
            tommysdk Tommy Tynjä made changes -
            Assignee Patrik Boström [ patbos ] Tommy Tynjä [ tommysdk ]
            Resolution Won't Fix [ 2 ]
            Status Open [ 1 ] Resolved [ 5 ]
            Hide
            pmatignon Patrice Matignon added a comment -

            FWIW, I think this is closed a bit unfairly.
            This proposed code change doesn't alter the default behavior, where all the jobs will be shown. It only kicks in when the pipeline is configured to specifically hide jobs.
            I think the use case(s) for it were laid out pretty convincingly, I know that some jobs in my pipelines have no value for the target audience and are therefore distracting/confusing.
            Also, there were some suggestions to ease some of the given concerns - mine was to have a blinking indicator ("hidden job running") on the pipeline instance, when a hidden job is executing in the background. It can be easily adapted to show a status (red/blue) as well as a toggle button to show the hidden jobs if desired. Design-wise, if a job is preferred to be hidden, it probably doesn't matter to the pipeline audience whether it succeeded or failed.
            For my use cases, the jobs I want to hide are not in-between visible jobs, they are side-branches or leaves of the pipeline tree anyway. It would not be making a lot of sense to hide jobs on the pipeline critical path(s) I'd have to agree.

            Show
            pmatignon Patrice Matignon added a comment - FWIW, I think this is closed a bit unfairly. This proposed code change doesn't alter the default behavior, where all the jobs will be shown. It only kicks in when the pipeline is configured to specifically hide jobs. I think the use case(s) for it were laid out pretty convincingly, I know that some jobs in my pipelines have no value for the target audience and are therefore distracting/confusing. Also, there were some suggestions to ease some of the given concerns - mine was to have a blinking indicator ("hidden job running") on the pipeline instance, when a hidden job is executing in the background. It can be easily adapted to show a status (red/blue) as well as a toggle button to show the hidden jobs if desired. Design-wise, if a job is preferred to be hidden, it probably doesn't matter to the pipeline audience whether it succeeded or failed. For my use cases, the jobs I want to hide are not in-between visible jobs, they are side-branches or leaves of the pipeline tree anyway. It would not be making a lot of sense to hide jobs on the pipeline critical path(s) I'd have to agree.
            rtyler R. Tyler Croy made changes -
            Workflow JNJira [ 160201 ] JNJira + In-Review [ 196363 ]

              People

              Assignee:
              tommysdk Tommy Tynjä
              Reporter:
              mapo Marc Popp
              Votes:
              7 Vote for this issue
              Watchers:
              13 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: