Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-48510

Pipeline Milestone step: Adding parameter for milestone classification

      Adding a parameter for milestone classification, so an older build will not override a newer build if only they belong to the same group.

      The parameter would group different milestones. The ordinal parameter is used to order the milestones in the group. The abort behavior in a group is the same as now.
      extended simplified script from above:
      sleep PARAMETER
      milestone label: 'mileStone', ordinalGroup: projectName
      sleep 5

      Example1:
      project1: started with PARAMETER = 20 (sleeps 20 sec), ordinalGroup: 'p1'
      project2: started after project1 with PARAMETER = 5, ordinalGroup: 'p2'
      --> project1 and project2 run in the same job

      Example2:
      project1: started with PARAMETER = 20 (sleeps 20 sec), ordinalGroup: 'p1'
      project2: started after project1 with PARAMETER = 5, ordinalGroup: 'p2'
      project1: started after project2 with PARAMETER = 7 (sleeps 7 sec), ordinalGroup: 'p1'
      -> first p1 is aborted through 2nd p1 build, p2 runs through.

          [JENKINS-48510] Pipeline Milestone step: Adding parameter for milestone classification

          Carlos Rodríguez López created issue -
          Carlos Rodríguez López made changes -
          Description Original: Adding a parameter for milestone classification, so an older build will not override a newer build if only they belong to the same group.

          The parameter groups the different milestones. The ordinal parameter is used to order the milestones in the group. The abort behavior in a group is the same as now.
          extended simplified script from above:
          sleep PARAMETER
          milestone label: 'mileStone', ordinalGroup: projectName
          sleep 5

          Example1:
          project1: started with PARAMETER = 20 (sleeps 20 sec), ordinalGroup: 'p1'
          project2: started after project1 with PARAMETER = 5, ordinalGroup: 'p2'
          --> project1 and project2 run in the same job

          Example2:
          project1: started with PARAMETER = 20 (sleeps 20 sec), ordinalGroup: 'p1'
          project2: started after project1 with PARAMETER = 5, ordinalGroup: 'p2'
          project1: started after project2 with PARAMETER = 7 (sleeps 7 sec), ordinalGroup: 'p1'
          -> first p1 is aborted through 2nd p1 build, p2 runs through.
          New: Adding a parameter for milestone classification, so an older build will not override a newer build if only they belong to the same group.

          The parameter would group different milestones. The ordinal parameter is used to order the milestones in the group. The abort behavior in a group is the same as now.
          extended simplified script from above:
          sleep PARAMETER
          milestone label: 'mileStone', ordinalGroup: projectName
          sleep 5

          Example1:
          project1: started with PARAMETER = 20 (sleeps 20 sec), ordinalGroup: 'p1'
          project2: started after project1 with PARAMETER = 5, ordinalGroup: 'p2'
          --> project1 and project2 run in the same job

          Example2:
          project1: started with PARAMETER = 20 (sleeps 20 sec), ordinalGroup: 'p1'
          project2: started after project1 with PARAMETER = 5, ordinalGroup: 'p2'
          project1: started after project2 with PARAMETER = 7 (sleeps 7 sec), ordinalGroup: 'p1'
          -> first p1 is aborted through 2nd p1 build, p2 runs through.

          The proposed ordinalGroup feature would be a workaround for a good practice: one branch, one job. Having 2 projects/branches been built by the same job is not desirable. Avoiding this is the whole point of multibranch projects. Each project should have its own multibranch job. If projects live in branches of the same repository, then use specific {{Jenkinsfile}}s in branches.

          Antonio Muñiz added a comment - The proposed ordinalGroup feature would be a workaround for a good practice: one branch, one job. Having 2 projects/branches been built by the same job is not desirable. Avoiding this is the whole point of multibranch projects. Each project should have its own multibranch job. If projects live in branches of the same repository, then use specific {{Jenkinsfile}}s in branches.

          Jesse Glick added a comment -

          I do not see a clear use case for this. The example given in the issue description is artificial so that does not help give context.

          Jesse Glick added a comment - I do not see a clear use case for this. The example given in the issue description is artificial so that does not help give context.

          amuniz: I would like to understand why it is not desirable to have 2 projects built by the same job? In my opinion it is a limitation as

          • additional maintenance needed (DRY principle is not fulfilled)
          • higher dependency to Jenkins / more Jenkins centric view
          • e.g. multi branch has limitations (e.g. try to handle Subversion and GIT in one job)

          For me, Jenkins is a infrastructure tool. I use it to define where data shall be processed but not how. Representation of the result is not done with Jenkins (could this be the main reason to have one job per project?)

          E.g. my generalized workflow

          • build is triggered by external tool
          • build pulls data depending on provided parameters
          • build processes data
          • build pushes result to other tools (e.g. dashboard, artifactory ...)

          So in case of double builds / same data shall be processed twice I would like to abort one build. Unfortunately, it is not possible with the current implementation of the milestone.

          Tobias Peitzsch added a comment - amuniz : I would like to understand why it is not desirable to have 2 projects built by the same job? In my opinion it is a limitation as additional maintenance needed (DRY principle is not fulfilled) higher dependency to Jenkins / more Jenkins centric view e.g. multi branch has limitations (e.g. try to handle Subversion and GIT in one job) For me, Jenkins is a infrastructure tool. I use it to define where data shall be processed but not how. Representation of the result is not done with Jenkins (could this be the main reason to have one job per project?) E.g. my generalized workflow build is triggered by external tool build pulls data depending on provided parameters build processes data build pushes result to other tools (e.g. dashboard, artifactory ...) So in case of double builds / same data shall be processed twice I would like to abort one build. Unfortunately, it is not possible with the current implementation of the milestone.

          additional maintenance needed (DRY principle is not fulfilled)

          Not necessarily. Use a shared library loaded from a separated SCM, then create a really simple Jenkinsfile on each project (this is the approach of the Jenkins Plugins project, this is the Jenkinsfile for the milestone plugin for example).

          Maintenance would be mostly centralized in the shared library.

          higher dependency to Jenkins / more Jenkins centric view

          Can you elaborate? I see the same grade of dependency in both approaches.

          multi branch has limitations (e.g. try to handle Subversion and GIT in one job)

          You can use more than one call to `checkout` step to get Git and Subversion workspaces in the same pipeline. What's the limitation?

          I use it to define where data shall be processed but not how

          Pipelines are designed to define "how" the project is built (among other things), not sure what other use you could do of it.

          So in case of double builds

          Why would you have double builds? A commit in a specific repository should trigger a single build in the corresponding job.

          Antonio Muñiz added a comment - additional maintenance needed (DRY principle is not fulfilled) Not necessarily. Use a shared library loaded from a separated SCM, then create a really simple Jenkinsfile on each project (this is the approach of the Jenkins Plugins project, this is the Jenkinsfile for the milestone plugin for example ). Maintenance would be mostly centralized in the shared library. higher dependency to Jenkins / more Jenkins centric view Can you elaborate? I see the same grade of dependency in both approaches. multi branch has limitations (e.g. try to handle Subversion and GIT in one job) You can use more than one call to `checkout` step to get Git and Subversion workspaces in the same pipeline. What's the limitation? I use it to define where data shall be processed but not how Pipelines are designed to define "how" the project is built (among other things), not sure what other use you could do of it. So in case of double builds Why would you have double builds? A commit in a specific repository should trigger a single build in the corresponding job.

          Markus Maga added a comment -

          I've also been looking for something similar. Our use case is that we have a shared job for deployment, we have multiple multibranch jobs which all trigger a parameterised pipeline job. In this pipeline job I'd like to combine milestones & locks to prevent running simultaneous deployments for the same app to the same environment. I can use locks to only deploy one of each app at a time (different apps can deploy in parallel). But it would be nice to use the milestone with the suggested ordinalGroup in combination with locks using inversePrecedence: true. Do you have an alternative solution to this amuniz? I see no need to have to create new jobs for each possible application & environment combination.

          Did you find any solution tpeitzsch?

          Markus Maga added a comment - I've also been looking for something similar. Our use case is that we have a shared job for deployment, we have multiple multibranch jobs which all trigger a parameterised pipeline job. In this pipeline job I'd like to combine milestones & locks to prevent running simultaneous deployments for the same app to the same environment. I can use locks to only deploy one of each app at a time (different apps can deploy in parallel). But it would be nice to use the milestone with the suggested ordinalGroup in combination with locks using inversePrecedence: true. Do you have an alternative solution to this amuniz ? I see no need to have to create new jobs for each possible application & environment combination. Did you find any solution tpeitzsch ?

          Jesse Glick added a comment -

          prevent running simultaneous deployments for the same app to the same environment

          This sort of logic should probably be moved out of Jenkins itself into a deployment tool.

          Jesse Glick added a comment - prevent running simultaneous deployments for the same app to the same environment This sort of logic should probably be moved out of Jenkins itself into a deployment tool.

          Markus Maga added a comment -

          jglick perhaps, suggestions welcome 

          But having milestones that can be bound to one or more parameters makes sense to me.

          But I don't see a reason not to offer this flexibility, it would give more super powers to parameterised builds

          Markus Maga added a comment - jglick  perhaps, suggestions welcome  But having milestones that can be bound to one or more parameters makes sense to me. But I don't see a reason not to offer this flexibility, it would give more super powers to parameterised builds

          To be more concrete, the best practice "one branch, one job" will not work in large companies with separated teams for SW development and infrastructure development due to job maintenance (E.g. are the latest released script versions used by the SW development teams). Here the build script version must be specified centrally. Please think about more than 1000 different SW development projects / repositories with at least 10 branches (note: multibranch is not possible, legacy SCM) but only 4 different workflows.

          It is a hard job to change some global configurations for more 10000 jobs  (even with Jobs DSL).

          To reduce the amount of jobs I want to use one job per released pipeline script version for all related projects / repositories (or 10000 jobs running in one Job). Unfortunately the milestone step cannot be used.

          flydiverny: Unfortunately not. I haven't found a real solution for this issue :/. E.g. the Job creation via Jobs DSL introduce additional infrastructure to handle the build parameters (some of them can be changed by the users and would be overwritten by the Jobs DSL)

          Tobias Peitzsch added a comment - To be more concrete, the best practice "one branch, one job" will not work in large companies with separated teams for SW development and infrastructure development due to job maintenance (E.g. are the latest released script versions used by the SW development teams). Here the build script version must be specified centrally. Please think about more than 1000 different SW development projects / repositories with at least 10 branches (note: multibranch is not possible, legacy SCM) but only 4 different workflows. It is a hard job to change some global configurations for more 10000 jobs  (even with Jobs DSL). To reduce the amount of jobs I want to use one job per released pipeline script version for all related projects / repositories (or 10000 jobs running in one Job). Unfortunately the milestone step cannot be used. flydiverny : Unfortunately not. I haven't found a real solution for this issue :/. E.g. the Job creation via Jobs DSL introduce additional infrastructure to handle the build parameters (some of them can be changed by the users and would be overwritten by the Jobs DSL)

            amuniz Antonio Muñiz
            carlosrodlop Carlos Rodríguez López
            Votes:
            6 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated: