Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30519

Declarative job properties in multibranch

    XMLWordPrintable

Details

    Description

      Since a multibranch project creates all of its subprojects (WorkflowJob), any customizations you may want to make on them must be defined at the multibranch level. So far this has been handled with BranchProperty, but that is undesirable in most cases for several reasons:

      1. It defeats the idea of code-as-configuration: you are back to keeping important job configuration in $JENKINS_HOME/jobs/*/config.xml, rather than your own SCM.
      2. Adjusting properties on a per-branch basis is awkward. You can use NamedExceptionsBranchPropertyStrategy, but this is extra work to configure if you have a lot of branches, and again cannot be adjusted outside the Jenkins configuration UI, or set right away for newly created branches (unless you follow a strict naming convention).
      3. While we can hardcode support for certain aspects of static job configuration, as we do with WorkflowParameterDefinitionBranchProperty, BuildRetentionBranchProperty, and RateLimitBranchProperty¹, this is not extensible to JobProperty implementations defined by other plugins.

      kohsuke came up with a novel idea to solve this for workflow-multibranch: add a Step which simply (re-)sets the List<JobProperty> associated with the job it is run on. You need to run one build to get the properties set, but that is usually not onerous since multibranch will always trigger an initial build for every branch anyway, and few properties really need to be set in the short interval between the new project being created and the first build getting as far as this step. Jenkinsfile can feel free to hardcode certain properties, or dynamically include some, or configure some according to branch name, etc. While it may sound weird at first blush to have a dynamic script update static properties, this is actually analogous to how Workflow supports SCM polling and triggers without the need for static SCM configuration.

      So then we need to:

      1. define the new step
      2. delete WorkflowParameterDefinitionBranchProperty (subsumed by the new system)
      3. retrofit various fields of WorkflowJob to be configured instead by JobProperty
        1. buildDiscarder (suppress the existing BuildRetentionBranchProperty)
        2. triggers (note that SCMTrigger, GitHubTrigger, etc. would be superfluous in a multibranch project, since SCMSource covers that need, but TimerTrigger and miscellaneous plugin contributions may be wanted)
        3. quietPeriod (if anyone cares)
        4. concurrentBuild
        5. authToken²

      TBD whether the literate plugin would want to follow suit, or stick with the existing BranchProperty-driven approach.

      Some "closed-class" BranchProperty implementations will still be needed, such as UntrustedBranchProperty or a Workflow equivalent TBD.

      ¹Probably this should never have been a BranchProperty at all; it has no apparent logical dependency on multibranch. Really it should be a separate plugin with a general-purpose JobProperty.

      ²Trickier since it is normally a per-Job secret. Fine for it to be the same for all children of a WorkflowMultiBranchProject, but should of course not be hardcoded in the script. Could use StringCredentials, though this will not work well with automatic repository scanning: you would not want to manually create a token for each repository. Nicer would be to keep a HMACConfidentialKey, with the per-project token somehow displayed to anyone with Item.BUILD.

      Attachments

        Issue Links

          Activity

            abayer Andrew Bayer added a comment -

            So, before I file a JIRA, just want to make sure - triggers are still not possible, right? So there's no way to currently have a Multibranch branch Jenkinsfile that gets executed @daily, etc?

            abayer Andrew Bayer added a comment - So, before I file a JIRA, just want to make sure - triggers are still not possible, right? So there's no way to currently have a Multibranch branch Jenkinsfile that gets executed @daily , etc?
            oleg_nenashev Oleg Nenashev added a comment -

            I suspect that you can create an external periodic FreeStyle project, which would trigger the desired branches (dirty hack)

            oleg_nenashev Oleg Nenashev added a comment - I suspect that you can create an external periodic FreeStyle project, which would trigger the desired branches (dirty hack)
            jglick Jesse Glick added a comment -

            I filed JENKINS-34005 for it.

            jglick Jesse Glick added a comment - I filed JENKINS-34005 for it.
            tknerr Torben Knerr added a comment -

            jglick stephenconnolly desperately looking for a way to configure the build retention strategy on a multibranch pipeline job level. It looks like this JIRA issue directly addresses that, but it is totally unclear to me how to actually make use of it.

            Is this possible right now to define job properties on a multibranch level?

            I see there is a "Properties" section in the UI of the multibranch pipeline job configuration, but there is no way to add items via the UI.

            Then there is the "Property Strategy" under "Branch Sources", which also sounds plausible, but the only property you can add here is "Suppress automatic SCM triggering", but none of the others available in https://github.com/jenkinsci/branch-api-plugin/tree/master/src/main/java/jenkins/branch *Property.java

            Any advice how to deal with this?

            P.S.: I'd love to do that via JobDSL, but as I don't see any way to do that via the UI I doubt it's possible via JobDSL...

            tknerr Torben Knerr added a comment - jglick stephenconnolly desperately looking for a way to configure the build retention strategy on a multibranch pipeline job level. It looks like this JIRA issue directly addresses that, but it is totally unclear to me how to actually make use of it. Is this possible right now to define job properties on a multibranch level? I see there is a "Properties" section in the UI of the multibranch pipeline job configuration, but there is no way to add items via the UI. Then there is the "Property Strategy" under "Branch Sources", which also sounds plausible, but the only property you can add here is "Suppress automatic SCM triggering", but none of the others available in https://github.com/jenkinsci/branch-api-plugin/tree/master/src/main/java/jenkins/branch  *Property.java Any advice how to deal with this? P.S.: I'd love to do that via JobDSL, but as I don't see any way to do that via the UI I doubt it's possible via JobDSL...
            tknerr Torben Knerr added a comment -

            I finally found a way to do that via the auto-generated JobDSL. After generating the Jobs these properties are also visible in the UI, so it seems it's simply a UI bug that prevented me from setting the properties via the "Property Strategy" under "Branch Sources".

            The JobDSL API viewer shows the generated JobDSL here:
            http://your-jenkins:8080/plugin/job-dsl/api-viewer/index.html#path/multibranchPipelineJob-branchSources-branchSource

            Example JobDSL snippet for a multibranchPipelineJob with BuildRetentionBranchProperty configured:
            https://gist.github.com/tknerr/c79a514db4bdbfb4956aaf0ee53836c8 

             

             

            tknerr Torben Knerr added a comment - I finally found a way to do that via the auto-generated JobDSL. After generating the Jobs these properties are also visible in the UI, so it seems it's simply a UI bug that prevented me from setting the properties via the "Property Strategy" under "Branch Sources". The JobDSL API viewer shows the generated JobDSL here: http://your-jenkins:8080/plugin/job-dsl/api-viewer/index.html#path/multibranchPipelineJob-branchSources-branchSource Example JobDSL snippet for a multibranchPipelineJob with BuildRetentionBranchProperty configured: https://gist.github.com/tknerr/c79a514db4bdbfb4956aaf0ee53836c8      

            People

              jglick Jesse Glick
              jglick Jesse Glick
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: