In some cases it is inappropriate to keep Jenkinsfile in the same repository as project sources, because it has to refer to a particular CI environment that not all users care about. There are other cases where multiple repositories must be consulted during the course of the build. In all these cases currently you can use a standalone Workflow project to run builds, but you lose the advantages of multibranch:
- ability to see the effect of changes made in unmerged branches
- ability to customize Jenkinsfile atomically to match other changes, where relevant
One possibility is another mode for workflow-multibranch, where the script comes not from a coversioned Jenkinsfile in the same repo but from some independent source. So sort of like CpsScmScriptDefinition in terms of picking up the script definition, but like current multibranch in terms of automatic generation of branch projects. You would not be able to make changes to the project sources (e.g., pom.xml) in tandem with changes to the Workflow script—you are back to having a single job definition which must be valid for all active branches—but this might be an acceptable price to pay for the ability to keep specifics of a particular CI environment out of the general project sources.
Another option would be to use for example Git submodules (or Repo), whereby the project sources remain pristine in one repo, and Jenkinsfile for a particular CI environment lives in a super repository with a submodule for the main sources. The issue there is that you need to choose between strict versioning of modules, meaning you need a dummy commit to the super repo to test any source changes; vs. lax versioning (referring to a submodule by a ref rather than hash, if Git even allows that), meaning you lose determinacy in the revision. This also forces use of more exotic SCM features that are not universally available or desirable, though it does allow precise control of the linkage between repository versions.
Perhaps we need an SCMSource which delegates to two or more regular sources and allows you to create a workspace out of multiple repositories, keeping deterministic revisions and allowing configurable options for matching up branches (so you could have a separate repository which could copy branch names from the main repository when necessary to customize Jenkinsfile on a per-branch basis). A potential problem is that some current SCMSource implementations attempt to checkcast the owner's sources list to their own type for various purposes, such as webhooks or status notifications. There needs to be an API in scm-api allowing an SCMSource to indicate that it is a composition.