Thanks for submitting it ndeloof! I totally agree we should streamline it at some point and offer a single solution

I'm a big fan of GitOps and using source code to define what version of jenkins core, dockerfile, plugins and their versions etc that folks wanna use. Then to upgrade a plugin/core its a Pull Request.

It'd be nice to enumerate all the different ways folks configure jenkins core + plugins + versions via source code to generate/setup their Jenkins server.

e.g. there's the classic Dockerfile + plugins.txt approach like this...

https://github.com/garethjevans/jenkins-quickstart01/tree/master/jenkinsConfig/departmentFoo/teamA

or there's Custom War Packager style:

https://github.com/garethjevans/jenkins-cwp-quickstart01/blob/master/packager-config.yml#L11

then there's CasC YAML too.

It shouldn't be too hard to handle those 3 source code formats.  

Then we could create a plugin for updatebot https://github.com/jenkins-x/updatebot/ to automate the generation of PRs for folks using GitOps to upgrade jenkins and/or plugins using either Evergreen,  incrementals:  https://repo.jenkins-ci.org/incrementals/ or releases: https://repo.jenkins-ci.org/releases/

Part of the problem here is that those tools rely on the Update Center layout to bake download URL for version of plugins to install, while this layout is not defined by a spec.

So we need

1. define|document the plugin update center mechanism (metadata signature, how to retrieve hosted plugin versions and download URL, ...)
2. offer an implementation that can be used in various installation context

For (2) I'd like we extract PluginManager into a standalone executable jar so

•  it can run as a CLI jar and been used in docker images as a replacement for install-plugins script
• it can be used by custom war packager as an utility library to implement the same need
• the exact same bits are used by jenkins PluginManager (to enforce consistency)

FWIW, CWP and Evergreen are using (no enforcement/checker yet, so there's probably smallish discrepancies) both use the Bill Of Materials JEP-309: https://github.com/jenkinsci/jep/tree/master/jep/309.

So we already have the standard format defined. So we should all use it. And if by chance it's deemed to miss some things, then JEP-309 should just be amended so that we agree on something. I agree with ndeloof this subject of handling Jenkins dependencies is becoming wasteful, fluffier and fluffier with all these variants, and should become a tool on its own so that people stop rewriting the (square) wheel everywhere.

I've thought about all this just a bit and I wonder technically this should be:

• I find a tool like this would be good to be usable everwhere easily using CLI, so writing it in Go seems a good direction
• at the same time, Jenkins ecosystem and consumers would probably need it in Java. Ideally, in that case, even Jenkins core associated logic could be extracted (cannot judge of the associated complexity though) so that really everyone use the same resolver tool. A CLI usable packaged version of this tooling (fatjar or whatever) could then be used:
  • in the Docker image builds
  • in Evergreen generation of resolved ingest.json from the essentials.yaml
  • from Jenkins-X's updatebot
  • from CWP
  • ...

Like for Maven dependencies, we need a tool able to:

• compute a dependency tree without (or with as few as possible) downloads
• download a whole dependency tree (including optional dependencies or not, here lies some combinatorial too).

There is already a tool which understands how to update versions of components including incremental versions from a specified branch (such as origin master); as per JENKINS-51929, it would “just” need a shim to also read/write the JEP-309 format. That is only going to be useful, however, if the file is actually listing all the plugins you are bundling, as discussed in JENKINS-53506.