Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49651

Extend plugin/update center metadata with known incompatibilities

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Major Major
    • core

      The current plugin metadata allows it to express minimum required versions of core and various other plugins, and this information is mirrored in update center metadata. That is enough for typical scenarios involving compatible upgrades of API components.

      However, we have repeatedly encountered cases where Jenkins developers know that there are combinations of software components which will be broken for users. These include:

      • Removal of deprecated and unsupportable APIs (such as Java methods). We can update all "known" plugins which were using those APIs, but we need to force users to install those updates if they also update the API (core or plugin) component.
      • Major refactorings such as SCM API 2.x which must be upgraded atomically.
      • Behavioral incompatibilities forcing plugin updates, such as cases found in JEP-200.
      • "Tombstone" releases of plugins in which a plugin is replaced by another—we typically wish to force users to update to the tombstone if they are updating related components.

      What all of these cases have in common is that there is some manner of incompatibility, and thus users who upgrade some components must also upgrade other components which would normally be in an inverse dependency relationship (or no expressed dependency at all) to avoid problems.

      Unlike regular plugin dependencies, these relationships cannot be expressed in the downstream plugin manifest. For example, the fact that api 2.0 made incompatible changes which would break installations of impl 1.3 (which depended on api 1.2) cannot be expressed in impl 1.3's manifest—this might have been released years before the rewrite of api was even conceived. It is only the release of impl 2.0 which reveals the upgrade path. So we have two options:

      • Keep the incompatibility marking in the upstream plugin manifest (or some similar metadata resource in the case of core). For example, api 2.0's manifest could express something like:
          Incompatible-With: impl < 2.0, other-impl < 2.0, ...
          

        This has the advantage that the relationships can be enforced by any part of the system, including plugin loading during startup, Docker image builders, JEP-201 if extended to declarative plugin configuration, etc. On the other hand you would need to a release an update to api every time another incompatible implementation is discovered (and a release using api 2.x cut).

      • Advertise all this metadata in update-center.json, like we do for example with notifications of plugin releases containing disclosed security vulnerabilities. The advantage is that we can easily publish new information as it arrives. On the other hand, the metadata is only easily available to the update center code and thus applicable only to traditional GUI upgrades.

      Users participating in JEP-301 "Evergreen" should have less need of a system like this, since they would automatically receive only combinations of components which were pretested and verified compatible. But even Evergreen users could benefit if they are also adding plugins from outside the Essentials set, which might well be known to be broken by changes being made to other components (inside or outside Essentials).

          [JENKINS-49651] Extend plugin/update center metadata with known incompatibilities

          Jesse Glick added a comment -

          Another use case: for JEP-200 we added several whitelist entries to core that actually apply to classes bundled in plugins, to make sure people could do a core upgrade without having to know about plugins to update. With this API, once the plugin releases appear which (for now redundantly) include those entries, we could delete them from core with a notice that the newer core versions are incompatible with the older plugin versions.

          Jesse Glick added a comment - Another use case: for JEP-200 we added several whitelist entries to core that actually apply to classes bundled in plugins, to make sure people could do a core upgrade without having to know about plugins to update. With this API, once the plugin releases appear which (for now redundantly) include those entries, we could delete them from core with a notice that the newer core versions are incompatible with the older plugin versions.

          Maybe go shopping in the RPM dependency documentation for this: http://rpm.org/user_doc/dependencies.html http://rpm.org/user_doc/more_dependencies.html http://rpm.org/user_doc/boolean_dependencies.html

          And here is a guide from openSUSE on how to do package splits, merges, renames, ...:

          https://en.opensuse.org/openSUSE:Package_dependencies#Basic_dependencies

           

          Ing. Christoph Obexer added a comment - Maybe go shopping in the RPM dependency documentation for this: http://rpm.org/user_doc/dependencies.html http://rpm.org/user_doc/more_dependencies.html http://rpm.org/user_doc/boolean_dependencies.html And here is a guide from openSUSE on how to do package splits, merges, renames, ...: https://en.opensuse.org/openSUSE:Package_dependencies#Basic_dependencies  

          Jesse Glick added a comment -

          Yes Linux package manager docs would be a good source of potential use cases to validate a solution against. I think we can get away with a simpler system, since:

          • Jenkins does not really need to deal with platform-specific issues.
          • Most plugin development happens under a single organization’s umbrella. Even if plugin maintainers operate autonomously they are still delivering software targeted toward a core produced by that same organization and using tools and an update center maintained by the same core developers. This is a much simpler situation than Linux, in which both the kernel and various userspace software projects are maintained by completely disparate groups, and multiple distributions compete to package those components independently, picking versions and backports, with these decisions often being made by people other than the upstream developer.
          • Evergreen reduces the pressure to support exotic scenarios that might occur with people running the many LTS flavors offered by various Linux distributions. The Jenkins project produces only one official “long-term” support line, and (until Evergreen) no “pinned” plugin versioning at all.

          Jesse Glick added a comment - Yes Linux package manager docs would be a good source of potential use cases to validate a solution against. I think we can get away with a simpler system, since: Jenkins does not really need to deal with platform-specific issues. Most plugin development happens under a single organization’s umbrella. Even if plugin maintainers operate autonomously they are still delivering software targeted toward a core produced by that same organization and using tools and an update center maintained by the same core developers. This is a much simpler situation than Linux, in which both the kernel and various userspace software projects are maintained by completely disparate groups, and multiple distributions compete to package those components independently, picking versions and backports, with these decisions often being made by people other than the upstream developer. Evergreen reduces the pressure to support exotic scenarios that might occur with people running the many LTS flavors offered by various Linux distributions. The Jenkins project produces only one official “long-term” support line, and (until Evergreen) no “pinned” plugin versioning at all.

          Ing. Christoph Obexer added a comment - - edited

          I think we can get away with a simpler system

          Sure, I just think that with Requirements and Incompatible-With (aka Conflicts) you almost have everything in place to enable:

          • Renames
          • Splits
          • Merges

          So I added the RPM documentation links here to save the implementer the time to come up with compatible semantics themselves.

          Thinking about this some more I think Obsoletes and also handling core updates in the same resolver would net you almost everything. (IIRC KK talked about wanting to update core like the plugins)

          Ing. Christoph Obexer added a comment - - edited I think we can get away with a simpler system Sure, I just think that with Requirements and Incompatible-With (aka Conflicts) you almost have everything in place to enable: Renames Splits Merges So I added the RPM documentation links here to save the implementer the time to come up with compatible semantics themselves. Thinking about this some more I think Obsoletes and also handling core updates in the same resolver would net you almost everything. (IIRC KK talked about wanting to update core like the plugins)

          Jesse Glick added a comment -

          JENKINS-31096 is another example of where this would be useful.

          Jesse Glick added a comment - JENKINS-31096 is another example of where this would be useful.

          Jesse Glick added a comment -

          JENKINS-41854 is another example (DynamicContext needed a workflow-cps update to be interpreted).

          Jesse Glick added a comment - JENKINS-41854 is another example ( DynamicContext needed a workflow-cps update to be interpreted).

          Jesse Glick added a comment -

          JENKINS-5303 would be a major use case.

          Jesse Glick added a comment - JENKINS-5303 would be a major use case.

          Jesse Glick added a comment -

          Some more recent examples:

          • git-client 4.x pulls in incompatible changes to JGit (e.g., renamed packages) which break some features in other plugins.
          • kubernetes-api can be updated to a 6.x version but this is incompatible in some respects, and e.g. kubernetes needs to be updated to match.

          Jesse Glick added a comment - Some more recent examples: git-client 4.x pulls in incompatible changes to JGit (e.g., renamed packages) which break some features in other plugins. kubernetes-api can be updated to a 6.x version but this is incompatible in some respects, and e.g. kubernetes needs to be updated to match.

          Jesse Glick added a comment -

          JENKINS-70528 is another example (workflow-durable-task-step upgrade ought to be matched by a workflow-basic-steps upgrade).

          Jesse Glick added a comment - JENKINS-70528 is another example ( workflow-durable-task-step upgrade ought to be matched by a workflow-basic-steps upgrade).

            Unassigned Unassigned
            jglick Jesse Glick
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: