Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-57023

Split external library functionality into its own plugin

      (Splitting this task out of JENKINS-55582, which I realized was not really a prerequisite anyway.)

      Split the older and deprecated half of workflow-cps-global-lib to into its own plugin so that the org.jenkinsci.plugins.workflow.libs package can be used without reference to sshd / git-server. This would lighten its footprint and simplify some functional test configuration.

          [JENKINS-57023] Split external library functionality into its own plugin

          Basil Crow added a comment -

          Hey jglick, I might be interested in taking this on, depending on what is involved. I think this is what would need to be done:

          • Create a new repository with the older and deprecated half of `workflow-cps-global-lib`, get it to compile, make sure tests are passing, etc.
          • Request hosting for the new plugin.
          • File a PR to remove the older and deprecated half of `workflow-cps-global-lib` from `workflow-cps-global-lib` itself.
          • Release new versions of each plugin.

          What I'm not entirely clear about is how the transition process would work for users. Would we just need to mention in the release notes that users which want the old functionality need to install a new plugin? Is there anything else that I'm missing from the above steps?

          Basil Crow added a comment - Hey jglick , I might be interested in taking this on, depending on what is involved. I think this is what would need to be done: Create a new repository with the older and deprecated half of `workflow-cps-global-lib`, get it to compile, make sure tests are passing, etc. Request hosting for the new plugin. File a PR to remove the older and deprecated half of `workflow-cps-global-lib` from `workflow-cps-global-lib` itself. Release new versions of each plugin. What I'm not entirely clear about is how the transition process would work for users. Would we just need to mention in the release notes that users which want the old functionality need to install a new plugin? Is there anything else that I'm missing from the above steps?

          Jesse Glick added a comment -

          Great! I would rather suggest inverting this:

          • Keep the workflow-cps-global-lib-plugin repository, which would continue to host a plugin of the same name.
          • Create a new plugin repository, probably with a more intuitive name like pipeline-groovy-libraries-plugin, and a smaller dependency tree.
          • Move the org.jenkinsci.plugins.workflow.libs package and the UserDefinedGlobalVariable and GrapeHack classes to pipeline-groovy-libraries. (Do not forget src/test/ and src/main/resources/!)
          • Make workflow-cps-global-lib depend on pipeline-groovy-libraries.
          • Adjust the pom.xml / index.jelly of both plugins to make it clear that pipeline-groovy-libraries is what most people know and love/hate as Groovy libraries, whereas workflow-cps-global-lib is deprecated and only around for compatibility.
          • Release pipeline-groovy-libraries, then the workflow-cps-global-lib update, mentioning the refactoring in the changelog for the latter and explaining that you can uninstall it if you were not using the Jenkins-hosted Git server feature.
          • Replace mentions of workflow-cps-global-lib with pipeline-groovy-libraries wherever you can find them: workflow-aggregator, in particular, since jenkinsci/jenkins/core/src/main/resources/jenkins/install/platform-plugins.json still “recommends” this (against my advice).

          Jesse Glick added a comment - Great! I would rather suggest inverting this: Keep the workflow-cps-global-lib-plugin repository, which would continue to host a plugin of the same name. Create a new plugin repository, probably with a more intuitive name like pipeline-groovy-libraries-plugin , and a smaller dependency tree. Move the org.jenkinsci.plugins.workflow.libs package and the UserDefinedGlobalVariable and GrapeHack classes to pipeline-groovy-libraries . (Do not forget src/test/ and src/main/resources/ !) Make workflow-cps-global-lib depend on pipeline-groovy-libraries . Adjust the pom.xml / index.jelly of both plugins to make it clear that pipeline-groovy-libraries is what most people know and love/hate as Groovy libraries, whereas workflow-cps-global-lib is deprecated and only around for compatibility. Release pipeline-groovy-libraries , then the workflow-cps-global-lib update, mentioning the refactoring in the changelog for the latter and explaining that you can uninstall it if you were not using the Jenkins-hosted Git server feature. Replace mentions of workflow-cps-global-lib with pipeline-groovy-libraries wherever you can find them: workflow-aggregator , in particular, since jenkinsci/jenkins/core/src/main/resources/jenkins/install/platform-plugins.json still “recommends” this (against my advice).

          Oleg Nenashev added a comment -

          Today I have strarted a similar thread in the developer mailng list: https://groups.google.com/forum/#!topic/jenkinsci-dev/L5obfNyRnz4 

          The suggestion there is to proceed with a more simple breaking change, but I am ready to follow the suggestion from jglick which retains compatibility at the cost of a more complicated delivery process

          Oleg Nenashev added a comment - Today I have strarted a similar thread in the developer mailng list:  https://groups.google.com/forum/#!topic/jenkinsci-dev/L5obfNyRnz4   The suggestion there is to proceed with a more simple breaking change, but I am ready to follow the suggestion from jglick which retains compatibility at the cost of a more complicated delivery process

          Jesse Glick added a comment -

          I am certainly not opposed to the possibility of a breaking change in something we are confident is rarely used if it makes Jenkins architecture significantly simpler. In this case the fully compatible change leads to an equally simple architecture for new installations, and the delivery steps are not all that difficult if you are used to doing this sort of thing and can get a HOSTING request approved (which you would need in either case). And since the hosted Git repository was the only option for libraries for a long time, I suspect there are still a number of users out there who might be unpleasantly surprised by a breaking change.

          The thread mentioned “startup overhead” but it is not clear to me what this was about—the SSHD server is disabled by default already, and the handful of extensions involved in serving libraries or other Git repositories via either HTTP or SSH should not be contributing that much to class loading time.

          By the way, to retain commit history (git blame etc.), you could try doing this by creating a clone (Git clone, not GH fork!) of workflow-cps-global-lib-plugin from which WorkflowLibRepository and related old code is deleted in a new commit; while in the original repo, the newer code is deleted in a new commit. On the other hand git-filter-repo is claimed to be far easier to work with than what we used to use for plugin splits and similar refactorings.

          Jesse Glick added a comment - I am certainly not opposed to the possibility of a breaking change in something we are confident is rarely used if it makes Jenkins architecture significantly simpler. In this case the fully compatible change leads to an equally simple architecture for new installations, and the delivery steps are not all that difficult if you are used to doing this sort of thing and can get a HOSTING request approved (which you would need in either case). And since the hosted Git repository was the only option for libraries for a long time, I suspect there are still a number of users out there who might be unpleasantly surprised by a breaking change. The thread mentioned “startup overhead” but it is not clear to me what this was about—the SSHD server is disabled by default already, and the handful of extensions involved in serving libraries or other Git repositories via either HTTP or SSH should not be contributing that much to class loading time. By the way, to retain commit history ( git blame etc.), you could try doing this by creating a clone (Git clone, not GH fork!) of workflow-cps-global-lib-plugin from which WorkflowLibRepository and related old code is deleted in a new commit; while in the original repo, the newer code is deleted in a new commit. On the other hand git-filter-repo is claimed to be far easier to work with than what we used to use for plugin splits and similar refactorings.

          Jesse Glick added a comment -

          https://github.com/jenkinsci/jep/blob/master/jep/230/README.adoc#cleanup-options suggests this:

          workflow-cps-global-lib can be given a dependency on git-server and thus sshd for its original Git-based design, but formally deprecated and the SCM-based library system almost exclusively used these days split into a new plugin such as pipeline-libraries without this dependency.

          Jesse Glick added a comment - https://github.com/jenkinsci/jep/blob/master/jep/230/README.adoc#cleanup-options suggests this: workflow-cps-global-lib can be given a dependency on git-server and thus sshd for its original Git-based design, but formally deprecated and the SCM-based library system almost exclusively used these days split into a new plugin such as pipeline-libraries without this dependency.

          Jesse Glick added a comment -

          Still working through downstream changes and integration into BOM. Very slow progress due to the need to wait for PR reviews and slow/flaky CI.

          Jesse Glick added a comment - Still working through downstream changes and integration into BOM. Very slow progress due to the need to wait for PR reviews and slow/flaky CI.

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: