Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-51965

Add support of JAXB unpackaging for Java 9+ in WAR Exploder

    XMLWordPrintable

Details

    Description

      As discussed in JENKINS-51821, Jenkins won't work on Java 11 due to removed JAXB modules. I propose the following approach:

      • Package 3 JAXB libs (jaxb-api-2.3.0.jar, jaxb-core-2.3.0.1.jar, jaxb-impl-2.3.0.1.jar) to a WAR resource which does not get classloaded by default. E.g. "WEB-INF/platform-compat/java11" (or 9 ?)
      • When Jenkins starts up on Java 9+, Jenkins WAR exploder copies the libraries to "war/WEB-INF/lib" so that they get picked by classloader automatically

      It will allow to have a transparent support of Java 9..11 within the WAR file (without adding modules, etc.)

      A better way forward would be Multi-Version JAR, but we need a massive tooling update to support it

      Attachments

        Issue Links

          Activity

            I've worked a bit in the last few days on checking the current state of Jenkins without the --add-modules switches: I've tried looking for an easy case to reproduce this issue, but it's actually not anymore easy to make Jenkins fail for a JAXB error, which is good. That is mostly because since when the issues were filed, many were fixed in plac to not use JAXB anymore where this was possible.

            On the opensource side, it's close to impossible to check through the UI because of a few plugins have the majority javax.xml.bind imports
            https://github.com/search?q=org%3Ajenkinsci+%22javax.xml.bind%22&unscoped_q=%22javax.xml.bind%22, so I had to go through the GitHub API using curl and some shell scripting to get the full list.

            On the OSS side, the full list of repositories using "javax.xml.bind":

                286 "coverity-plugin"
                194 "inflectra-spira-integration-plugin"
                105 "zephyr-enterprise-test-management-plugin"
                 24 "codebeamer-coverage-publisher-plugin"
                 21 "codesonar-plugin"
                 14 "java-client-api"
                 13 "packageversion-plugin"
                  9 "performance-signature-dynatrace-plugin"
                  9 "hp-operations-orchestration-plugin"
                  8 "meliora-testlab-plugin"
                  8 "mabl-integration-plugin"
                  8 "jelly"
                  8 "app.io-plugin"
                  7 "sloccount-plugin"
                  7 "jira-issues-versioning-plugin"
                  7 "dom4j"
                  6 "extras-ec2-launcher"
                  6 "dynatrace-plugin"
                  5 "xlrelease-plugin"
                  5 "xcode-plugin"
                  5 "libdtkit"
                  5 "github-coverage-reporter-plugin"
                  4 "zanata-plugin"
                  3 "cppcheck-plugin"
                  2 "ucm4svn-plugin"
                  2 "tfs-plugin"
                  2 "stride-notification-plugin"
                  2 "refit-plugin"
                  2 "notification-plugin"
                  2 "maven-metadata-plugin"
                  2 "hp-quality-center-plugin"
                  2 "ec2-deployment-dashboard"
                  2 "docker-workflow-plugin"
                  2 "docker"
                  1 "windows-azure-storage-plugin"
                  1 "timestamper-plugin"
                  1 "testinium-plugin"
                  1 "testcomplete-plugin"
                  1 "testcomplete11-xunit-plugin"
                  1 "qualityclouds-plugin"
                  1 "policycenter-gate-validator-plugin"
                  1 "pipeline-aws-plugin"
                  1 "performance-plugin"
                  1 "maven-hudson-dev-plugin"
                  1 "maven-artifact-choicelistprovider-plugin"
                  1 "localization-plugin"
                  1 "klocwork-plugin"
                  1 "jwsdp-sqe-plugin"
                  1 "job-import-plugin"
                  1 "jenkow-plugin"
                  1 "jclouds-plugin"
                  1 "jaxb-plugin"
                  1 "ibm-cloud-devops-plugin"
                  1 "fortify360-plugin"
                  1 "dtkit-plugin"
                  1 "agile-cockpit-notification-plugin"
                  1 "acceptance-test-harness"
            

            Some notes

            Conclusion:

            Given:

            • the long term right fix is likely to actually not need --add-modules anymore and allow plugins to do like nothing changed
            • we already somehow communicated the fact we were shifting gears, and interpreting to my advantage "we are willing to break some stuff to gain what we really need" . In other words: yes we could obviously fix this, but do we really need/want to spend more time on making JAXB usable like before when there seems to be no real need .

            I'm inclined to say we should not bother adapting JAXB in Jenkins so plugins can work like before forever. At least I hope my summary above shows there's no real traction to do this.

            I would rather encourage plugins to not depend on JAXB anymore.

            What do you think?

            batmat Baptiste Mathus added a comment - I've worked a bit in the last few days on checking the current state of Jenkins without the --add-modules switches: I've tried looking for an easy case to reproduce this issue, but it's actually not anymore easy to make Jenkins fail for a JAXB error, which is good. That is mostly because since when the issues were filed, many were fixed in plac to not use JAXB anymore where this was possible. On the opensource side, it's close to impossible to check through the UI because of a few plugins have the majority javax.xml.bind imports https://github.com/search?q=org%3Ajenkinsci+%22javax.xml.bind%22&unscoped_q=%22javax.xml.bind%22 , so I had to go through the GitHub API using curl and some shell scripting to get the full list. On the OSS side, the full list of repositories using "javax.xml.bind": 286 "coverity-plugin" 194 "inflectra-spira-integration-plugin" 105 "zephyr-enterprise-test-management-plugin" 24 "codebeamer-coverage-publisher-plugin" 21 "codesonar-plugin" 14 "java-client-api" 13 "packageversion-plugin" 9 "performance-signature-dynatrace-plugin" 9 "hp-operations-orchestration-plugin" 8 "meliora-testlab-plugin" 8 "mabl-integration-plugin" 8 "jelly" 8 "app.io-plugin" 7 "sloccount-plugin" 7 "jira-issues-versioning-plugin" 7 "dom4j" 6 "extras-ec2-launcher" 6 "dynatrace-plugin" 5 "xlrelease-plugin" 5 "xcode-plugin" 5 "libdtkit" 5 "github-coverage-reporter-plugin" 4 "zanata-plugin" 3 "cppcheck-plugin" 2 "ucm4svn-plugin" 2 "tfs-plugin" 2 "stride-notification-plugin" 2 "refit-plugin" 2 "notification-plugin" 2 "maven-metadata-plugin" 2 "hp-quality-center-plugin" 2 "ec2-deployment-dashboard" 2 "docker-workflow-plugin" 2 "docker" 1 "windows-azure-storage-plugin" 1 "timestamper-plugin" 1 "testinium-plugin" 1 "testcomplete-plugin" 1 "testcomplete11-xunit-plugin" 1 "qualityclouds-plugin" 1 "policycenter-gate-validator-plugin" 1 "pipeline-aws-plugin" 1 "performance-plugin" 1 "maven-hudson-dev-plugin" 1 "maven-artifact-choicelistprovider-plugin" 1 "localization-plugin" 1 "klocwork-plugin" 1 "jwsdp-sqe-plugin" 1 "job-import-plugin" 1 "jenkow-plugin" 1 "jclouds-plugin" 1 "jaxb-plugin" 1 "ibm-cloud-devops-plugin" 1 "fortify360-plugin" 1 "dtkit-plugin" 1 "agile-cockpit-notification-plugin" 1 "acceptance-test-harness" Some notes the biggest user above, coverity-plugin, is actually already deprecated: https://github.com/jenkinsci/coverity-plugin and replaced by https://github.com/jenkinsci/synopsys-coverity-plugin the second one, inflectra-spira-integration-plugin, is installed 7 times worldwide https://plugins.jenkins.io/inflectra-spira-integration https://plugins.jenkins.io/zephyr-enterprise-test-management is installed 163 times codebeamer-coverage-publisher-plugin installed 62 times codesonar has a decent install number (2200+). This plugin is maintained by Praqma, generally a very active contributor in the Jenkins community, so I filed https://issues.jenkins-ci.org/browse/JENKINS-55572 to see if this could be fixed there too the rest look generally like plugins that are either not used a lot, or quite easily fixable given they use JAXB in very few places, ala https://github.com/jenkinsci/instance-identity-module/pull/11 for instance that was fixed by Kohsuke. Conclusion: Given: the long term right fix is likely to actually not need --add-modules anymore and allow plugins to do like nothing changed we already somehow communicated the fact we were shifting gears , and interpreting to my advantage "we are willing to break some stuff to gain what we really need" . In other words: yes we could obviously fix this, but do we really need/want to spend more time on making JAXB usable like before when there seems to be no real need . I'm inclined to say we should not bother adapting JAXB in Jenkins so plugins can work like before forever. At least I hope my summary above shows there's no real traction to do this. I would rather encourage plugins to not depend on JAXB anymore. What do you think?

            (BTW, I'm now thinking fixing the list above might be a good candidate for FOSDEM's hackfest. It's easy enough from a code perspective and then would allow people to go through the PR cycle, and a bit more code-oriented than the usual typo-fixes we do in such events)

            batmat Baptiste Mathus added a comment - (BTW, I'm now thinking fixing the list above might be a good candidate for FOSDEM's hackfest. It's easy enough from a code perspective and then would allow people to go through the PR cycle, and a bit more code-oriented than the usual typo-fixes we do in such events)

            A new update: after Oleg recommended I try the SlocCount plugin, I did. And it does fail as expected on Java 11 by default. I filed JENKINS-55620 to track this.

            So then I tried dropping directly jaxb-api-2.3.0.jar and jaxb-core-2.3.0.1.jar into various locations to see how it behaves. Everything worked fine.

              Java 8 Java 11
            Plugin's WEB-INF/lib  OK OK
            Jenkins' war/WEB-INF/lib OK OK

            To summarize, it seems like hardcoding the dependency against JAXB directly in the plugin would be simpler. I'm not saying it will work in every single case, where/if a given plugin does something more convoluted, trying for instance to use JAXB instances coming from another plugin (which I think, but untested yet, would result in having the classes seen as different given classloaders would be different).

            Circling back to my previous point, and adding up to Jesse's comment above:

            which plugins would actually be affected? It is maybe a five-line change to switch the thread context loader¹, and it would be nice to get JAXB out of core if we could.

            • I think I provided some data about "which plugins would actually be affected?": the answer is likely "very few" or something along this.
            • "get JAXB out of core": given my testing on sloccount-plugin, I think this is possible.

            Now, I'm aware I've not tested the whole Jenkins ecosystem, but so I'm still wondering if this subject is as prevalent as we were imagining. I guess, again, the fact many fixes were pushed already makes this less and less of an issue.

            I'm eager to hear what people think.

            batmat Baptiste Mathus added a comment - A new update: after Oleg recommended I try the SlocCount plugin, I did. And it does fail as expected on Java 11 by default. I filed JENKINS-55620 to track this. So then I tried dropping directly jaxb-api-2.3.0.jar and jaxb-core-2.3.0.1.jar into various locations to see how it behaves. Everything worked fine.   Java 8 Java 11 Plugin's WEB-INF/lib  OK OK Jenkins' war/WEB-INF/lib OK OK To summarize, it seems like hardcoding the dependency against JAXB directly in the plugin would be simpler. I'm not saying it will work in every single case, where/if a given plugin does something more convoluted, trying for instance to use JAXB instances coming from another plugin (which I think, but untested yet, would result in having the classes seen as different given classloaders would be different). Circling back to my previous point, and adding up to Jesse's comment above: which plugins would actually be affected? It is maybe a five-line change to switch the thread context loader¹, and it would be nice to get JAXB out of core if we could. I think I provided some data about "which plugins would actually be affected?": the answer is likely "very few" or something along this. "get JAXB out of core": given my testing on sloccount-plugin, I think this is possible. Now, I'm aware I've not tested the whole Jenkins ecosystem, but so I'm still wondering if this subject is as prevalent as we were imagining. I guess, again, the fact many fixes were pushed already makes this less and less of an issue. I'm eager to hear what people think.

            Filed JENKINS-55944 to track removing JAXB usages where possible.

            batmat Baptiste Mathus added a comment - Filed  JENKINS-55944 to track removing JAXB usages where possible.

            AFAICT, this is superseded by JENKINS-55681, which landed in 2.163 as per https://github.com/jenkinsci/jenkins/pull/3865

            batmat Baptiste Mathus added a comment - AFAICT, this is superseded by JENKINS-55681 , which landed in 2.163 as per https://github.com/jenkinsci/jenkins/pull/3865

            People

              kohsuke Kohsuke Kawaguchi
              oleg_nenashev Oleg Nenashev
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: