-
Improvement
-
Resolution: Fixed
-
Major
The dependencies in plugin POMs are typically a mess, especially when deep dependency trees are in use, as is typical for Pipeline suite components. The problem is Maven's transitive dependency resolution algorithm, which prefers the "nearest" version, even if it is older than what some other dependency requires. This does not work well for plugin dependencies in particular, and Jenkins 2.19.x and newer will in fact refuse to load plugins which fail to satisfy their dependencies, rather than producing cryptic linkage errors later. You can see this behavior via
mvn -Djenkins.version=2.19.3 -Dtest=InjectedTest test
Probably the best way forward is to use the Enforcer plugin to require upper bound dependencies. You can try this in a single plugin:
<build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-enforcer-plugin</artifactId> <configuration> <rules combine.children="append"> <requireUpperBoundDeps/> </rules> </configuration> </plugin> </plugins> </build>
I think andresrc tried adding this to the parent POM before but ran into issues. While it does detect plugin version mismatches, it also detects some problems that are apparently harmless (and currently RequireUpperBoundDeps has no excludes option). In particular I note from workflow-aggregator as an example:
- jenkins-core depends on com.google.guava:guava:11.0.1, which we would like to retain in plugin test classpaths for fidelity to runtime behavior, yet Stapler depends on 14.0
- that old Guava depends on com.google.code.findbugs:jsr305:1.3.9, whereas we would prefer Stapler's 2.0.1 (at least)
- org.kohsuke:github-api depends on com.infradna.tool:bridge-method-annotation:1.14, newer than core
- net.sourceforge.htmlunit:htmlunit and org.apache.httpcomponents:httpclient depend on commons-logging:commons-logging:1.2, newer than core's 1.1.3
- org.jenkins-ci.lib:lib-jenkins-maven-embedder, used from org.jenkins-ci.main:jenkins-test-harness, depends on org.apache.ant:ant:1.9.2 while core still uses 1.8.4
- org.jenkins-ci.lib:lib-jenkins-maven-embedder also uses org.codehaus.plexus:plexus-utils:3.0.10, newer than the 2.1 used by other paths in jenkins-test-harness; there is a similar problem with org.codehaus.plexus:plexus-classworlds 2.4 vs. 2.4.2
The Stapler-vs.-core conflict probably cannot be resolved in historical core baselines, so we may need to patch the Enforcer to allow an excludes list. Some of the other conflicts look solvable.
- depends on
-
-
- Resolved
-
- relates to
-
JENKINS-45047 Easy way to run integration tests in plugin repositories
-
- In Progress
-
-
-
- Resolved
-
- links to
[JENKINS-41631] Use requireUpperBoundDeps in plugin POM
I think in those cases we would want to add an <exclusion> for the transitive library dependency(ies) which we know we do not care about.
does maven support wildcard exclusions yet? last time I looked it didn't which leads to a overly complex pom to do what should be
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
Example of a test failure that is more easily caught by the enforcer rule:
$ mvn -f parameterized-trigger-plugin -Dtest=InjectedTest -Djenkins.version=2.32.3 test … … jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Jenkins promoted builds plugin v2.25 (promoted-builds) java.io.IOException: Jenkins promoted builds plugin v2.25 failed to load. - Token Macro Plugin v1.5.1 is older than required. To fix, install v1.10 or later. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:621) at hudson.PluginManager$2$1$1.run(PluginManager.java:516) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:1085) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) … jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Pipeline: Supporting APIs v2.13 (workflow-support) java.io.IOException: Pipeline: Supporting APIs v2.13 failed to load. - Script Security Plugin v1.13 is older than required. To fix, install v1.21 or later. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:621) at hudson.PluginManager$2$1$1.run(PluginManager.java:516) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:1085) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) … jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Pipeline: Nodes and Processes v2.9 (workflow-durable-task-step) java.io.IOException: Pipeline: Nodes and Processes v2.9 failed to load. - Pipeline: Supporting APIs v2.13 failed to load. Fix this plugin first. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:621) at hudson.PluginManager$2$1$1.run(PluginManager.java:516) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:1085) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) … jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Pipeline: Job v2.9 (workflow-job) java.io.IOException: Pipeline: Job v2.9 failed to load. - Pipeline: Supporting APIs v2.2 failed to load. Fix this plugin first. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:621) at hudson.PluginManager$2$1$1.run(PluginManager.java:516) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:1085) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) … jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Pipeline: Groovy v2.27 (workflow-cps) java.io.IOException: Pipeline: Groovy v2.27 failed to load. - Pipeline: Supporting APIs v2.13 failed to load. Fix this plugin first. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:621) at hudson.PluginManager$2$1$1.run(PluginManager.java:516) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$7.runTask(Jenkins.java:1085) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:748) …
which with the proposed POM update fails earlier and more helpfully:
--- maven-enforcer-plugin:1.4.2-SNAPSHOT:enforce (display-info) @ parameterized-trigger ---
Ignoring requireUpperBoundDeps in com.google.code.findbugs:jsr305
Ignoring requireUpperBoundDeps in commons-logging:commons-logging
Ignoring requireUpperBoundDeps in com.google.guava:guava
Rule 5: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for org.jenkins-ci.plugins:scm-api:0.2 paths to dependency are:
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins:subversion:2.5.7
+-org.jenkins-ci.plugins:scm-api:0.2
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-cps:2.27
+-org.jenkins-ci.plugins:scm-api:1.1
,
Require upper bound dependencies error for org.jenkins-ci.plugins:script-security:1.13 paths to dependency are:
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins:matrix-project:1.6
+-org.jenkins-ci.plugins:script-security:1.13
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-cps:2.27
+-org.jenkins-ci.plugins:script-security:1.24
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-durable-task-step:2.9
+-org.jenkins-ci.plugins:script-security:1.25
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-support:2.13
+-org.jenkins-ci.plugins:script-security:1.21
,
Require upper bound dependencies error for org.apache.ant:ant:1.8.4 paths to dependency are:
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.main:jenkins-core:2.32.3
+-org.apache.ant:ant:1.8.4
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins:promoted-builds:2.25
+-org.jenkins-ci.main:maven-plugin:2.0
+-org.jenkins-ci.main.maven:maven-agent:1.4
+-org.apache.ant:ant:1.8.4
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins:promoted-builds:2.25
+-org.jenkins-ci.main:maven-plugin:2.0
+-org.jenkins-ci.lib:lib-jenkins-maven-embedder:3.11
+-org.apache.ant:ant:1.9.2
,
Require upper bound dependencies error for org.jenkins-ci:symbol-annotation:1.1 paths to dependency are:
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.main:jenkins-core:2.32.3
+-org.jenkins-ci:symbol-annotation:1.1
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-cps:2.27
+-org.jenkins-ci.plugins:structs:1.5
+-org.jenkins-ci:symbol-annotation:1.5
,
Require upper bound dependencies error for org.jenkins-ci.plugins.icon-shim:icon-set:1.0.5 paths to dependency are:
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.main:jenkins-core:2.32.3
+-org.jenkins-ci.plugins.icon-shim:icon-set:1.0.5
and
+-org.jenkins-ci.plugins:parameterized-trigger:2.34-SNAPSHOT
+-org.jenkins-ci.plugins:matrix-auth:1.4
+-org.jenkins-ci.plugins.icon-shim:icon-shim:2.0.3
+-org.jenkins-ci.plugins.icon-shim:icon-set:2.0.3
]
The token-macro problem is not yet getting picked up, because it is an optional dependency of promoted-builds yet a mandatory dependency of conditional-buildstep. Since Maven dependency resolution does not pick up the transitive closure of optional dependencies, Maven only considers there to be one dependency path to token-macro. Jenkins however treats a low version of a dependency as an error for the depending plugin, even in the case of an optional dependency, since the plugin APIs offer no way to obtain a dependency iff it is supposed to be available. So this case remains one that needs to be tested by rerunning InjectedTest.
I have also been experimenting with a Maven EventSpy that could mutate the dependency list of a project during a build so as to both set specific dependency versions to supplied values (presumably newer than what was written in the committed POM), and also force all other dependencies to the computed upper bound. So far I have not been able to get it to work well—I can patch MavenProject.dependencies at various points and affect the behavior of mojos (surefire:test is of principle interest), but patching MavenProject.artifacts has no visible effect; and also I have not managed to run RequireUpperBoundDepsVisitor or an equivalent so as to determine what transitive dependencies should be set to. Alternatives would be to patch the pom.xml in a local checkout, as plugin-compat-tester does (clumsily), or versions-maven-plugin does (but there is no AbstractVersionsDependencyUpdaterMojo subtype which would specifically make requireUpperBoundDeps work, and its infrastructure so far only supports replacing existing source dependencies but not adding new ones). Advice from stephenconnolly would be appreciated.
The purpose would be to allow automated integration tests between plugins. For example, I would like for a PR to workflow-cps to automatically run tests in other plugins, like workflow-cps-global-lib, with the workflow-cps dependency set to the locally built snapshot. Now for simple cases this could be done easily already with no tricks: ensure that the workflow-cps-global-lib POM defines a workflow-cps-plugin.version property, and override that (as well as jenkins.version where necessary). The trouble comes when the PR to workflow-cps is downstream of a PR to, say, workflow-api. Presumably workflow-cps-global-lib is still using an older (release) version of workflow-api, so the integration test will fail. You could make it define a workflow-api-plugin.version property too, but this could become an endless exercise in replacing every dependency version with a property, and anyway the Jenkinsfile for the workflow-cps PR would then need to be patched to explicitly request the new (timestamped!) snapshot version of workflow-api, duplicating information in pom.xml. This seems unmaintainable. I would rather run the integration test Maven build with some special option, or with some special preprocessing, that automatically updates the workflow-cps version and any transitive dependencies (ideally including the Jenkins baseline too—currently workflow-cps in fact uses a newer baseline than workflow-cps-global-lib) to the upper bounds.
Another variant, drawing on a suggestion by stephenconnolly, would be to define a mojo in maven-hpi-plugin which delegates to surefire:test while customizing the additional classpath and dependency excludes. Then the ITs would be called something like
mvn -f workflow-cps-global-lib-plugin test-compile hpi:it-test -Doverrides=org.jenkins-ci-plugins.workflow:workflow-cps:2.35-SNAPSHOT
Cold also try just setting a project property and seeing if surefire:test picks that up.
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/org/jvnet/hudson/test/HudsonTestCase.java
src/main/java/org/jvnet/hudson/test/JenkinsRule.java
http://jenkins-ci.org/commit/jenkins-test-harness/075d331d23b7b3fd44434cbca78200168cbeaf8b
Log:
JENKINS-41631 Deleting the Maven Embedder dependency from the test harness.
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/org/jvnet/hudson/test/HudsonTestCase.java
src/main/java/org/jvnet/hudson/test/JenkinsRule.java
http://jenkins-ci.org/commit/jenkins-test-harness/f9c67d5527bd1ef7cf7b6139108f9eb200ee5c3a
Log:
Merge pull request #62 from jglick/no-maven-embedder-JENKINS-41631
JENKINS-41631 Deleting the Maven Embedder dependency from the test harness
Compare: https://github.com/jenkinsci/jenkins-test-harness/compare/c517db8200c3...f9c67d5527bd
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
http://jenkins-ci.org/commit/plugin-pom/3f0a3eac85a8d3783b514e456615c3395629dc01
Log:
JENKINS-41631 Running requireUpperBoundDeps.
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
http://jenkins-ci.org/commit/plugin-pom/17cc56db940a78065a3da3344ff5cc93a6538738
Log:
Merge pull request #67 from jglick/requireUpperBoundDeps-JENKINS-41631
JENKINS-41631 Running requireUpperBoundDeps
Compare: https://github.com/jenkinsci/plugin-pom/compare/79ef0ec509dc...17cc56db940a
Michael Fowler
added a comment - jglick can this be disabled for test dependencies? There's not much I can do about this:
Rule 5: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message: Failed while enforcing RequireUpperBoundDeps. The error(s) are [ Require upper bound dependencies error for org.objenesis:objenesis:2.1 paths to dependency are: +-org.jenkins-ci.plugins:coverity:1.9.3-SNAPSHOT +-org.powermock:powermock-api-mockito:1.6.6 +-org.mockito:mockito-core:1.10.19 +-org.objenesis:objenesis:2.1 and +-org.jenkins-ci.plugins:coverity:1.9.3-SNAPSHOT +-org.powermock:powermock-module-junit4:1.6.6 +-org.powermock:powermock-module-junit4-common:1.6.6 +-org.powermock:powermock-reflect:1.6.6 +-org.objenesis:objenesis:2.4
Michael Fowler
added a comment - jglick can this be disabled for test dependencies? There's not much I can do about this:
Rule 5: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for org.objenesis:objenesis:2.1 paths to dependency are:
+-org.jenkins-ci.plugins:coverity:1.9.3-SNAPSHOT
+-org.powermock:powermock-api-mockito:1.6.6
+-org.mockito:mockito-core:1.10.19
+-org.objenesis:objenesis:2.1
and
+-org.jenkins-ci.plugins:coverity:1.9.3-SNAPSHOT
+-org.powermock:powermock-module-junit4:1.6.6
+-org.powermock:powermock-module-junit4-common:1.6.6
+-org.powermock:powermock-reflect:1.6.6
+-org.objenesis:objenesis:2.4
Sure there is. Decide which version of objenesis you meant to use and exclude the other. At least until PowerMock fixes their dependencies to be consistent.
I think we need to enable this on the core too. Without this, we can release a Jenkins version that doesn't respect this itself, hence will make plugins build trying to bump to it fail everytime.
For instance, trying to bump to 2.71:
[snip]
+-com.cloudbees.operations-center.common:operations-center-agent:2.60.0.3-SNAPSHOT
+-org.jenkins-ci.main:jenkins-war:2.71
+-org.jenkins-ci.modules:sshd:2.0
+-org.slf4j:slf4j-api:1.6.1
and
+-com.cloudbees.operations-center.common:operations-center-agent:2.60.0.3-SNAPSHOT
+-org.jenkins-ci.main:jenkins-war:2.71
+-org.jenkins-ci.modules:sshd:2.0
+-org.apache.sshd:sshd-core:1.6.0
+-org.slf4j:slf4j-api:1.7.25
When both are actually pulled in from org.jenkins-ci.main:jenkins-war:2.71.
And maybe also enabled on core modules like sshd, example in that case: https://github.com/jenkinsci/sshd-module/blob/f709b950581d8a46ab35e6f22a664fe26e086ac1/pom.xml#L39 vs. https://github.com/apache/mina-sshd/blob/3df4a6a14edd1e22513430b75fc50dd5b5de989e/pom.xml#L112
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
http://jenkins-ci.org/commit/plugin-pom/0782ed6172638173dadb4f5d38cd9d7a8db6e3f7
Log:
JENKINS-41631 Work better with new cores.
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
http://jenkins-ci.org/commit/plugin-pom/d05e6ad500223d7426a93c75e3f1907a5b50a17b
Log:
Merge pull request #73 from jglick/requireUpperBoundDeps-JENKINS-41631
JENKINS-41631 Make requireUpperBoundDeps work better with new cores
Compare: https://github.com/jenkinsci/plugin-pom/compare/e245b2e2b919...d05e6ad50022
Code changed in jenkins
User: Jesse Glick
Path:
core/pom.xml
pom.xml
test/pom.xml
war/pom.xml
http://jenkins-ci.org/commit/jenkins/b8f6246d7600a6e7d8b732da9c3153fb33f5ddde
Log:
JENKINS-41631 Enforce upper bound deps on Jenkins core (#2956)
JENKINS-41631Enforce upper bound deps on Jenkins core.
- stapler 1.252
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
http://jenkins-ci.org/commit/parameterized-trigger-plugin/95056c1d3460da6bf207f9b56f7e97648d7a9754
Log:
JENKINS-41631 Demonstration of POM update (#113)
JENKINS-41631Demonstration of POM update.
- plugin-pom 2.31
Don't we just want upper bounds on other plugins - (aka <packaging>hpi</packaging>) and not other things like libraries (<packaging>jar</packaging>)?
The upper bounds in libraries seems a little bit whacky - esp as in Jenkins you may not get the upper bound of a library if you have a library used in core (slf4j) and in a plugin...
This seems to be mostly an issue when running tests with JenkinsRule