[JENKINS-50099] Downstream pipelines are not triggered when the artifact is a a WAR and consumed by war-overlay

Type: Bug
Resolution: Unresolved
Priority: Minor
Component/s: pipeline-maven-plugin
Labels:
None
Environment:
Jenkins ver. 2.89.4, Slave on ubuntu, freshly updated plugins; Pipeline Maven Integration 3.4.2, maven 3.5.3

Similar Issues:
Powered by SuggestiMate

Show

Problem

When a SNAPSHOT artifact is built (packaging = 'war'), its downstream project building a WAR-overlay depending on the upstream SNAPSHOT is not triggered.

How to reproduce

A test project (maven_war_overlay_project) with the described behaviour is available in the test_maven_projects directory on the fork in https://github.com/benzht/pipeline-maven-plugin/tree/JENKINS-50099 under the test_maven_projects (due to time constraints I could not integrate it into the test suite....).

The new test project, maven_war_overlay_project, depends on the existing maven_war_project, which in turn depends on maven_jar_project.
After manually building all of them, building the first correctly triggers the second, but maven_war_overlay_project is not triggered.
Looking at the logging, it becomes clear that in the war_overlay_project no dependency is detected.

Output from maven_war_project correctly records its dependency:

...
[withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
[withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
[withMaven] pipelineGraphPublisher - Record dependency: jenkins.mvn.test:mono-module-maven-app:0.1-SNAPSHOT, ignoreUpstreamTriggers: false
[withMaven] pipelineGraphPublisher - Skip recording release dependency: javax.servlet:javax.servlet-api:3.1.0
[withMaven] pipelineGraphPublisher - Skip recording release dependency: junit:junit:4.12
[withMaven] pipelineGraphPublisher - Skip recording release dependency: org.hamcrest:hamcrest-core:1.3
[withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/pom.xml
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-20180311.164646-4, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/target/maven-war-app-0.1-SNAPSHOT.war
...

Output from maven_war_overlay_project does not record any dependeny:

...
[withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
[withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
[withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/pom.xml
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-20180311.164534-3, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/target/maven-war-overlay-app.war

Jenkinsfile of maven_war_overlay_project:

node {
   stage('Preparation') { // for display purposes
      git changelog: false, credentialsId: '3360632d-fb8d-4f68-be83-b650ec243e94', poll: false, url: 'http://........git'
   }
   stage('Build') {
        withMaven(jdk: 'JDK8', maven: 'M3',
                globalMavenSettingsConfig: 'globalMavenSettings', options: [dependenciesFingerprintPublisher(), pipelineGraphPublisher(), artifactsPublisher(disabled: true)]) {
            writeFile file: '.archive-jenkins-maven-event-spy-logs', text: ''
            sh "mvn clean deploy"
        }
   }
}

is blocked by

JENKINS-50205 "runtime" scope dependencies are not tracked

Open

is related to

JENKINS-50475 Scope import / Maven BOM dependencies

Open

relates to

JENKINS-59500 Downstream pipelines are not triggered when the artifact has type docker-info

Closed

Hartmut Benz created issue - 2018-03-11 17:20

Hartmut Benz made changes - 2018-03-11 17:26

Description

Original: h2. Problem

When a SNAPSHOT artifact is built (packaging = 'war'), its downstream project building a WAR-overlay depending on the upstream SNAPSHOT is not triggered.
h2. How to reproduce

A test project (maven_war_overlay_project) with the described behaviour is available in the _test_maven_projects_ directory on the fork in [https://github.com/benzht/pipeline-maven-plugin] under the test_maven_projects (due to time constraints I could not integrate it into the test suite....).

The new test project, maven_war_overlay_project, depends on the existing maven_war_project, which in turn depends on maven_jar_project.
After manually building all of them, building the first correctly triggers the second, but maven_war_overlay_project is not triggered.
Looking at the logging, it becomes clear that in the war_overlay_project no dependency is detected.

Output from maven_war_project correctly records its dependency:
{code:java}
...
[withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
[withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
[withMaven] pipelineGraphPublisher - Record dependency: jenkins.mvn.test:mono-module-maven-app:0.1-SNAPSHOT, ignoreUpstreamTriggers: false
[withMaven] pipelineGraphPublisher - Skip recording release dependency: javax.servlet:javax.servlet-api:3.1.0
[withMaven] pipelineGraphPublisher - Skip recording release dependency: junit:junit:4.12
[withMaven] pipelineGraphPublisher - Skip recording release dependency: org.hamcrest:hamcrest-core:1.3
[withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/pom.xml
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-20180311.164646-4, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/target/maven-war-app-0.1-SNAPSHOT.war
...
{code}
Output from maven_war_overlay_project does not record any dependeny:
{code:java}
...
[withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
[withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
[withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/pom.xml
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-20180311.164534-3, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/target/maven-war-overlay-app.war
{code}
Jenkinsfile of maven_war_overlay_project:
{code:java}
node {
   stage('Preparation') { // for display purposes
      git changelog: false, credentialsId: '3360632d-fb8d-4f68-be83-b650ec243e94', poll: false, url: 'http://........git'
   }
   stage('Build') {
        withMaven(jdk: 'JDK8', maven: 'M3',
                globalMavenSettingsConfig: 'globalMavenSettings', options: [dependenciesFingerprintPublisher(), pipelineGraphPublisher(), artifactsPublisher(disabled: true)]) {
            writeFile file: '.archive-jenkins-maven-event-spy-logs', text: ''
            sh "mvn clean deploy"
        }
   }
}
{code}

New: h2. Problem

When a SNAPSHOT artifact is built (packaging = 'war'), its downstream project building a WAR-overlay depending on the upstream SNAPSHOT is not triggered.
h2. How to reproduce

A test project (maven_war_overlay_project) with the described behaviour is available in the _test_maven_projects_ directory on the fork in https://github.com/benzht/pipeline-maven-plugin/tree/JENKINS-50099 under the test_maven_projects (due to time constraints I could not integrate it into the test suite....).

The new test project, maven_war_overlay_project, depends on the existing maven_war_project, which in turn depends on maven_jar_project.
After manually building all of them, building the first correctly triggers the second, but maven_war_overlay_project is not triggered.
Looking at the logging, it becomes clear that in the war_overlay_project no dependency is detected.

Output from maven_war_project correctly records its dependency:
{code:java}
...
[withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
[withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
[withMaven] pipelineGraphPublisher - Record dependency: jenkins.mvn.test:mono-module-maven-app:0.1-SNAPSHOT, ignoreUpstreamTriggers: false
[withMaven] pipelineGraphPublisher - Skip recording release dependency: javax.servlet:javax.servlet-api:3.1.0
[withMaven] pipelineGraphPublisher - Skip recording release dependency: junit:junit:4.12
[withMaven] pipelineGraphPublisher - Skip recording release dependency: org.hamcrest:hamcrest-core:1.3
[withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/pom.xml
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-20180311.164646-4, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/target/maven-war-app-0.1-SNAPSHOT.war
...
{code}
Output from maven_war_overlay_project does not record any dependeny:
{code:java}
...
[withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
[withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
[withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/pom.xml
[withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-20180311.164534-3, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/target/maven-war-overlay-app.war
{code}
Jenkinsfile of maven_war_overlay_project:
{code:java}
node {
   stage('Preparation') { // for display purposes
      git changelog: false, credentialsId: '3360632d-fb8d-4f68-be83-b650ec243e94', poll: false, url: 'http://........git'
   }
   stage('Build') {
        withMaven(jdk: 'JDK8', maven: 'M3',
                globalMavenSettingsConfig: 'globalMavenSettings', options: [dependenciesFingerprintPublisher(), pipelineGraphPublisher(), artifactsPublisher(disabled: true)]) {
            writeFile file: '.archive-jenkins-maven-event-spy-logs', text: ''
            sh "mvn clean deploy"
        }
   }
}
{code}

Cyrille Le Clerc made changes - 2018-03-14 22:30

Assignee

Original: Alvaro Lobato [ alobato ]

New: Cyrille Le Clerc [ cleclerc ]

Cyrille Le Clerc added a comment - 2018-03-15 09:28 - edited

Reproducing a test case with

The runtime dependency of the overlay does NOT appear in the list of resolved dependencies of the maven project. Probably because it is flagged as a runtime dependency.

https://gist.github.com/cyrille-leclerc/bcb8e7d078570b7f7867eb55d0ed647c

The maven-war-plugin overlay docs clearly recommends to use a "runtime" scope

https://maven.apache.org/plugins/maven-war-plugin/overlays.html

<dependencies>
    <dependency>
      <groupId>com.example.projects</groupId>
      <artifactId>documentedprojectdependency</artifactId>
      <version>1.0-SNAPSHOT</version>
      <type>war</type>
      <scope>runtime</scope>
    </dependency>
    ...
  </dependencies>

Cyrille Le Clerc added a comment - 2018-03-15 09:28 - edited Reproducing a test case with https://github.com/cyrille-leclerc/my-war-overlay.git https://github.com/cyrille-leclerc/my-war.git The runtime dependency of the overlay does NOT appear in the list of resolved dependencies of the maven project. Probably because it is flagged as a runtime dependency. https://gist.github.com/cyrille-leclerc/bcb8e7d078570b7f7867eb55d0ed647c The maven-war-plugin overlay docs clearly recommends to use a "runtime" scope https://maven.apache.org/plugins/maven-war-plugin/overlays.html <dependencies> <dependency> <groupId> com.example.projects </groupId> <artifactId> documentedprojectdependency </artifactId> <version> 1.0-SNAPSHOT </version> <type> war </type> <scope> runtime </scope> </dependency> ... </dependencies>

Cyrille Le Clerc added a comment - 2018-03-15 18:49 - edited

benzht problem identified ^^.

Workaround: declare the war dependency of your overlay project with a scope "compile" instead of a scope "runtime". could this be acceptable for you?

Solving JENKINS-50205 will solve this issue.

Cyrille Le Clerc added a comment - 2018-03-15 18:49 - edited benzht problem identified ^^. Workaround: declare the war dependency of your overlay project with a scope "compile" instead of a scope "runtime". could this be acceptable for you? Solving JENKINS-50205 will solve this issue.

Cyrille Le Clerc made changes - 2018-03-15 19:09

Link

New: This issue is blocked by JENKINS-50205 [ JENKINS-50205 ]

Hartmut Benz added a comment - 2018-03-15 20:24

The work-around certainly makes the war-overlay being triggered after the war.jar has been built - thank you for that already.
During the weekend, I will study which other effects the change from 'runtime' to 'compile' has on the generated artifact and post the results here.

Thanks again for your support

Hartmut Benz added a comment - 2018-03-15 20:24 The work-around certainly makes the war-overlay being triggered after the war.jar has been built - thank you for that already. During the weekend, I will study which other effects the change from 'runtime' to 'compile' has on the generated artifact and post the results here. Thanks again for your support

Cyrille Le Clerc added a comment - 2018-03-15 20:55

Thanks for the kind word benzht. I don't know yet how we will be able to discover these runtime dependencies as they seem to not be catch by the org.apache.maven.project.DependencyResolutionResult event we are tracking.

Cyrille Le Clerc added a comment - 2018-03-15 20:55 Thanks for the kind word benzht . I don't know yet how we will be able to discover these runtime dependencies as they seem to not be catch by the org.apache.maven.project.DependencyResolutionResult event we are tracking.

Hartmut Benz added a comment - 2018-03-18 20:33

The work-around for the WAR works: the WAR versions built with compile and runtime scope are identical.

Sadly, the build-trigger is still not working but that seems to be related to ~~JENKINS-46313~~ and may require another ticket and some time to build a test scenario. My upstream project is multi-branch, multi-(maven)-module. The downstream WAR overlay basically depends on all branches with the right version number. But rom the point of view of the dependency-relationships, the branches all depend upon one another (but don't trigger each other).

Effectively, the downstream project depends on the upstream (branch) project directly (a–>b) and at the same time indirectly via other upstream branch projects (a–>a'–>b). And that seems to be the trigger-exclusion criteria of ~~JENKINS-46313~~.

cleclerc, do you see here a misconfiguration on my side or should I build a ticket with demo projects for it?

Hartmut Benz added a comment - 2018-03-18 20:33 The work-around for the WAR works: the WAR versions built with compile and runtime scope are identical. Sadly, the build-trigger is still not working but that seems to be related to JENKINS-46313 and may require another ticket and some time to build a test scenario. My upstream project is multi-branch, multi-(maven)-module. The downstream WAR overlay basically depends on all branches with the right version number. But rom the point of view of the dependency-relationships, the branches all depend upon one another (but don't trigger each other). Effectively, the downstream project depends on the upstream (branch) project directly (a–>b) and at the same time indirectly via other upstream branch projects (a–>a'–>b). And that seems to be the trigger-exclusion criteria of JENKINS-46313 . cleclerc , do you see here a misconfiguration on my side or should I build a ticket with demo projects for it?

Cyrille Le Clerc added a comment - 2018-03-20 13:58

I may have reproduced the problem. I'll add details to the recently introduced maven dashboard for build results (in master branch but not released yet) to help troubleshooting.

Cyrille Le Clerc added a comment - 2018-03-20 13:58 I may have reproduced the problem. I'll add details to the recently introduced maven dashboard for build results (in master branch but not released yet) to help troubleshooting.

Assignee:: Unassigned

Reporter:: Hartmut Benz

Votes:: 2 Vote for this issue

Watchers:: 5 Start watching this issue

Created:: 2018-03-11 17:20

Updated:: 2024-05-13 00:10

Jenkins

Details

Description

Problem

How to reproduce

Attachments

Issue Links

Activity

Collapse comment: Cyrille Le Clerc added a comment - 2018-03-15 09:28, Edited by Cyrille Le Clerc - 2018-03-15 18:43

Expand comment: Cyrille Le Clerc added a comment - 2018-03-15 09:28, Edited by Cyrille Le Clerc - 2018-03-15 18:43

Collapse comment: Cyrille Le Clerc added a comment - 2018-03-15 18:49, Edited by Cyrille Le Clerc - 2018-03-15 19:09

Expand comment: Cyrille Le Clerc added a comment - 2018-03-15 18:49, Edited by Cyrille Le Clerc - 2018-03-15 19:09

Collapse comment: Hartmut Benz added a comment - 2018-03-15 20:24

Expand comment: Hartmut Benz added a comment - 2018-03-15 20:24

Collapse comment: Cyrille Le Clerc added a comment - 2018-03-15 20:55

Expand comment: Cyrille Le Clerc added a comment - 2018-03-15 20:55

Collapse comment: Hartmut Benz added a comment - 2018-03-18 20:33

Expand comment: Hartmut Benz added a comment - 2018-03-18 20:33

Collapse comment: Cyrille Le Clerc added a comment - 2018-03-20 13:58

Expand comment: Cyrille Le Clerc added a comment - 2018-03-20 13:58

People

Dates