Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-50099

Downstream pipelines are not triggered when the artifact is a a WAR and consumed by war-overlay

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • pipeline-maven-plugin
    • None
    • Jenkins ver. 2.89.4, Slave on ubuntu, freshly updated plugins; Pipeline Maven Integration 3.4.2, maven 3.5.3

      Problem

      When a SNAPSHOT artifact is built (packaging = 'war'), its downstream project building a WAR-overlay depending on the upstream SNAPSHOT is not triggered.

      How to reproduce

      A test project (maven_war_overlay_project) with the described behaviour is available in the test_maven_projects directory on the fork in https://github.com/benzht/pipeline-maven-plugin/tree/JENKINS-50099 under the test_maven_projects (due to time constraints I could not integrate it into the test suite....).

      The new test project, maven_war_overlay_project, depends on the existing maven_war_project, which in turn depends on maven_jar_project.
      After manually building all of them, building the first correctly triggers the second, but maven_war_overlay_project is not triggered.
      Looking at the logging, it becomes clear that in the war_overlay_project no dependency is detected.

      Output from maven_war_project correctly records its dependency:

      ...
      [withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
      [withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
      [withMaven] pipelineGraphPublisher - Record dependency: jenkins.mvn.test:mono-module-maven-app:0.1-SNAPSHOT, ignoreUpstreamTriggers: false
      [withMaven] pipelineGraphPublisher - Skip recording release dependency: javax.servlet:javax.servlet-api:3.1.0
      [withMaven] pipelineGraphPublisher - Skip recording release dependency: junit:junit:4.12
      [withMaven] pipelineGraphPublisher - Skip recording release dependency: org.hamcrest:hamcrest-core:1.3
      [withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
      [withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/pom.xml
      [withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-app:0.1-SNAPSHOT, version: 0.1-20180311.164646-4, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_project/target/maven-war-app-0.1-SNAPSHOT.war
      ...
      

      Output from maven_war_overlay_project does not record any dependeny:

      ...
      [withMaven] pipelineGraphPublisher - recordParentProject - filter: versions[snapshot: true, release: false]
      [withMaven] pipelineGraphPublisher - recordDependencies - filter: versions[snapshot: true, release: false], scopes:[compile, provided, runtime]
      [withMaven] pipelineGraphPublisher - recordGeneratedArtifacts...
      [withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-SNAPSHOT, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/pom.xml
      [withMaven] pipelineGraphPublisher - Record generated artifact: jenkins.mvn.test:maven-war-overlay-app:0.1-SNAPSHOT, version: 0.1-20180311.164534-3, executedLifecyclePhases: [clean, process-resources, compile, process-test-resources, test-compile, test, package, install, deploy], skipDownstreamTriggers: false, lifecycleThreshold:deploy, file: /home/jenkins/slave/workspace/maven_war_overlay_project/target/maven-war-overlay-app.war
      

      Jenkinsfile of maven_war_overlay_project:

      node {
         stage('Preparation') { // for display purposes
            git changelog: false, credentialsId: '3360632d-fb8d-4f68-be83-b650ec243e94', poll: false, url: 'http://........git'
         }
         stage('Build') {
              withMaven(jdk: 'JDK8', maven: 'M3',
                      globalMavenSettingsConfig: 'globalMavenSettings', options: [dependenciesFingerprintPublisher(), pipelineGraphPublisher(), artifactsPublisher(disabled: true)]) {
                  writeFile file: '.archive-jenkins-maven-event-spy-logs', text: ''
                  sh "mvn clean deploy"
              }
         }
      }
      

          [JENKINS-50099] Downstream pipelines are not triggered when the artifact is a a WAR and consumed by war-overlay

          Pinned comments

          Pinned by Pino Silvaggio

          Pino Silvaggio added a comment - - edited

          A quick look at the code, I don't see "runtime"?

          Wouldn't adding "runtime" work?

          DependencyResolutionResultHandler

          line 50:

          private Set<String> includedScopes = new HashSet<String>(Arrays.asList("compile", "provided", "test"));

          to

          private Set<String> includedScopes = new HashSet<String>(Arrays.asList("compile", "runtime", "provided", "test")); 

           

           

           

          Pino Silvaggio added a comment - - edited A quick look at the code, I don't see "runtime"? Wouldn't adding "runtime" work? DependencyResolutionResultHandler line 50: private Set< String > includedScopes = new HashSet< String >(Arrays.asList( "compile" , "provided" , "test" )); to private Set< String > includedScopes = new HashSet< String >(Arrays.asList( "compile" , "runtime" , "provided" , "test" ));      

          All comments

          Thanks for the kind word benzht. I don't know yet how we will be able to discover these runtime dependencies as they seem to not be catch by the org.apache.maven.project.DependencyResolutionResult event we are tracking.

          Cyrille Le Clerc added a comment - Thanks for the kind word benzht . I don't know yet how we will be able to discover these runtime dependencies as they seem to not be catch by the org.apache.maven.project.DependencyResolutionResult event we are tracking.

          Hartmut Benz added a comment -

          The work-around for the WAR works: the WAR versions built with compile and runtime scope are identical.

          Sadly, the build-trigger is still not working but that seems to be related to JENKINS-46313 and may require another ticket and some time to build a test scenario. My upstream project is multi-branch, multi-(maven)-module. The downstream WAR overlay basically depends on all branches with the right version number. But rom the point of view of the dependency-relationships, the branches all depend upon one another (but don't trigger each other).

          Effectively, the downstream project depends on the upstream (branch) project directly (a–>b) and at the same time indirectly via other upstream branch projects (a–>a'–>b). And that seems to be the trigger-exclusion criteria of JENKINS-46313.

          cleclerc, do you see here a misconfiguration on my side or should I build a ticket with demo projects for it?

          Hartmut Benz added a comment - The work-around for the WAR works: the WAR versions built with compile and runtime scope are identical. Sadly, the build-trigger is still not working but that seems to be related to  JENKINS-46313 and may require another ticket and some time to build a test scenario. My upstream project is multi-branch, multi-(maven)-module. The downstream WAR overlay basically depends on all branches with the right version number. But rom the point of view of the dependency-relationships, the branches all depend upon one another (but don't trigger each other). Effectively, the downstream project depends on the upstream (branch) project directly (a–>b) and at the same time indirectly via other upstream branch projects (a–>a'–>b). And that seems to be the trigger-exclusion criteria of  JENKINS-46313 . cleclerc , do you see here a misconfiguration on my side or should I build a ticket with demo projects for it?

          I may have reproduced the problem. I'll add details to the recently introduced maven dashboard for build results (in master branch but not released yet) to help troubleshooting.

          Cyrille Le Clerc added a comment - I may have reproduced the problem. I'll add details to the recently introduced maven dashboard for build results (in master branch but not released yet) to help troubleshooting.

          Cyrille Le Clerc added a comment - - edited

          benzht can you install v3.5.0 and look at the "Maven" build details screen for your war pipeline and your war-overlay pipeline?

          Please look at the "dependencies" of the war-overlay pipeline and at the "downstream pipelines" of the war pipeline

          Cyrille Le Clerc added a comment - - edited benzht can you install v3.5.0 and look at the "Maven" build details screen for your war pipeline and your war-overlay pipeline? Please look at the "dependencies" of the war-overlay pipeline and at the "downstream pipelines" of the war pipeline

          Hartmut Benz added a comment - - edited

          cleclerc, I have upgraded everything to 'current' including the 3.5 version.

          The upstream project (on matrix level) list the following
          Downstream Jobs

          • Downstream1 » develop
          • Downstream2 » develop
          • upstream » develop
          • upstream » feature/jenkinsWork
          • upstream » feature/t6482_keycloak
          • upstream » feature/testServer

          The downstream projects Downstream1.develop and Downstream2.develop list the following
          Upstream Builds

          • upstream » feature/testNewServer #78
          • upstream » feature/jenkinsWork #24
          • upstream » develop #40        <—this is the most recent build number
          • upstream » feature/t6482_keycloak #3

          The downstream matrix project config has Build on Snapshot dependency set.

          But Upstream-develop Build #40 did not trigger either of the Downstream projects. This is the end of the logs:

          [withMaven] Not triggering Downsteam1 » develop because it has dependencies in the downstream project list
          [withMaven] Not triggering Downsteam2 » develop because it has dependencies in the downstream project list
          Finished: SUCCESS
          

          Is there anything else I can test, log, or try to help you further?

          Hartmut Benz added a comment - - edited cleclerc , I have upgraded everything to 'current' including the 3.5 version. The upstream project (on matrix level) list the following Downstream Jobs Downstream1 » develop Downstream2 » develop upstream » develop upstream » feature/jenkinsWork upstream » feature/t6482_keycloak upstream » feature/testServer The downstream projects Downstream1.develop and Downstream2.develop list the following Upstream Builds upstream » feature/testNewServer #78 upstream » feature/jenkinsWork #24 upstream » develop #40        <—this is the most recent build number upstream » feature/t6482_keycloak #3 The downstream matrix project config has Build on Snapshot dependency set. But Upstream-develop Build #40 did not trigger either of the Downstream projects. This is the end of the logs: [withMaven] Not triggering Downsteam1 » develop because it has dependencies in the downstream project list [withMaven] Not triggering Downsteam2 » develop because it has dependencies in the downstream project list Finished: SUCCESS Is there anything else I can test, log, or try to help you further?

          Hartmut Benz added a comment -

          My best guess would be to exclude all sister-branches in a multi-branch project from being considered as being downstream. That way they would not prevent the real downstream projects from being triggered.

          Hartmut Benz added a comment - My best guess would be to exclude all sister-branches in a multi-branch project from being considered as being downstream . That way they would not prevent the real downstream projects from being triggered.

          Maxime Michel added a comment - - edited

          benzht I haven't had time to look at the setup you described and compare it with ours, however, I am seeing the exact same message with a non-multibranch setup:

          Basically, a module pipeline is included in a webapp pipeline, which is then overlaid in another pipeline. Building the module results in:

          [withMaven] Not triggering webapp because it has dependencies in the downstream project list
          [withMaven] Not triggering overlaid-webapp because it has dependencies in the downstream project list
          

          To me the issue seems to be that both webapps are listed twice as dependency to themselves in the Maven dashboard. That doesn't look right, shouldn't self be excluded from being a possible dependency to any job?

          Maxime Michel added a comment - - edited benzht I haven't had time to look at the setup you described and compare it with ours, however, I am seeing the exact same message with a non-multibranch setup: Basically, a module pipeline is included in a webapp pipeline, which is then overlaid in another pipeline. Building the module results in: [withMaven] Not triggering webapp because it has dependencies in the downstream project list [withMaven] Not triggering overlaid-webapp because it has dependencies in the downstream project list To me the issue seems to be that both webapps are listed twice as dependency to themselves in the Maven dashboard. That doesn't look right, shouldn't self be excluded from being a possible dependency to any job?

          Note: as war-overlay "dependency" is not a "standard Maven dependency", we may look at solving this problem in conjunction with solving JENKINS-50475 (Maven BOM Dependency) which is not either a "standard Maven dependency".

          Cyrille Le Clerc added a comment - Note: as war-overlay "dependency" is not a "standard Maven dependency", we may look at solving this problem in conjunction with solving  JENKINS-50475 (Maven BOM Dependency) which is not either a "standard Maven dependency".

          war-overlay and maven-bom dependencies are not "standard Maven dependencies", we may look at solving these 2 problems simultaneously.

          Cyrille Le Clerc added a comment - war-overlay and maven-bom dependencies are not "standard Maven dependencies", we may look at solving these 2 problems simultaneously.

          Pinned by Pino Silvaggio

          Pino Silvaggio added a comment - - edited

          A quick look at the code, I don't see "runtime"?

          Wouldn't adding "runtime" work?

          DependencyResolutionResultHandler

          line 50:

          private Set<String> includedScopes = new HashSet<String>(Arrays.asList("compile", "provided", "test"));

          to

          private Set<String> includedScopes = new HashSet<String>(Arrays.asList("compile", "runtime", "provided", "test")); 

           

           

           

          Pino Silvaggio added a comment - - edited A quick look at the code, I don't see "runtime"? Wouldn't adding "runtime" work? DependencyResolutionResultHandler line 50: private Set< String > includedScopes = new HashSet< String >(Arrays.asList( "compile" , "provided" , "test" )); to private Set< String > includedScopes = new HashSet< String >(Arrays.asList( "compile" , "runtime" , "provided" , "test" ));      

            Unassigned Unassigned
            benzht Hartmut Benz
            Votes:
            2 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated: