Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-6174

Build whenever a SNAPSHOT dependency is built trigger is not working or seems to be hazardous

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • maven-plugin
    • None

      We have a quite large setup of Hudson with ~110 maven2 jobs categorized in server, common and full-build categories. As you'd expect, server-type jobs depend on some of the commons, and some common jobs depend on each other. Jobs in server and common categories always produce one JAR per each job, run the tests and deploy the artifact in the maven repo, if the build is stable. Piece of cake.

      The only job in the full-build category assembles an EAR file from the available artifacts in maven (we don't build the full product from source and I will not tell you why, there are reasons for it), deploys it to a container and says hello to developers. Piece of cake again.

      Accurev is setup to trigger the appropriate job if a commit (promote) arrives in the depot, this works fine.

      To formalize our setup:
      jobC1 builds moduleC1 which creates and deploys artifactC1
      jobC2 builds moduleC2 which creates and deploys artifactC2
      jobC3 builds moduleC3 which creates and deploys artifactC3, depends on artifactC1

      jobS1 builds moduleS1 which creates and deploys artifactS1, depends on artifactC1
      jobS1 builds moduleS1 which creates and deploys artifactS1, depends on artifactC1, artifactC2
      jobS1 builds moduleS1 which creates and deploys artifactS1, depends on artifactC2, artifactC3

      jobF builds moduleF which creates artifactF (the full build), depends on all artifacts

      All produced artifacts are maven2 SNAPSHOT versions. Always.

      What I'd like to achieve is that if the Accurev post-promote trigger pokes jobC1 (it does), that build should create artifactC1 (it does), and after this, all jobs that depend on artifactC1 (jobS1, jobS2, jobS3) should be triggered and built. AFAIK this was achievable with the "Build whenever a SNAPSHOT dependency is built" trigger in the 1.2xx series (last time I've checked Hudson was something like a year ago), but if I check that option in all jobs, no automatic dependency building happens. I know I could check all modules manually with "build other projects", but no thanks, that would cost a week's work and would cause lots of unflexibility.

      What's interesting is, though, that jobF (the full EAR assembly build) becomes the downstream job of all other jobs but it seems that it gets triggered only occasionally (or most probably never).

      Any hints or tips or fixes are welcome, I'm more than happy to help investigating this issue.

      We're using the most recent Hudson version (1.353) with updated plugins. The Accurev plugin uses no workspace, just pops the contents of a stream.

          [JENKINS-6174] Build whenever a SNAPSHOT dependency is built trigger is not working or seems to be hazardous

          This issue is not solved.
          We use version 1.443 with maven 2.

          We have a project like this:

          master project (pom.xml type pom)
          – core project (pom.xml type jar)
          – web project (pom.xml type war, depends on core)
          – ear project (pom.xml type ear, depends on web)

          In jenkins, we define 4 jobs, one for each module:
          project-master > clean install -e -X -N (non recursive)
          project-core > clean install -e -X
          project-web > clean install -e -X
          project-ear > clean install -e -X

          And for each of them, we check the box "build whenever a snapshot dependency is built"

          => it works fine, for example, when the core is built, it triggers the web, then the ear.

          The problem comes when we create a new job, let say "project-nightly" that will be executed each night on the complete project.
          project-nightly > clean deploy -e -X

          The creation of this last jobs breaks the triggering of all other jobs !!
          So when the core is built, the web is not triggered.
          And when the nightly is build, all other jobs are triggered !!!

          Olivier Lambert added a comment - This issue is not solved. We use version 1.443 with maven 2. We have a project like this: master project (pom.xml type pom) – core project (pom.xml type jar) – web project (pom.xml type war, depends on core) – ear project (pom.xml type ear, depends on web) In jenkins, we define 4 jobs, one for each module: project-master > clean install -e -X -N (non recursive) project-core > clean install -e -X project-web > clean install -e -X project-ear > clean install -e -X And for each of them, we check the box "build whenever a snapshot dependency is built" => it works fine, for example, when the core is built, it triggers the web, then the ear. The problem comes when we create a new job, let say "project-nightly" that will be executed each night on the complete project. project-nightly > clean deploy -e -X The creation of this last jobs breaks the triggering of all other jobs !! So when the core is built, the web is not triggered. And when the nightly is build, all other jobs are triggered !!!

          kutzi added a comment -

          Olivier, this works as designed. There's always only the dependency between one upstream project and the downstream project. So after configuring the "project-nightly" job, Jenkins decided that this job should be the preferred one, because deploy goals have prefeference over install.

          I think, I've got an open ToDo to document the exact rules used to determine the up-/downstream relationship by Jenkins.

          kutzi added a comment - Olivier, this works as designed. There's always only the dependency between one upstream project and the downstream project. So after configuring the "project-nightly" job, Jenkins decided that this job should be the preferred one, because deploy goals have prefeference over install. I think, I've got an open ToDo to document the exact rules used to determine the up-/downstream relationship by Jenkins.

          kutzi added a comment -

          Note that there are several open issues (e.g. JENKINS-1295) which ask for a more fine-grained configuration of up-/downstream relationships.

          The current automatic approach is one which works IMHO in most of the cases 'as expected', but there are surely always cases - like yours - were an automatic mechanism doesn't fit the expectations.

          kutzi added a comment - Note that there are several open issues (e.g. JENKINS-1295 ) which ask for a more fine-grained configuration of up-/downstream relationships. The current automatic approach is one which works IMHO in most of the cases 'as expected', but there are surely always cases - like yours - were an automatic mechanism doesn't fit the expectations.

          Ok. Right !

          But let me explain the behaviour that I would like to have...

          • I have one slave and the master.
          • I would like to define jobs for each modules (so 4 jobs), that builds on the slave (clean install), and that are triggered by a commit (so polling). The goal is to check continuously that all components build fine with the modifications done by developpers during the day. So if a commit is done in project-core, then the project-web should also build ... aso => because it's just an install, developpers are not impacted by new snapshot artifacts during the day.
          • I also would like to have a nightly build that is executed on the master (clean deploy) each night. This way, new snapshot artifacts will be available for download on the following morning. Pay attention that the nightly build should not trigger component jobs on the slave.

          Do you have a solution or some tips to help me configuring this kind of behaviour ?

          Olivier Lambert added a comment - Ok. Right ! But let me explain the behaviour that I would like to have... I have one slave and the master. I would like to define jobs for each modules (so 4 jobs), that builds on the slave (clean install), and that are triggered by a commit (so polling). The goal is to check continuously that all components build fine with the modifications done by developpers during the day. So if a commit is done in project-core, then the project-web should also build ... aso => because it's just an install, developpers are not impacted by new snapshot artifacts during the day. I also would like to have a nightly build that is executed on the master (clean deploy) each night. This way, new snapshot artifacts will be available for download on the following morning. Pay attention that the nightly build should not trigger component jobs on the slave. Do you have a solution or some tips to help me configuring this kind of behaviour ?

          kutzi added a comment -

          I guess a fix of JENKINS-1295 would be the right fix ....

          There's one thing you could try, but no guarantees that it'll work - I've never tried it.

          • Put the deploy goal (or a deploy:deploy) in an M2 post-build step while only having an install in the normal goals.
          • Rename the deploy job so that it come alphabetically after all other jobs (i.e. put a 'Z' in front)

          That way it could work

          BTW: please redirect all further questions into the users list. This is not the right place for discussions

          kutzi added a comment - I guess a fix of JENKINS-1295 would be the right fix .... There's one thing you could try, but no guarantees that it'll work - I've never tried it. Put the deploy goal (or a deploy:deploy) in an M2 post-build step while only having an install in the normal goals. Rename the deploy job so that it come alphabetically after all other jobs (i.e. put a 'Z' in front) That way it could work BTW: please redirect all further questions into the users list. This is not the right place for discussions

          Hannes Kogler added a comment -

          I dont know why this issue is marked as "not reproduceable"?

          We are using 1.518 version of Jenkins, and the problem is still here.

          Like "Olivier Lambert" described it... => The problem is that the automated downstream dependency triggering chain won't work, if you have any other newer job on the same Jenkins instance, where the same versioned SNAPSHOT-project gets built.

          But that's a killer! We really need other jobs that build the same projects, because we have the module- Continous integration - build conecpt on the one hand and the monolothic build approach for release builds on the other hand.

          so what would be the problem, that Jenkins just BUILDS ALL jobs with related project dependency versions? (instead of just the "newest" like it obviously does)

          Hannes Kogler added a comment - I dont know why this issue is marked as "not reproduceable"? We are using 1.518 version of Jenkins, and the problem is still here. Like "Olivier Lambert" described it... => The problem is that the automated downstream dependency triggering chain won't work, if you have any other newer job on the same Jenkins instance, where the same versioned SNAPSHOT-project gets built. But that's a killer! We really need other jobs that build the same projects, because we have the module- Continous integration - build conecpt on the one hand and the monolothic build approach for release builds on the other hand. so what would be the problem, that Jenkins just BUILDS ALL jobs with related project dependency versions? (instead of just the "newest" like it obviously does)

          kutzi added a comment -

          As stated in an earlier comment, this is currently working as designed.
          If you want a different behaviour, you should open a new feature request with a concrete description what you like to have.

          PS: it's not the 'newest' job which is triggering, but the one with the most relevance (e.g. deploying jobs)
          PPS: maybe you can achieve what you want with explicitly configured dependencies between your jobs

          kutzi added a comment - As stated in an earlier comment, this is currently working as designed. If you want a different behaviour, you should open a new feature request with a concrete description what you like to have. PS: it's not the 'newest' job which is triggering, but the one with the most relevance (e.g. deploying jobs) PPS: maybe you can achieve what you want with explicitly configured dependencies between your jobs

          Boris Romashov added a comment - - edited

          kutzi - I suppose that when I check "Build whenever a SNAPSHOT dependency is built" I have no idea about the fact that Jenkins is going to choose just one upstream project for anything. The only proper behaviour of this option - is to build absolutely everything that depends on this job. At least, this is the most expected behaviour.
          So you should first rename current option or at least give some self-explanatory hint about this fact.
          And second - add another option that does what everyone expects.

          Is this worth creating another issue? Maybe we can edit this issue to make it the feature request ?

          Boris Romashov added a comment - - edited kutzi - I suppose that when I check "Build whenever a SNAPSHOT dependency is built" I have no idea about the fact that Jenkins is going to choose just one upstream project for anything. The only proper behaviour of this option - is to build absolutely everything that depends on this job. At least, this is the most expected behaviour. So you should first rename current option or at least give some self-explanatory hint about this fact. And second - add another option that does what everyone expects. Is this worth creating another issue? Maybe we can edit this issue to make it the feature request ?

          kutzi added a comment -

          Boris IMO there are several good reasons for having it the way as it is.
          E.g. you could have a cascade of jobs building the upstream project, each one running longer and longer tests/reports.
          E.g. build+unit tests -> component tests -> performance tests -> reports (FindBugs, Code Coverage etc.)
          Usually you want only one of these jobs to trigger the downstream job(s) and not each one

          That being said, I can also see that there may be other valid scenarios where you want each upstream job to trigger downstream jobs. And the documentation for the existing can certainly be improved.

          Yes, as I said already: please create a feature request if you want this to change

          kutzi added a comment - Boris IMO there are several good reasons for having it the way as it is. E.g. you could have a cascade of jobs building the upstream project, each one running longer and longer tests/reports. E.g. build+unit tests -> component tests -> performance tests -> reports (FindBugs, Code Coverage etc.) Usually you want only one of these jobs to trigger the downstream job(s) and not each one That being said, I can also see that there may be other valid scenarios where you want each upstream job to trigger downstream jobs. And the documentation for the existing can certainly be improved. Yes, as I said already: please create a feature request if you want this to change

          Boris Romashov added a comment - - edited

          My feature request - JENKINS-13390.
          Anybody here - please vote.

          Boris Romashov added a comment - - edited My feature request - JENKINS-13390 . Anybody here - please vote.

            tamasmezei tamasmezei
            tamasmezei tamasmezei
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: