Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-22026

Propagate failure causes from downstream builds

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Original Jira :
      This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

      When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

      Updated bug :
      Looking at the BFA change log this is indeed supported as of version Version 1.6.0 (released Mar 10, 2014)
      However, testing this with a latest fresh install of Jenkins and an older (Jenkins version 1.625.2) with the minimum amount of plugin installed, I was unable to get this working. I've added several screenshots of the issue :

      Here is a screenshot of the master project I have setup :

      This project calls two instances of the slave project that way using Jenkins Parameterized Trigger plugin:

      When I browse to the two executions from this however BFA successfully triggered on the content of the subprojects :


      I have tried using different ways to call the sub project as I thought that this could be that, I managed to get the master call the slave project as if they were "Downstream projects" instead of "Subprojects" using different triggers system. Calling them as post-build action, calling them as build action, none of this was able to get the failure cause propagated upstream.

        Attachments

        1. screenshot-1.png
          screenshot-1.png
          67 kB
        2. screenshot-2.png
          screenshot-2.png
          51 kB
        3. screenshot-3.png
          screenshot-3.png
          51 kB

          Activity

          abayer Andrew Bayer created issue -
          Hide
          abayer Andrew Bayer added a comment -

          Turns out this is actually already done on master!

          Show
          abayer Andrew Bayer added a comment - Turns out this is actually already done on master!
          abayer Andrew Bayer made changes -
          Field Original Value New Value
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          rtyler R. Tyler Croy made changes -
          Workflow JNJira [ 154103 ] JNJira + In-Review [ 194790 ]
          mcorbion m corbion made changes -
          Issue Type Improvement [ 4 ] Bug [ 1 ]
          mcorbion m corbion made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          mcorbion m corbion made changes -
          Description This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.
          This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          mcorbion m corbion made changes -
          Description This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          Original Jira :
          This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          Updated bug :
          Looking at the BFA change log this is indeed supported as of version Version 1.6.0 (released Mar 10, 2014)
          However, testing this with a latest fresh install of Jenkins and an older (Jenkins version 1.625.2) with the minimum amount of plugin installed, I was unable to get this working. I've added several screenshots of the issue :
          mcorbion m corbion made changes -
          Attachment screenshot-1.png [ 33821 ]
          mcorbion m corbion made changes -
          Attachment screenshot-2.png [ 33822 ]
          mcorbion m corbion made changes -
          Attachment screenshot-3.png [ 33823 ]
          mcorbion m corbion made changes -
          Description Original Jira :
          This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          Updated bug :
          Looking at the BFA change log this is indeed supported as of version Version 1.6.0 (released Mar 10, 2014)
          However, testing this with a latest fresh install of Jenkins and an older (Jenkins version 1.625.2) with the minimum amount of plugin installed, I was unable to get this working. I've added several screenshots of the issue :
           !screenshot-1.png|thumbnail! Original Jira :
          This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          Updated bug :
          Looking at the BFA change log this is indeed supported as of version Version 1.6.0 (released Mar 10, 2014)
          However, testing this with a latest fresh install of Jenkins and an older (Jenkins version 1.625.2) with the minimum amount of plugin installed, I was unable to get this working. I've added several screenshots of the issue :

           !screenshot-2.png|thumbnail!


           !screenshot-3.png|thumbnail!
          mcorbion m corbion made changes -
          Description  !screenshot-1.png|thumbnail! Original Jira :
          This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          Updated bug :
          Looking at the BFA change log this is indeed supported as of version Version 1.6.0 (released Mar 10, 2014)
          However, testing this with a latest fresh install of Jenkins and an older (Jenkins version 1.625.2) with the minimum amount of plugin installed, I was unable to get this working. I've added several screenshots of the issue :

           !screenshot-2.png|thumbnail!


           !screenshot-3.png|thumbnail!
          Original Jira :
          This would be nice to have - I've got a fairly complex multi-level build setup, with a group of "generic" jobs for each platform we build on. Those generic jobs are then called by each of our 20+ component builds, passing certain information (i.e., what to build, where to find source packages) to the generic jobs as parameters.

          When the generic jobs (i.e., the downstream builds) fail, they get the appropriate known cause associated with them, but the parent build doesn't, since it doesn't actually have the cause of the downstream failure in its log. It'd be handy if we could propagate failure causes up the stack from the downstream builds to their upstream builds - it'd make analysis much simpler.

          Updated bug :
          Looking at the BFA change log this is indeed supported as of version Version 1.6.0 (released Mar 10, 2014)
          However, testing this with a latest fresh install of Jenkins and an older (Jenkins version 1.625.2) with the minimum amount of plugin installed, I was unable to get this working. I've added several screenshots of the issue :

          Here is a screenshot of the master project I have setup :

           !screenshot-1.png|thumbnail!

           This project calls two instances of the slave project that way using Jenkins Parameterized Trigger plugin:

           !screenshot-2.png|thumbnail!

          When I browse to the two executions from this however BFA successfully triggered on the content of the subprojects :

           !screenshot-3.png|thumbnail!
           I have tried using different ways to call the sub project as I thought that this could be that, I managed to get the master call the slave project as if they were "Downstream projects" instead of "Subprojects" using different triggers system. Calling them as post-build action, calling them as build action, none of this was able to get the failure cause propagated upstream.
          Hide
          ssbarnea Sorin Sbarnea added a comment - - edited

          This bug is still here and is it presents a serious usability issue on complex Jenkins setups. We have LOTS of Jenkins jobs that are starting other smaller jenkins jobs and due to this bug it's impossible to know what caused the failure of the top (parent) jobs ones because the BFA errors are not sent upwards.

          In some cases we need to navigate 2-3 levels down in order to discover that these failures.

          I think that the default behaviour of BFA should be to to inherit all the errors from from the child jobs. Example: http://s3.sbarnea.com/ss/python-saharaclient-nightly-rhos-10.0-coreci_46_rhos-10.0-patches_Jenkins_2017-01-09_18-43-42.png

          I think that most of our "master" jobs are using the [MultiJob Plugin](https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin).

          Show
          ssbarnea Sorin Sbarnea added a comment - - edited This bug is still here and is it presents a serious usability issue on complex Jenkins setups. We have LOTS of Jenkins jobs that are starting other smaller jenkins jobs and due to this bug it's impossible to know what caused the failure of the top (parent) jobs ones because the BFA errors are not sent upwards. In some cases we need to navigate 2-3 levels down in order to discover that these failures. I think that the default behaviour of BFA should be to to inherit all the errors from from the child jobs. Example: http://s3.sbarnea.com/ss/python-saharaclient-nightly-rhos-10.0-coreci_46_rhos-10.0-patches_Jenkins_2017-01-09_18-43-42.png I think that most of our "master" jobs are using the [MultiJob Plugin] ( https://wiki.jenkins-ci.org/display/JENKINS/Multijob+Plugin ).
          Hide
          t_westling Tomas Westling added a comment -

          I can also see this bug in newer (2.x) Jenkins versions, but not in 1.6.
          I'll dig a little deeper.

          Show
          t_westling Tomas Westling added a comment - I can also see this bug in newer (2.x) Jenkins versions, but not in 1.6. I'll dig a little deeper.
          Hide
          ssbarnea Sorin Sbarnea added a comment -

          In case this helps, I observed the lack of propagation on Jenkins ver. 1.625.1 – I do not own this instance which means that I will not be able to upgrade it. Still, if you have reasons to believe that is fixed in newer versions, I could try to move my jobs to an instance that is running Jenkins ver. 2.19.1 as this was planned anyway.

          Still, based on your comment I think that this may not solve the problem.

          Show
          ssbarnea Sorin Sbarnea added a comment - In case this helps, I observed the lack of propagation on Jenkins ver. 1.625.1 – I do not own this instance which means that I will not be able to upgrade it. Still, if you have reasons to believe that is fixed in newer versions, I could try to move my jobs to an instance that is running Jenkins ver. 2.19.1 as this was planned anyway. Still, based on your comment I think that this may not solve the problem.
          Hide
          t_westling Tomas Westling added a comment -

          From what I've observed, it works in 1.6 but not in 2.x, so upgrading shouldn't help anything.

          I can see that the issue comes from our use of reflection to find the Parameterized Trigger classes in order to find the downstream builds.
          I am not sure why this works in 1.6 but not in 2.x, but some changes to classloaders must have been made.
          Once https://github.com/jenkinsci/parameterized-trigger-plugin/pull/109 has been merged, we won't need the failing reflection code at all,
          and this issue should be solved.

          Show
          t_westling Tomas Westling added a comment - From what I've observed, it works in 1.6 but not in 2.x, so upgrading shouldn't help anything. I can see that the issue comes from our use of reflection to find the Parameterized Trigger classes in order to find the downstream builds. I am not sure why this works in 1.6 but not in 2.x, but some changes to classloaders must have been made. Once https://github.com/jenkinsci/parameterized-trigger-plugin/pull/109 has been merged, we won't need the failing reflection code at all, and this issue should be solved.
          Hide
          jjasie Jared Jasie added a comment -

          Tomas Westling rsandell

           

          Hi guys - has this fix been abandoned? 

          Show
          jjasie Jared Jasie added a comment - Tomas Westling rsandell   Hi guys - has this fix been abandoned? 
          Hide
          tomera Tomer Ab added a comment -

          Hi

          I also face this issue on ver 2.160.

          Is there a way i can contribute to fix this bug?

          Show
          tomera Tomer Ab added a comment - Hi I also face this issue on ver  2.160 . Is there a way i can contribute to fix this bug?
          Hide
          t_westling Tomas Westling added a comment -

          I haven't prioritized this issue, but will have a second look now. I'll analyze it again and get back to you.

          Show
          t_westling Tomas Westling added a comment - I haven't prioritized this issue, but will have a second look now. I'll analyze it again and get back to you.
          Hide
          pauxus Stephan Pauxberger added a comment - - edited

          I created a PR (https://github.com/jenkinsci/build-failure-analyzer-plugin/pull/107) which should fix this by using the downstream-build-cache, which does contain that information anyway.

          I will also work with pipeline jobs.

          Show
          pauxus Stephan Pauxberger added a comment - - edited I created a PR ( https://github.com/jenkinsci/build-failure-analyzer-plugin/pull/107) which should fix this by using the downstream-build-cache, which does contain that information anyway. I will also work with pipeline jobs.
          Hide
          fsteff Flemming Steffensen added a comment -

          I just tested the 1.23.0 release of this plugin on Jenkins 2.191 (on windows).

          Our setup is freestyle projects, calling other freestyle sub-projects.

          On failure, there is failures causes output on both the main project and the sub-projects, but  sub-project causes do not propagate into the main project. 

          Are there any special settings to be made for this to work?

          Show
          fsteff Flemming Steffensen added a comment - I just tested the 1.23.0 release of this plugin on Jenkins 2.191 (on windows). Our setup is freestyle projects, calling other freestyle sub-projects. On failure, there is failures causes output on both the main project and the sub-projects, but  sub-project causes do not propagate into the main project.  Are there any special settings to be made for this to work?

            People

            Assignee:
            t_westling Tomas Westling
            Reporter:
            abayer Andrew Bayer
            Votes:
            7 Vote for this issue
            Watchers:
            12 Start watching this issue

              Dates

              Created:
              Updated: