Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-7976

Aggregate downstream test results does not work for Matrix projects

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • core, matrix-project-plugin
    • None
    • Windows 7, Java 1.6.0_22, Hudson 1.383, xunit 1.11, copyartifact 1.10

      • create matrix projectA
      • add axis named x, values: a b
      • add build step that touches a file file.txt
      • archive file.txt as artifact, and fingerprint it
      • select to aggregate downstream test results
      • duplicate projectA, name it projectB
      • add build step to copy artifact file.txt from projectA/x=$x
      • fingerprint file.txt
      • seletc to publish testing tools report, do not fail build if tests not updated
      • build projectB once to create workspace, put a testresult xml in each
      • add post-build step to projectA 'build other projects' and select projectB
      • build projectA

      Test results from B do no make it into A:

      • status page of projectA, configuration a shows 'Latest Test Result (no tests)'
      • latest build page of projectA, configuration a shows 'Aggregated Test Result (no tests)'

      Following the same procedure, but for a normal project gives all the exepcted results though.

      (attached a zip with project config/workspaces that should be ready to try)

          [JENKINS-7976] Aggregate downstream test results does not work for Matrix projects

          Frode Carlsen added a comment -

          Any update on when this will be fixed?
          Also, will the solution work for matrix builds that run across multiple slaves?

          Frode Carlsen added a comment - Any update on when this will be fixed? Also, will the solution work for matrix builds that run across multiple slaves?

          nick tulett added a comment -

          I need to be able to report totals for run/failed tests in a matrix with multiple slaves and use the following URL:

          http://<JENKINS_HOST>/job/<JOB_NAME>/api/json?depth=2&tree=downstreamProjects[builds[actions[causes[upstreamBuild],failCount,totalCount]]]

          I then loop over the JSON and aggregate by upstreamBuild. It's not ideal but at least gives me some numbers to play with.

          nick tulett added a comment - I need to be able to report totals for run/failed tests in a matrix with multiple slaves and use the following URL: http://<JENKINS_HOST>/job/<JOB_NAME>/api/json?depth=2&tree=downstreamProjects[builds[actions[causes [upstreamBuild] ,failCount,totalCount]]] I then loop over the JSON and aggregate by upstreamBuild. It's not ideal but at least gives me some numbers to play with.

          It seems that matrix job don't aggregate test results at all from each axis. There is an issue with getting test results from axis? Could you please look into this deeper? The best example here are tests executed as one of axis and dasboard-view-plugin with Test Results Grid always displays 0 for matrix builds with tests.

          Ireneusz Makowski added a comment - It seems that matrix job don't aggregate test results at all from each axis. There is an issue with getting test results from axis? Could you please look into this deeper? The best example here are tests executed as one of axis and dasboard-view-plugin with Test Results Grid always displays 0 for matrix builds with tests.

          evernat added a comment -

          Is this highly voted issue reproduced with a recent Jenkins version?
          If yes, is someone wanting to care about it?

          evernat added a comment - Is this highly voted issue reproduced with a recent Jenkins version? If yes, is someone wanting to care about it?

          I still have it. +1 for the fix

          Ireneusz Makowski added a comment - I still have it. +1 for the fix

          Daniel Beck added a comment -

          If this still occurs on recent Jenkins versions (no older than eight weeks), please specify your Jenkins version, Matrix Project version, and JUnit plugin version (if Jenkins 1.577+).

          Daniel Beck added a comment - If this still occurs on recent Jenkins versions (no older than eight weeks), please specify your Jenkins version, Matrix Project version, and JUnit plugin version (if Jenkins 1.577+).

          Eli Stevens added a comment -

          This is still occurring for us:

          Jenkins ver. 1.580
          Matrix Project Plugin: 1.3
          JUnit Plugin: 1.1

          The linkage to downstream jobs also doesn't work.

          Eli Stevens added a comment - This is still occurring for us: Jenkins ver. 1.580 Matrix Project Plugin: 1.3 JUnit Plugin: 1.1 The linkage to downstream jobs also doesn't work.

          Jeremy Rampon added a comment -

          Same here.

          Jenkins ver. 1.581
          Matrix Project Plugin: 1.3
          JUnit Plugin: 1.1

          Jeremy Rampon added a comment - Same here. Jenkins ver. 1.581 Matrix Project Plugin: 1.3 JUnit Plugin: 1.1

          Darren Ball added a comment - - edited

          How can I reference the same logic as the filter when specifying downstream jobs?

          I see:

          You can workaround this by explicitly specifying the jobs to aggregate, rather than relying on the downstream builds logic, and specifying the matrix axes (in quotes) explicitly - i.e., NonMatrixJob,"MatrixJob/label=l_centos5_x86" - the quotes in case of commas in your axis.

          I need to supply multiple axis information for the jobs, can this be done and what would the syntax be?

          Update: I've figured this out - auto complete came into play once I started typing the matrix jobname. nice!

          Darren Ball added a comment - - edited How can I reference the same logic as the filter when specifying downstream jobs? I see: You can workaround this by explicitly specifying the jobs to aggregate, rather than relying on the downstream builds logic, and specifying the matrix axes (in quotes) explicitly - i.e., NonMatrixJob,"MatrixJob/label=l_centos5_x86" - the quotes in case of commas in your axis. I need to supply multiple axis information for the jobs, can this be done and what would the syntax be? Update: I've figured this out - auto complete came into play once I started typing the matrix jobname. nice!

          trejkaz added a comment - - edited

          Happens for us too.

          Jenkins ver. 2.170
          Matrix Project Plugin 1.12
          JUnit 1.26.1

          More info:

          • /job/matrix-job/563/ says "Test Result (no tests)"
          • /job/matrix-job/563/testReport/ shows the row for "os=windows,test=lib/metadata_item#Derby" but the test counts all say 0
          • If you click the link on that row it goes to /job/matrix-job/563/os=windows,test=lib%2Fmetadata_item%23Derby/testReport/ which shows the proper stats.

          Similar deal when viewing from the API.

          • /job/matrix-job/563/api/xml contains all 0 for MatrixTestResult
          • /job/matrix-job/563/testReport/api/xml shows all the expected elements for downstream reports, but counts are all 0
          • /job/matrix-job/563/os=windows,test=lib%2Fmetadata_item%23Derby/testReport/api/xml shows the proper counts

          So it's really like the aggregation gets all the links to individual test reports correct, but always shows 0 for the total tests run?

          trejkaz added a comment - - edited Happens for us too. Jenkins ver. 2.170 Matrix Project Plugin 1.12 JUnit 1.26.1 More info: /job/matrix-job/563/ says "Test Result (no tests)" /job/matrix-job/563/testReport/ shows the row for "os=windows,test=lib/metadata_item#Derby" but the test counts all say 0 If you click the link on that row it goes to /job/matrix-job/563/os=windows,test=lib%2Fmetadata_item%23Derby/testReport/ which shows the proper stats. Similar deal when viewing from the API. /job/matrix-job/563/api/xml contains all 0 for MatrixTestResult /job/matrix-job/563/testReport/api/xml shows all the expected elements for downstream reports, but counts are all 0 /job/matrix-job/563/os=windows,test=lib%2Fmetadata_item%23Derby/testReport/api/xml shows the proper counts So it's really like the aggregation gets all the links to individual test reports correct, but always shows 0 for the total tests run?

            Unassigned Unassigned
            apostasia apostasia
            Votes:
            48 Vote for this issue
            Watchers:
            48 Start watching this issue

              Created:
              Updated: