[JENKINS-48020] Warning log spam "[hudson.tasks.junit.CaseResult getRun] In getOwner(), suiteResult.getParent() is null."

Type: Bug
Resolution: Fixed
Priority: Critical
Component/s: junit-plugin
Labels:
None

Similar Issues:
Powered by SuggestiMate

Show

After upgrading Jenkins plugins (presumably irrelevant durable-task => 1.16): junit 1.21 => 1.22.1

***@***:~$ grep "In getOwner" /var/log/jenkins/jenkins.log
...
2017-11-15 10:54:05 WARNING [hudson.tasks.junit.CaseResult getRun]   In getOwner(), suiteResult.getParent() is null.
***@***:~$ grep "In getOwner" /var/log/jenkins/jenkins.log | wc -l
820

Actually I had hoped to benefit from ~~JENKINS-27395~~ (due to having two different stages using the junit step for recording the test results), but in fact everything looks like before, except the new excessive warning log spam. Since the number of tests is in fact 678, I am also wondering about the number of 820 logs.

The junit step is called like this: junit(keepLongStdio: true, testResults: phpUnitXmlFilename); as aforementioned in two different stages for different XML Files, each one stemming from PHPUnit 6.4.3

causes

JENKINS-61787 jenkins.model.Jenkins loaded from parser callable via TestNameTransformer

Resolved

is related to

JENKINS-27395 distinguish test reports filed from different parts of workflow

Resolved

links to

PR #89

Andrew Bayer added a comment - 2017-11-15 14:30

Any chance you could post your Jenkinsfile and your test reports xml?

Andrew Bayer added a comment - 2017-11-15 14:30 Any chance you could post your Jenkinsfile and your test reports xml?

Andrew Bayer added a comment - 2017-11-15 14:49

Ok, got one part - the failed tests lists don't include the stage name. If you drill down in the full test list to a particular test class, you'll see the test names including the stage name, but not in the list of failed tests, or the list of packages or classes. The latter two are sadly just what we're stuck with - it's entirely possible for a package or class result to contain tests from multiple stages, so there's no viable way I've found yet to connect package/class results with a specific stage. The failed tests, though, should have the stage included, but they pull their display name differently than in the case list, so I need to fix that.

I can't reproduce the warnings, though - as mentioned, your Jenkinsfile, and particularly your test results XML file that the step is reading, would be hugely helpful for that.

Andrew Bayer added a comment - 2017-11-15 14:49 Ok, got one part - the failed tests lists don't include the stage name. If you drill down in the full test list to a particular test class, you'll see the test names including the stage name, but not in the list of failed tests, or the list of packages or classes. The latter two are sadly just what we're stuck with - it's entirely possible for a package or class result to contain tests from multiple stages, so there's no viable way I've found yet to connect package/class results with a specific stage. The failed tests, though, should have the stage included, but they pull their display name differently than in the case list, so I need to fix that. I can't reproduce the warnings, though - as mentioned, your Jenkinsfile, and particularly your test results XML file that the step is reading, would be hugely helpful for that.

Andrew Bayer added a comment - 2017-11-15 15:04

Check that, got the warning. It shows up when getting trends. Working on it.

Andrew Bayer added a comment - 2017-11-15 15:04 Check that, got the warning. It shows up when getting trends. Working on it.

Reinhold Füreder added a comment - 2017-11-15 15:07

Well, the Jenkinsfile actually depends heavily on a shared pipeline library, but the basic principle is that in two different stages ("docker-unit-tests" and "docker-integration-tests") PHPUnit (v6.4.3) is executed in a Docker Container via "phpunit --log-junit phpunit-result.xml ..."
... and the resulting XML file processed like this:

def phpUnitXmlFilename = 'phpunit-result.xml'
...
junit(keepLongStdio: true, testResults: phpUnitXmlFilename)

(Maybe mind that it is "only" one single XML file in each run, and that the filename does not change.)

Which of the test reports XML do you need: the one written by Jenkins in "/var/lib/jenkins/jobs/<job name>/builds/<build number>/junitResult.xml" or the PHPUnit output files?

The former one looks like this:

<?xml version='1.0' encoding='UTF-8'?>
<result plugin="junit@1.22.1">
  <suites>
    <suite>
      <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file>
      <name>ACME_Accounting_Accountings_FileImporters_UBSTest</name>
      <duration>0.009598</duration>
      <time>0.009598</time>
      <nodeId>51</nodeId>
      <enclosingBlocks>
        <string>34</string>
      </enclosingBlocks>
      <enclosingBlockNames>
        <string>docker-unit-tests</string>
      </enclosingBlockNames>
      <cases>
        <case>
          <duration>0.00624</duration>
          <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className>
          <testName>testCheckFile</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
        <case>
          <duration>0.002944</duration>
          <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className>
          <testName>testImport</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
        <case>
          <duration>4.14E-4</duration>
          <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className>
          <testName>testPossibleTenantsKeys</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
      </cases>
    </suite>
    <suite>
      <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file>
      <name>ACME_Accounting_Export_Abacus_GeneratorTest</name>
...
    <suite>
      <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file>
      <name>TestFramework_DbFixtureGenerator_MeasuringModelBuilderTest</name>
      <duration>0.017027</duration>
      <time>0.017027</time>
      <nodeId>80</nodeId>
      <enclosingBlocks>
        <string>61</string>
      </enclosingBlocks>
      <enclosingBlockNames>
        <string>docker-integration-tests</string>
      </enclosingBlockNames>
      <cases>
        <case>
          <duration>0.004158</duration>
          <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
          <testName>testCreate</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
        <case>
          <duration>0.00341</duration>
          <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
          <testName>testCreateNotCurrent</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
        <case>
          <duration>0.004291</duration>
          <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
          <testName>testCreateWithOneDefaultPoint</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
        <case>
          <duration>0.005168</duration>
          <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className>
          <testName>testCreateWithPoints</testName>
          <skipped>false</skipped>
          <failedSince>0</failedSince>
        </case>
      </cases>
    </suite>
  </suites>
  <duration>16.892485</duration>
  <keepLongStdio>true</keepLongStdio>
</result>

The latter ones I don't have now (they are thrown away), but I could of course try to get them if needed...

Oh, I just found one from yesterday which should IMHO still be up-to-date:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="./unittests/unit/" tests="482" assertions="925" errors="0" failures="0" skipped="0" time="0.780566">
    <testsuite name="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" tests="3" assertions="76" errors="0" failures="0" skipped="0" time="0.010088">
      <testcase name="testCheckFile" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="11" assertions="2" time="0.006981"/>
      <testcase name="testImport" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="21" assertions="73" time="0.002706"/>
      <testcase name="testPossibleTenantsKeys" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="45" assertions="1" time="0.000401"/>
    </testsuite>
    <testsuite name="ACME\Accounting\Export\Abacus\GeneratorTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/export/abacus/GeneratorTest.php" tests="1" assertions="1" errors="0" failures="0" skipped="0" time="0.013121">
...
  </testsuite>
</testsuites>

Reinhold Füreder added a comment - 2017-11-15 15:07 Well, the Jenkinsfile actually depends heavily on a shared pipeline library, but the basic principle is that in two different stages ("docker-unit-tests" and "docker-integration-tests") PHPUnit (v6.4.3) is executed in a Docker Container via " phpunit --log-junit phpunit-result.xml ... " ... and the resulting XML file processed like this: def phpUnitXmlFilename = 'phpunit-result.xml' ... junit(keepLongStdio: true , testResults: phpUnitXmlFilename) (Maybe mind that it is "only" one single XML file in each run, and that the filename does not change.) Which of the test reports XML do you need: the one written by Jenkins in "/var/lib/jenkins/jobs/<job name>/builds/<build number>/junitResult.xml" or the PHPUnit output files? The former one looks like this: <?xml version='1.0' encoding='UTF-8'?> <result plugin="junit@1.22.1"> <suites> <suite> <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file> <name>ACME_Accounting_Accountings_FileImporters_UBSTest</name> <duration>0.009598</duration> <time>0.009598</time> <nodeId>51</nodeId> <enclosingBlocks> <string>34</string> </enclosingBlocks> <enclosingBlockNames> <string>docker-unit-tests</string> </enclosingBlockNames> <cases> <case> <duration>0.00624</duration> <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className> <testName>testCheckFile</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.002944</duration> <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className> <testName>testImport</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>4.14E-4</duration> <className>ACME_Accounting_Accountings_FileImporters_UBSTest</className> <testName>testPossibleTenantsKeys</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> </cases> </suite> <suite> <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file> <name>ACME_Accounting_Export_Abacus_GeneratorTest</name> ... <suite> <file>/var/lib/jenkins/workspace/ACME-Pipeline-WS/phpunit-result.xml</file> <name>TestFramework_DbFixtureGenerator_MeasuringModelBuilderTest</name> <duration>0.017027</duration> <time>0.017027</time> <nodeId>80</nodeId> <enclosingBlocks> <string>61</string> </enclosingBlocks> <enclosingBlockNames> <string>docker-integration-tests</string> </enclosingBlockNames> <cases> <case> <duration>0.004158</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreate</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.00341</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreateNotCurrent</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.004291</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreateWithOneDefaultPoint</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> <case> <duration>0.005168</duration> <className>TestFramework.DbFixtureGenerator.MeasuringModelBuilderTest</className> <testName>testCreateWithPoints</testName> <skipped>false</skipped> <failedSince>0</failedSince> </case> </cases> </suite> </suites> <duration>16.892485</duration> <keepLongStdio>true</keepLongStdio> </result> The latter ones I don't have now (they are thrown away), but I could of course try to get them if needed... Oh, I just found one from yesterday which should IMHO still be up-to-date: <?xml version="1.0" encoding="UTF-8"?> <testsuites> <testsuite name="./unittests/unit/" tests="482" assertions="925" errors="0" failures="0" skipped="0" time="0.780566"> <testsuite name="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" tests="3" assertions="76" errors="0" failures="0" skipped="0" time="0.010088"> <testcase name="testCheckFile" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="11" assertions="2" time="0.006981"/> <testcase name="testImport" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="21" assertions="73" time="0.002706"/> <testcase name="testPossibleTenantsKeys" class="ACME_Accounting_Accountings_FileImporters_UBSTest" classname="ACME_Accounting_Accountings_FileImporters_UBSTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/accountings/fileimporters/ubsTest.php" line="45" assertions="1" time="0.000401"/> </testsuite> <testsuite name="ACME\Accounting\Export\Abacus\GeneratorTest" file="/var/lib/jenkins/workspace/ACME-Pipeline-WS@3/unittests/unit/app/admin/classes/accounting/export/abacus/GeneratorTest.php" tests="1" assertions="1" errors="0" failures="0" skipped="0" time="0.013121"> ... </testsuite> </testsuites>

Andrew Bayer added a comment - 2017-11-15 15:12

Perfect - thanks!

Andrew Bayer added a comment - 2017-11-15 15:12 Perfect - thanks!

Reinhold Füreder added a comment - 2017-11-15 15:15 - edited

Frankly I am not sure that I understood your comment in https://issues.jenkins-ci.org/browse/JENKINS-48020?focusedCommentId=319785&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-319785

Note that in my example:

there were no failing tests
there are definitely different test cases in same test classes (= test classes with same name) executed in the two different PHPUnit runs (filtered executions based on "@group" test annotations)
- but I would strongly assume there are some "unique" test classes per PHPUnit run as well

Reinhold Füreder added a comment - 2017-11-15 15:15 - edited Frankly I am not sure that I understood your comment in https://issues.jenkins-ci.org/browse/JENKINS-48020?focusedCommentId=319785&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-319785 Note that in my example: there were no failing tests there are definitely different test cases in same test classes (= test classes with same name) executed in the two different PHPUnit runs (filtered executions based on " @group " test annotations) but I would strongly assume there are some "unique" test classes per PHPUnit run as well

Andrew Bayer added a comment - 2017-11-15 15:28

Basically, I was saying that the stage info isn't shown except for on individual tests - so, not in the list of packages on the initial test results screen, not on the list of classes when you click on one of the packages. You will see it in the list of cases within an individual test class, though. And with https://github.com/jenkinsci/junit-plugin/pull/89, the lists of failed tests will have the stage info as well.

(this is all in the classic UI - Blue Ocean isn't yet updated to take advantage of this information)

Andrew Bayer added a comment - 2017-11-15 15:28 Basically, I was saying that the stage info isn't shown except for on individual tests - so, not in the list of packages on the initial test results screen, not on the list of classes when you click on one of the packages. You will see it in the list of cases within an individual test class, though. And with https://github.com/jenkinsci/junit-plugin/pull/89 , the lists of failed tests will have the stage info as well. (this is all in the classic UI - Blue Ocean isn't yet updated to take advantage of this information)

Andrew Bayer added a comment - 2017-11-15 15:29

Also, I slept badly last night so may not be fully coherent. My apologies. =)

Andrew Bayer added a comment - 2017-11-15 15:29 Also, I slept badly last night so may not be fully coherent. My apologies. =)

Reinhold Füreder added a comment - 2017-11-15 15:42

OK, now I got it, thanks for your patience; and I can actually see/confirm that ("shown for individual tests")

Just out of interest: are there any plans to pimp the visualization and not merge all test results together, but – frankly I don't know what the best UX would be – have one TestResult tab/entry per run or stage or so?

Reinhold Füreder added a comment - 2017-11-15 15:42 OK, now I got it, thanks for your patience; and I can actually see/confirm that ("shown for individual tests") Just out of interest: are there any plans to pimp the visualization and not merge all test results together, but – frankly I don't know what the best UX would be – have one TestResult tab/entry per run or stage or so?

Andrew Bayer added a comment - 2017-11-15 15:48

Maaaaybe? That's definitely on the roadmap for Blue Ocean, and I'll see if I can figure out something that makes sense for classic UI as well.

Andrew Bayer added a comment - 2017-11-15 15:48 Maaaaybe? That's definitely on the roadmap for Blue Ocean, and I'll see if I can figure out something that makes sense for classic UI as well.

Andrew Bayer added a comment - 2017-11-15 18:13

Releasing 1.22.2 with this fix right now.

Andrew Bayer added a comment - 2017-11-15 18:13 Releasing 1.22.2 with this fix right now.

SCM/JIRA link daemon added a comment - 2017-11-15 18:13

Code changed in jenkins
User: Andrew Bayer
Path:
src/main/java/hudson/tasks/junit/CaseResult.java
src/main/java/hudson/tasks/junit/SuiteResult.java
src/test/java/hudson/tasks/junit/pipeline/JUnitResultsStepTest.java
http://jenkins-ci.org/commit/junit-plugin/9ec93955137b9eeba7715e6f118e558be7dbb712
Log:
[FIXED JENKINS-48020] Show stage in failed tests, fix warnings

First, this changes `CaseResult#getFullDisplayName()` to include
stage/branch names like `CaseResult#getDisplayName()`, so that the
full information will show up in lists of failed tests, for example.

Second, stop using `CaseResult#getDisplayName()` for indexing
`CaseResult`s in `SuiteResult`. That method will spam warnings if
called too early in the run now due to `CaseResult#getRun()` being
called before the `SuiteResult`'s parent is set.

Third, fix `JUnitResultsStepTest#testTrends` to actually test what
it's supposed to, to fail if it doesn't find the expected display
names, to test full display names as well, and to use the same
filename for the test results each time due to one of the stages' test
file not containing an explicit suite name, leading to its suite's
name being set to the filename, breaking `SuiteResult#getPreviousResult()`.

SCM/JIRA link daemon added a comment - 2017-11-15 18:13 Code changed in jenkins User: Andrew Bayer Path: src/main/java/hudson/tasks/junit/CaseResult.java src/main/java/hudson/tasks/junit/SuiteResult.java src/test/java/hudson/tasks/junit/pipeline/JUnitResultsStepTest.java http://jenkins-ci.org/commit/junit-plugin/9ec93955137b9eeba7715e6f118e558be7dbb712 Log: [FIXED JENKINS-48020] Show stage in failed tests, fix warnings First, this changes `CaseResult#getFullDisplayName()` to include stage/branch names like `CaseResult#getDisplayName()`, so that the full information will show up in lists of failed tests, for example. Second, stop using `CaseResult#getDisplayName()` for indexing `CaseResult`s in `SuiteResult`. That method will spam warnings if called too early in the run now due to `CaseResult#getRun()` being called before the `SuiteResult`'s parent is set. Third, fix `JUnitResultsStepTest#testTrends` to actually test what it's supposed to, to fail if it doesn't find the expected display names, to test full display names as well, and to use the same filename for the test results each time due to one of the stages' test file not containing an explicit suite name, leading to its suite's name being set to the filename, breaking `SuiteResult#getPreviousResult()`.

Reinhold Füreder added a comment - 2017-11-16 07:33

Thanks a lot for the quick fixes => looks good to me (at a first quick glance)

Reinhold Füreder added a comment - 2017-11-16 07:33 Thanks a lot for the quick fixes => looks good to me (at a first quick glance)

Assignee:: Andrew Bayer

Reporter:: Reinhold Füreder

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Created:: 2017-11-15 10:27

Updated:: 2020-04-02 15:59

Resolved:: 2017-11-15 18:13

Jenkins

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Andrew Bayer added a comment - 2017-11-15 14:30

Expand comment: Andrew Bayer added a comment - 2017-11-15 14:30

Collapse comment: Andrew Bayer added a comment - 2017-11-15 14:49

Expand comment: Andrew Bayer added a comment - 2017-11-15 14:49

Collapse comment: Andrew Bayer added a comment - 2017-11-15 15:04

Expand comment: Andrew Bayer added a comment - 2017-11-15 15:04

Collapse comment: Reinhold Füreder added a comment - 2017-11-15 15:07

Expand comment: Reinhold Füreder added a comment - 2017-11-15 15:07

Collapse comment: Andrew Bayer added a comment - 2017-11-15 15:12

Expand comment: Andrew Bayer added a comment - 2017-11-15 15:12

Collapse comment: Reinhold Füreder added a comment - 2017-11-15 15:15, Edited by Reinhold Füreder - 2017-11-15 15:15

Expand comment: Reinhold Füreder added a comment - 2017-11-15 15:15, Edited by Reinhold Füreder - 2017-11-15 15:15

Collapse comment: Andrew Bayer added a comment - 2017-11-15 15:28

Expand comment: Andrew Bayer added a comment - 2017-11-15 15:28

Collapse comment: Andrew Bayer added a comment - 2017-11-15 15:29

Expand comment: Andrew Bayer added a comment - 2017-11-15 15:29

Collapse comment: Reinhold Füreder added a comment - 2017-11-15 15:42

Expand comment: Reinhold Füreder added a comment - 2017-11-15 15:42

Collapse comment: Andrew Bayer added a comment - 2017-11-15 15:48

Expand comment: Andrew Bayer added a comment - 2017-11-15 15:48

Collapse comment: Andrew Bayer added a comment - 2017-11-15 18:13

Expand comment: Andrew Bayer added a comment - 2017-11-15 18:13

Collapse comment: SCM/JIRA link daemon added a comment - 2017-11-15 18:13

Expand comment: SCM/JIRA link daemon added a comment - 2017-11-15 18:13

Collapse comment: Reinhold Füreder added a comment - 2017-11-16 07:33

Expand comment: Reinhold Füreder added a comment - 2017-11-16 07:33

People

Dates