Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-65482

Problem with matrix jobs: No removed found with id...

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • analysis-model
    • None
    • analysis-model 10.0.0
      matrix-project-plugin 1.18

      We have a matrix job with a JDK axis and two sub-jobs (JDK8 and JDK11).

      Since upgrading to analysis-model 10.0.0 it seems that "issues" are mixed up between the sub jobs and the parent (matrix-) job.

      E.g. we have four green builds without any code changes: #643, #644, #645, #646
      I think #643 was done with an older version of Jenkins and the analysis-model plugin.

      #643 has 42 spotbugs warnings
      #644 has twice as many (84) spotbugs warnings, 42 new
      #645 has 84 warnings, no new
      #646 has 84 warnings, 42 new

      Looking at the individual child jobs, they both have 42 spotbugs warnings, so it looks like the parent job somehow aggregates the results, but doesn't fully recognize them, so they are marked as new.

      The result of this mixup is that the matrix (parent) job repeatedly fails since build #647 at the end, because it cannot find an issue:

      FATAL: No removed found with id 0758f19a-939f-43ea-813d-9de068897527.
edu.hm.hafner.util.NoSuchElementException: No removed found with id 0758f19a-939f-43ea-813d-9de068897527.
	at edu.hm.hafner.analysis.Report.remove(Report.java:366)
	at edu.hm.hafner.analysis.IssueDifference.remove(IssueDifference.java:92)
	at edu.hm.hafner.analysis.IssueDifference.matchIssuesByEquals(IssueDifference.java:62)
	at edu.hm.hafner.analysis.IssueDifference.<init>(IssueDifference.java:48)
	at io.jenkins.plugins.analysis.core.model.DeltaReport.<init>(DeltaReport.java:49)
	at io.jenkins.plugins.analysis.core.steps.IssuesPublisher.attachAction(IssuesPublisher.java:100)
	at io.jenkins.plugins.analysis.core.steps.IssuesRecorder.publishResult(IssuesRecorder.java:807)
	at io.jenkins.plugins.analysis.core.steps.IssuesAggregator.lambda$endBuild$52c4a486$1(IssuesAggregator.java:91)
	at org.eclipse.collections.impl.map.mutable.UnifiedMap.forEachKeyValue(UnifiedMap.java:998)
	at org.eclipse.collections.impl.multimap.AbstractMultimap.forEachKeyMultiValues(AbstractMultimap.java:188)
	at io.jenkins.plugins.analysis.core.steps.IssuesAggregator.endBuild(IssuesAggregator.java:89)
	at hudson.matrix.MatrixBuild$MatrixBuildExecution.post2(MatrixBuild.java:433)
	at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:698)
	at hudson.model.Run.execute(Run.java:1932)
	at hudson.matrix.MatrixBuild.run(MatrixBuild.java:323)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)

      Looking at the build output in the filesystem, I find the issue here in #646:

      grep -r 0758f19a-939f-43ea-813d-9de068897527
builds/646/spotbugs-outstanding-issues.xml:      <id>0758f19a-939f-43ea-813d-9de068897527</id>
configurations/axis-jdk/JDK11/builds/646/spotbugs-outstanding-issues.xml:      <id>0758f19a-939f-43ea-813d-9de068897527</id>

          [JENKINS-65482] Problem with matrix jobs: No removed found with id...

          Ulli Hafner added a comment -

          This seems to be related to JENKINS-65382. Somehow the mixing of different results does not work.

          Ulli Hafner added a comment - This seems to be related to JENKINS-65382 . Somehow the mixing of different results does not work.

          Carsten Pfeiffer added a comment -

          Thanks for the quick reply!
          Looking at builds/645/spotbugs-outstanding-issues.xml and builds/646/spotbugs-new-issues.xml, and checking a "new issue" in #646 I notice that

          • both issues have the same fingerprint
          • but a different id
            I guess that they do not match, because one of them stems from the JDK8 child job and the other from the JDK11 child job.

          So (wild guess), maybe the issues are aggregated from both child jobs and somewhere hashed by the fingerprint, and it is indeterministic from which child job the issue is then taken. When they alternate between builds, the issue is marked as new.

          Carsten Pfeiffer added a comment - Thanks for the quick reply! Looking at builds/645/spotbugs-outstanding-issues.xml and builds/646/spotbugs-new-issues.xml, and checking a "new issue" in #646 I notice that both issues have the same fingerprint but a different id I guess that they do not match, because one of them stems from the JDK8 child job and the other from the JDK11 child job. So (wild guess), maybe the issues are aggregated from both child jobs and somewhere hashed by the fingerprint, and it is indeterministic from which child job the issue is then taken. When they alternate between builds, the issue is marked as new.

          Carsten Pfeiffer added a comment -

          To clarify, the difference comes from the pathName element:

          <pathName>/var/jenkins_home/workspace/foo/validation/bar-validation/jdk/JDK8</pathName>

          vs.

          <pathName>/var/jenkins_home/workspace/foo/validation/bar-validation/jdk/JDK11</pathName>

          Carsten Pfeiffer added a comment - To clarify, the difference comes from the pathName element: <pathName>/ var /jenkins_home/workspace/foo/validation/bar-validation/jdk/JDK8</pathName> vs. <pathName>/ var /jenkins_home/workspace/foo/validation/bar-validation/jdk/JDK11</pathName>

          Ulli Hafner added a comment -

          Thanks for your help! id and pathName are actually not part of equals, because they change from build to build (see https://github.com/jenkinsci/analysis-model/blob/master/src/main/java/edu/hm/hafner/analysis/Issue.java#L851).
          I think the underlying problem is that in 10.x the results are stored in two sub results while in 9.x everything was part of one collection. I need to prepare a test case that shows the problem...

          Ulli Hafner added a comment - Thanks for your help! id and pathName are actually not part of equals , because they change from build to build (see https://github.com/jenkinsci/analysis-model/blob/master/src/main/java/edu/hm/hafner/analysis/Issue.java#L851 ). I think the underlying problem is that in 10.x the results are stored in two sub results while in 9.x everything was part of one collection. I need to prepare a test case that shows the problem...

          Carsten Pfeiffer added a comment -

          Alright, thanks – let me know if you need any more information.

          Carsten Pfeiffer added a comment - Alright, thanks – let me know if you need any more information.

          Ulli Hafner added a comment -

          Ok, I think I found the root cause for the problem: the new warning detection currently does not work if there are issues in a report of reports that are the same (with respect to equals). In version 9.x those issues have been counted as duplicate and were not added so this was no problem before. But now they are added if they are part of a different sub report.

          There are two ways to fix it: restore the old behavior and remove all duplicate issues, even if they are part of different sub-reports. Or I can try to make the delta computation aware of hierarchical reports.
          I wonder what you would like to see in your use case? When you are scanning n files that all contain the same warning, should there be just one warning or n of them?

          Ulli Hafner added a comment - Ok, I think I found the root cause for the problem: the new warning detection currently does not work if there are issues in a report of reports that are the same (with respect to equals ). In version 9.x those issues have been counted as duplicate and were not added so this was no problem before. But now they are added if they are part of a different sub report. There are two ways to fix it: restore the old behavior and remove all duplicate issues, even if they are part of different sub-reports. Or I can try to make the delta computation aware of hierarchical reports. I wonder what you would like to see in your use case? When you are scanning n files that all contain the same warning, should there be just one warning or n of them?

          Carsten Pfeiffer added a comment -

          I for one would prefer to skip duplicates. In the matrix job case, I could dive into the report of the child job to see the individual issues. In the aggregate report, I would rather not have n times the same issues, but rather the count of distinct issues.

          Carsten Pfeiffer added a comment - I for one would prefer to skip duplicates. In the matrix job case, I could dive into the report of the child job to see the individual issues. In the aggregate report, I would rather not have n times the same issues, but rather the count of distinct issues.

          Carsten Pfeiffer added a comment -

          Thank you so much for fixing it so quickly!

          Carsten Pfeiffer added a comment - Thank you so much for fixing it so quickly!

            drulli Ulli Hafner
            capf Carsten Pfeiffer
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: