[JENKINS-56510] Provide generic issue parser for custom tools - Jenkins Jira

Derrick Gibelyou created issue - 2019-03-11 21:47

Ulli Hafner added a comment - 2019-03-11 23:31 - edited

This is a good idea! If the parser would be part of warnings-ng (and not analysis-model) then Jenkins XML reading logic (XStream) could be used, see AnalysisResult.java.

Ulli Hafner added a comment - 2019-03-11 23:31 - edited This is a good idea! If the parser would be part of warnings-ng (and not analysis-model) then Jenkins XML reading logic (XStream) could be used, see AnalysisResult.java .

Derrick Gibelyou added a comment - 2019-03-17 03:56

That is an interesting idea.

Could that cause problems since it would be de-serializing XML generated by a different tool? I don't have any experience with XStream. Does it handle missing fields well? I wouldn't expect the external tools to populate all the possible fields of an issue.

Derrick Gibelyou added a comment - 2019-03-17 03:56 That is an interesting idea. Could that cause problems since it would be de-serializing XML generated by a different tool? I don't have any experience with XStream. Does it handle missing fields well? I wouldn't expect the external tools to populate all the possible fields of an issue.

Ulli Hafner added a comment - 2019-03-17 10:08

Yes, it handles missing fields quite well. And the serialization fallbacks are under our control, see https://github.com/jenkinsci/warnings-ng-plugin/blob/master/src/main/java/io/jenkins/plugins/analysis/core/model/IssueStream.java.

Ulli Hafner added a comment - 2019-03-17 10:08 Yes, it handles missing fields quite well. And the serialization fallbacks are under our control, see https://github.com/jenkinsci/warnings-ng-plugin/blob/master/src/main/java/io/jenkins/plugins/analysis/core/model/IssueStream.java .

Ulli Hafner made changes - 2019-03-22 14:29

Field	Original Value	New Value
Rank		Ranked higher

Ulli Hafner made changes - 2019-03-22 14:30

Assignee	Ulli Hafner [ drulli ]
Labels		help-wanted newbie-friendly

Ulli Hafner made changes - 2019-03-22 14:31

Epic Link

~~JENKINS-56456~~ [ 198054 ]

Ulli Hafner made changes - 2019-03-28 09:38

Priority

Minor [ 4 ]

Major [ 3 ]

Raphael Furch made changes - 2019-03-28 18:04

Assignee

Raphael Furch [ hobbyte ]

Raphael Furch added a comment - 2019-04-03 15:11

drulli I don't know exactly what to do here. Can you give me a hint please?

Raphael Furch added a comment - 2019-04-03 15:11 drulli I don't know exactly what to do here. Can you give me a hint please?

Ulli Hafner added a comment - 2019-04-03 15:52 - edited

We need a new tool class e.g. AnalysisReport (subclass of ReportScanningTool) in the warnings-ng project, see Detekt for an example. This class creates a new parser in the createParser method.
We need a new parser class e.g. ReportParser (extending IssueParser). The parse method of this class should use the same code as AnalysisResult.readIssues(). The code in AnalysisResult already reads a report with several issues, so we basically need to reuse or extract that part.

Ulli Hafner added a comment - 2019-04-03 15:52 - edited We need a new tool class e.g. AnalysisReport (subclass of ReportScanningTool ) in the warnings-ng project, see Detekt for an example. This class creates a new parser in the createParser method. We need a new parser class e.g. ReportParser (extending IssueParser ). The parse method of this class should use the same code as AnalysisResult.readIssues() . The code in AnalysisResult already reads a report with several issues, so we basically need to reuse or extract that part.

Jeremie Bresson added a comment - 2019-04-18 06:28

I have similar needs, I have made a proposal for a JSON based format in ~~JENKINS-57098~~.

Jeremie Bresson added a comment - 2019-04-18 06:28 I have similar needs, I have made a proposal for a JSON based format in JENKINS-57098 .

Jeremie Bresson made changes - 2019-04-18 08:16

Link

This issue is related to ~~JENKINS-57098~~ [ ~~JENKINS-57098~~ ]

Ulli Hafner made changes - 2019-04-18 22:29

Status

Open [ 1 ]

In Progress [ 3 ]

Ulli Hafner made changes - 2019-04-18 22:29

Rank

Ranked lower

Ulli Hafner made changes - 2019-04-22 16:40

Description

I have a handful of custom tools to generate custom reports on my code base and/or the compiled results. Right now I am abusing the output format of PMD to get close to the output I want. However, it would be nice if I could generate a file that mapped directly to the fields used by the plugin.

This might also be useful for people who do not have admin access to write groovy parsers. They could do some local analysis, then generate this file format.

An example of the input file format:
{code:xml}
<issues>
    <issue>
        <category>Category 1</category>
        <type>Type 1</type>
        <severity>WARNING_LOW</severity>
        <message>Message 1</message>
        <lineStart>0</lineStart>
        <lineEnd>0</lineEnd>
        <columnStart>0</columnStart>
        <columnEnd>0</columnEnd>
        <lineRanges/>
        <description>Description 1</description>
        <moduleName>Module 1</moduleName>
        <packageName>-</packageName>
        <fileName>-</fileName>
    </issue>
    <issue>
        <category>Category 2</category>
        <type>Type 2</type>
        <severity>WARNING_HIGH</severity>
        <message>Message 2</message>
        <lineStart>0</lineStart>
        <lineEnd>0</lineEnd>
        <columnStart>0</columnStart>
        <columnEnd>0</columnEnd>
        <lineRanges/>
        <description>Description 2</description>
        <moduleName>Module 2</moduleName>
        <packageName>-</packageName>
        <fileName>-</fileName>
    </issue>
</issues>
</code>
{code}
If this is of general interest I can post a PR.

I have a handful of custom tools to generate custom reports on my code base and/or the compiled results. Right now I am abusing the output format of PMD to get close to the output I want. However, it would be nice if I could generate a file that mapped directly to the fields used by the plugin.

This might also be useful for people who do not have admin access to write groovy parsers. They could do some local analysis, then generate this file format.

Current Serialization of an {{Issue}} instance using XStream:
{code:xml}
<?xml version="1.1" encoding="UTF-8"?>
<issue>
  <category>category</category>
  <type>type</type>
  <severity>HIGH</severity>
  <message>message</message>
  <lineStart>1</lineStart>
  <lineEnd>2</lineEnd>
  <columnStart>3</columnStart>
  <columnEnd>4</columnEnd>
  <lineRanges>
    <lineRange>
      <start>5</start>
      <end>6</end>
    </lineRange>
  </lineRanges>
  <id>63d61b1f-0cac-4e31-8bb9-f390ed1acfe8</id>
  <description>description</description>
  <reference>reference</reference>
  <origin>origin</origin>
  <moduleName>module-name</moduleName>
  <packageName>package-name</packageName>
  <fileName>file-name</fileName>
  <fingerprint>fingerprint</fingerprint>
</issue>
{code}

Current Serialization of a {{Report}} instance using XStream:

{code}
<?xml version="1.1" encoding="UTF-8"?>
<analysisReport>
  <elements class="linked-hash-set">
    <issue>
      <category>category</category>
      <type>type</type>
      <severity>HIGH</severity>
      <message>1</message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>c3b984a3-7f67-4332-953b-27d3943f232c</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      <moduleName>module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
    <issue>
      <category>category</category>
      <type>type</type>
      <severity reference="../../issue/severity"/>
      <message>2</message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>fbf2fee0-292f-4991-bd06-d8c5b13ace93</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      <moduleName>module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
  </elements>
  <infoMessages>
    <string>info</string>
  </infoMessages>
  <errorMessages>
    <string>error</string>
  </errorMessages>
  <duplicatesSize>0</duplicatesSize>
</analysisReport>
{code}

Ulli Hafner made changes - 2019-04-22 16:41

Description

I have a handful of custom tools to generate custom reports on my code base and/or the compiled results. Right now I am abusing the output format of PMD to get close to the output I want. However, it would be nice if I could generate a file that mapped directly to the fields used by the plugin.

This might also be useful for people who do not have admin access to write groovy parsers. They could do some local analysis, then generate this file format.

Current Serialization of an {{Issue}} instance using XStream:
{code:xml}
<?xml version="1.1" encoding="UTF-8"?>
<issue>
  <category>category</category>
  <type>type</type>
  <severity>HIGH</severity>
  <message>message</message>
  <lineStart>1</lineStart>
  <lineEnd>2</lineEnd>
  <columnStart>3</columnStart>
  <columnEnd>4</columnEnd>
  <lineRanges>
    <lineRange>
      <start>5</start>
      <end>6</end>
    </lineRange>
  </lineRanges>
  <id>63d61b1f-0cac-4e31-8bb9-f390ed1acfe8</id>
  <description>description</description>
  <reference>reference</reference>
  <origin>origin</origin>
  <moduleName>module-name</moduleName>
  <packageName>package-name</packageName>
  <fileName>file-name</fileName>
  <fingerprint>fingerprint</fingerprint>
</issue>
{code}

Current Serialization of a {{Report}} instance using XStream:

{code}
<?xml version="1.1" encoding="UTF-8"?>
<analysisReport>
  <elements class="linked-hash-set">
    <issue>
      <category>category</category>
      <type>type</type>
      <severity>HIGH</severity>
      <message>1</message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>c3b984a3-7f67-4332-953b-27d3943f232c</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      <moduleName>module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
    <issue>
      <category>category</category>
      <type>type</type>
      <severity reference="../../issue/severity"/>
      <message>2</message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>fbf2fee0-292f-4991-bd06-d8c5b13ace93</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      <moduleName>module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
  </elements>
  <infoMessages>
    <string>info</string>
  </infoMessages>
  <errorMessages>
    <string>error</string>
  </errorMessages>
  <duplicatesSize>0</duplicatesSize>
</analysisReport>
{code}

I have a handful of custom tools to generate custom reports on my code base and/or the compiled results. Right now I am abusing the output format of PMD to get close to the output I want. However, it would be nice if I could generate a file that mapped directly to the fields used by the plugin.

This might also be useful for people who do not have admin access to write groovy parsers. They could do some local analysis, then generate this file format.

Current Serialization of an {{Issue}} instance using XStream:
{code:xml}
<?xml version="1.1" encoding="UTF-8"?>
<issue>
  <category>category</category>
  <type>type</type>
  <severity>HIGH</severity>
  <message>message</message>
  <lineStart>1</lineStart>
  <lineEnd>2</lineEnd>
  <columnStart>3</columnStart>
  <columnEnd>4</columnEnd>
  <lineRanges>
    <lineRange>
      <start>5</start>
      <end>6</end>
    </lineRange>
  </lineRanges>
  <id>63d61b1f-0cac-4e31-8bb9-f390ed1acfe8</id>
  <description>description</description>
  <reference>reference</reference>
  <origin>origin</origin>
  <moduleName>module-name</moduleName>
  <packageName>package-name</packageName>
  <fileName>file-name</fileName>
  <fingerprint>fingerprint</fingerprint>
</issue>
{code}

Current Serialization of a {{Report}} instance using XStream:

{code:xml}
<?xml version="1.1" encoding="UTF-8"?>
<analysisReport>
  <elements class="linked-hash-set">
    <issue>
      <category>category</category>
      <type>type</type>
      <severity>HIGH</severity>
      <message>1</message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>c3b984a3-7f67-4332-953b-27d3943f232c</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      <moduleName>module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
    <issue>
      <category>category</category>
      <type>type</type>
      <severity reference="../../issue/severity"/>
      <message>2</message>
      <lineStart>1</lineStart>
      <lineEnd>2</lineEnd>
      <columnStart>3</columnStart>
      <columnEnd>4</columnEnd>
      <lineRanges>
        <lineRange>
          <start>5</start>
          <end>6</end>
        </lineRange>
      </lineRanges>
      <id>fbf2fee0-292f-4991-bd06-d8c5b13ace93</id>
      <description>description</description>
      <reference>reference</reference>
      <origin>origin</origin>
      <moduleName>module-name</moduleName>
      <packageName>package-name</packageName>
      <fileName>file-name</fileName>
      <fingerprint>fingerprint</fingerprint>
    </issue>
  </elements>
  <infoMessages>
    <string>info</string>
  </infoMessages>
  <errorMessages>
    <string>error</string>
  </errorMessages>
  <duplicatesSize>0</duplicatesSize>
</analysisReport>
{code}

Ulli Hafner made changes - 2019-04-26 12:11

Resolution		Fixed [ 1 ]
Status	In Progress [ 3 ]	Fixed but Unreleased [ 10203 ]

Ulli Hafner made changes - 2019-05-08 22:19

Released As		5.0.0 (analysis-model and warnings-ng)
Status	Fixed but Unreleased [ 10203 ]	Resolved [ 5 ]

Jeremie Bresson made changes - 2019-06-05 06:13

Link

This issue is related to ~~JENKINS-57854~~ [ ~~JENKINS-57854~~ ]

Jenkins

Provide generic issue parser for custom tools

Details

Description

Attachments

Issue Links

Activity

People

Dates