[JENKINS-55436] Duplicate symbols break declarative pipeline engine

Type: Bug
Resolution: Fixed
Priority: Blocker
Component/s: warnings-ng-plugin
Labels:
None
Environment:
JENKINS 2.156
Arch Linux

Relevant installed plugins:

warnings-ng 1.0.1
pmd 4.0.0
android-lint 2.6
checkstyle 4.0.0
workflow-cps 2.61.1

Similar Issues:
Powered by SuggestiMate

Show
Released As:
3.0.1

If two plugins define symbols with the same name then the declarative pipeline engine can't invoke a job anymore.

Example: warnings-ng plugin and androidLint define the same symbol androidLint (one is for a step in the androidLint plugin, one is for the selected tool in warnings-ng).

This is a much bigger problem as in ~~JENKINS-55328~~ for scripted pipelines. Here one simply can use the additional symbol (androidLintParser) to make the old and new plugin happy. This does not work for declarative pipelines.

Example (usage of OLD plugins):

pmd         canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml",        unHealthy: '', unstableTotalAll: '0'
checkstyle  canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/checkstyle.xml", unHealthy: '', unstableTotalAll: '0'
androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml",      unHealthy: '', unstableTotalAll: '0'

If the new warnings-ng plugin is installed (but not used) I get the following error:

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 67: Invalid parameter "canComputeNew", did you mean "name"? @ line 67, column 33.
                       pmd         canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml",        unHealthy: '', unstableTotalAll: '0'
                                   ^

WorkflowScript: 67: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 67, column 55.
           canComputeNew: false, canRunOnFa
                                 ^

WorkflowScript: 67: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 67, column 77.
    false, canRunOnFailed: true, defaultEnc
                                 ^

WorkflowScript: 67: Invalid parameter "healthy", did you mean "name"? @ line 67, column 98.
   d: true, defaultEncoding: '', healthy: '
                                 ^

WorkflowScript: 67: Invalid parameter "unHealthy", did you mean "name"? @ line 67, column 147.
   n: "$reports/pmd.xml",        unHealthy:
                                 ^

WorkflowScript: 67: Invalid parameter "unstableTotalAll", did you mean "name"? @ line 67, column 162.
   d.xml",        unHealthy: '', unstableTo
                                 ^

WorkflowScript: 69: Invalid parameter "canComputeNew", did you mean "name"? @ line 69, column 33.
                       androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml",      unHealthy: '', unstableTotalAll: '0'
                                   ^

WorkflowScript: 69: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 69, column 55.
   oidLint canComputeNew: false, canRunOnFa
                                 ^

WorkflowScript: 69: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 69, column 77.
    false, canRunOnFailed: true, defaultEnc
                                 ^

WorkflowScript: 69: Invalid parameter "healthy", did you mean "name"? @ line 69, column 98.
   d: true, defaultEncoding: '', healthy: '
                                 ^

10 errors

	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

I do not have these issues if I uninstall the warnings-ng plugin.

relates to

JENKINS-55328 recordIssues and ScanForIssues in Pipeline get NPE Errors

Resolved

JENKINS-55349 Duplicate symbols in different plugins cause NPEs

Open

Matthias Fuchs created issue - 2019-01-07 18:43

Ulli Hafner added a comment - 2019-01-07 21:29

warnings-ng is not backwards compatible due to different parameters it accepts

Yes, this is a design design to provide a clean API. I don't think that it is feasible to provide a general one-to-one mapping of parameters.

having the Android Lint plugin, the Checkstyle plugin, and the PMD plugin installed alongside warnings-ng breaks builds

Why is that? This should be no problem at all. Can you please elaborate and create a different issue?

This means that I have to transition all jobs to warnings-ng which is simply not feasible, given that there are multiple branches that are build and given our policies. Especially I would have to perform the transition at once to avoid failing builds.

You should use a shared library for your analysis checks. Then you need to change the code in one place only.

Ulli Hafner added a comment - 2019-01-07 21:29 warnings-ng is not backwards compatible due to different parameters it accepts Yes, this is a design design to provide a clean API. I don't think that it is feasible to provide a general one-to-one mapping of parameters. having the Android Lint plugin, the Checkstyle plugin, and the PMD plugin installed alongside warnings-ng breaks builds Why is that? This should be no problem at all. Can you please elaborate and create a different issue? This means that I have to transition all jobs to warnings-ng which is simply not feasible, given that there are multiple branches that are build and given our policies. Especially I would have to perform the transition at once to avoid failing builds. You should use a shared library for your analysis checks. Then you need to change the code in one place only.

Matthias Fuchs made changes - 2019-01-08 16:18

Environment

Original: JENKINS 2.156
Arch Linux

New: JENKINS 2.156
Arch Linux

Relevant installed plugins:

warnings-ng
pmd
android-lint
checkstyle

Matthias Fuchs made changes - 2019-01-08 16:21

Description

Original: Many of the jobs I use have collect static analysis the following way:
{code}
pmd canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml", unHealthy: '', unstableTotalAll: '0'
checkstyle canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/checkstyle.xml", unHealthy: '', unstableTotalAll: '0'
androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml", unHealthy: '', unstableTotalAll: '0'
{code}

Now I want to migrate them to warnings-ng with the following code:
{code}
recordIssues(tools: [androidLint(pattern: "$reports/lint*.xml"),
                     checkStyle(pattern: "$reports/checkstyle.xml"),
                     pmdParser(pattern: "$reports/pmd.xml")],
             aggregatingResults: true, enabledForFailure: true)
{code}

I face multiple issues though:
# warnings-ng is not backwards compatible due to different parameters it accepts
# having the Android Lint plugin, the Checkstyle plugin, and the PMD plugin installed alongside warnings-ng breaks builds
# This means that I have to transition *all* jobs to warnings-ng which is simply not feasible, given that there are multiple branches that are build and given our policies. Especially I would have to perform the transition at once to avoid failing builds.

I think the plugin should be backwards compatible by accepting the additional parameters I listed above.

New: Many of the jobs I use have collect static analysis the following way (OLD):
{code}
pmd canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml", unHealthy: '', unstableTotalAll: '0'
checkstyle canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/checkstyle.xml", unHealthy: '', unstableTotalAll: '0'
androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml", unHealthy: '', unstableTotalAll: '0'
{code}

Now I want to migrate them to warnings-ng with the following code (NEW):
{code}
recordIssues(tools: [androidLint(pattern: "$reports/lint*.xml"),
                     checkStyle(pattern: "$reports/checkstyle.xml"),
                     pmdParser(pattern: "$reports/pmd.xml")],
             aggregatingResults: true, enabledForFailure: true)
{code}

I face multiple issues though:
# warnings-ng is not backwards compatible due to different parameters it accepts
# having the Android Lint plugin, the Checkstyle plugin, and the PMD plugin installed alongside warnings-ng breaks builds
# This means that I have to transition *all* jobs to warnings-ng which is simply not feasible, given that there are multiple branches that are build and given our policies. Especially I would have to perform the transition at once to avoid failing builds.

I think the plugin should be backwards compatible by accepting the additional parameters I listed above.

Now when I build a job that uses OLD I get the following error:
{noformat}
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
WorkflowScript: 67: Invalid parameter "canComputeNew", did you mean "name"? @ line 67, column 33.
                       pmd canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml", unHealthy: '', unstableTotalAll: '0'
                                   ^

WorkflowScript: 67: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 67, column 55.
           canComputeNew: false, canRunOnFa
                                 ^

WorkflowScript: 67: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 67, column 77.
    false, canRunOnFailed: true, defaultEnc
                                 ^

WorkflowScript: 67: Invalid parameter "healthy", did you mean "name"? @ line 67, column 98.
   d: true, defaultEncoding: '', healthy: '
                                 ^

WorkflowScript: 67: Invalid parameter "unHealthy", did you mean "name"? @ line 67, column 147.
   n: "$reports/pmd.xml", unHealthy:
                                 ^

WorkflowScript: 67: Invalid parameter "unstableTotalAll", did you mean "name"? @ line 67, column 162.
   d.xml", unHealthy: '', unstableTo
                                 ^

WorkflowScript: 69: Invalid parameter "canComputeNew", did you mean "name"? @ line 69, column 33.
                       androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml", unHealthy: '', unstableTotalAll: '0'
                                   ^

WorkflowScript: 69: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 69, column 55.
   oidLint canComputeNew: false, canRunOnFa
                                 ^

WorkflowScript: 69: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 69, column 77.
    false, canRunOnFailed: true, defaultEnc
                                 ^

WorkflowScript: 69: Invalid parameter "healthy", did you mean "name"? @ line 69, column 98.
   d: true, defaultEncoding: '', healthy: '
                                 ^

10 errors

at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
{noformat}

I do not have these issues if I *uninstall* the warnings-ng plugin.
Therefore I think that the warnings-ng plugin is incompatible with *android-lint* and *pmd*. I do not have these issues with the checkstyle plugin.

Matthias Fuchs added a comment - 2019-01-08 16:23

I updated the story.
Of course I have no issues with the design decision as long as I can have both the old and new plugins installed and working side-by-side.
Which unfortunately does not seem to be the case, see my updates.

Matthias Fuchs added a comment - 2019-01-08 16:23 I updated the story. Of course I have no issues with the design decision as long as I can have both the old and new plugins installed and working side-by-side. Which unfortunately does not seem to be the case, see my updates.

Ulli Hafner added a comment - 2019-01-08 22:53

Interesting, on our CI we have both plugins working in parallel without problems:

Which version of the workflow cps plugin do you have installed?

Ulli Hafner added a comment - 2019-01-08 22:53 Interesting, on our CI we have both plugins working in parallel without problems: https://ci.jenkins.io/job/Plugins/job/warnings-ng-plugin/ https://ci.jenkins.io/job/Plugins/job/analysis-model/job/master/ Which version of the workflow cps plugin do you have installed?

Matthias Fuchs made changes - 2019-01-09 10:38

Environment

Original: JENKINS 2.156
Arch Linux

Relevant installed plugins:

warnings-ng
pmd
android-lint
checkstyle

New: JENKINS 2.156
Arch Linux

Relevant installed plugins:

warnings-ng 1.0.1
pmd 4.0.0
android-lint 2.6
checkstyle 4.0.0
workflow-cps 2.61.1

Matthias Fuchs added a comment - 2019-01-09 10:44 - edited

I updated the environment.

If this is the Jenkinsfile you use then I know why it works for you.
You use fewer parameters in the calls to pmd etc.

If I also use fewer parameters then it works as well. I just tried that.
So the backward incompatibility is for some of the parameters, as also indicated by the parsing error.

Matthias Fuchs added a comment - 2019-01-09 10:44 - edited I updated the environment. If this is the Jenkinsfile you use then I know why it works for you. You use fewer parameters in the calls to pmd etc. If I also use fewer parameters then it works as well. I just tried that. So the backward incompatibility is for some of the parameters, as also indicated by the parsing error.

Matthias Fuchs added a comment - 2019-01-09 19:05

Maybe the reason that checkstyle is working but not the others is the different names.
In the checkstyle plugin the exposed symbol in the pipeline is check*s*tyle while warning-ng exposes check*S*tyle. Thus there is no clash.
At the same time the exposed symbol for android lint is androidLint in both the android-lint plugin and the new plugin, respectively for pmd.

Matthias Fuchs added a comment - 2019-01-09 19:05 Maybe the reason that checkstyle is working but not the others is the different names. In the checkstyle plugin the exposed symbol in the pipeline is check*s*tyle while warning-ng exposes check*S*tyle. Thus there is no clash. At the same time the exposed symbol for android lint is androidLint in both the android-lint plugin and the new plugin, respectively for pmd.

Ulli Hafner added a comment - 2019-01-09 22:09

I tried the following on my local machine:

node {
    stage ('Checkout') {
        git branch:'master', url: 'file:///Users/hafner/Development/git/analysis-model'
    }

    stage ('Build and Analysis') {
        def mvnHome = tool 'mvn-default'

        sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean verify revapi:check -Dsurefire.useFile=false -Dmaven.test.failure.ignore"

        junit testResults: '**/target/surefire-reports/TEST-*.xml'

        pmd canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "target/pmd.xml",        unHealthy: '', unstableTotalAll: '0'
    }
}

This works quite well with the PMD plugin 4.0.0 and warnings-ng plugin. There still seems to be something different in your configuration...

[PMD] Collecting PMD analysis files...
[PMD] Searching for all files in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model that match the pattern target/pmd.xml
[PMD] Parsing 1 file in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model
[PMD] Successfully parsed file /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model/target/pmd.xml with 2 unique warnings and 0 duplicates.
Skipping warnings blame since pipelines do not have an SCM link.%n
[PMD] Ignore new warnings since this is the first valid build
[PMD] Plug-in Result: Unstable - <a href="pmdResult">2 warnings</a> exceed the threshold of 0 by 2

Ulli Hafner added a comment - 2019-01-09 22:09 I tried the following on my local machine: node { stage ( 'Checkout' ) { git branch: 'master' , url: 'file: ///Users/hafner/Development/git/analysis-model' } stage ( 'Build and Analysis' ) { def mvnHome = tool 'mvn- default ' sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean verify revapi:check -Dsurefire.useFile= false -Dmaven.test.failure.ignore" junit testResults: '**/target/surefire-reports/TEST-*.xml' pmd canComputeNew: false , canRunOnFailed: true , defaultEncoding: '', healthy: ' ', pattern: "target/pmd.xml" , unHealthy: ' ', unstableTotalAll: ' 0' } } This works quite well with the PMD plugin 4.0.0 and warnings-ng plugin. There still seems to be something different in your configuration... [PMD] Collecting PMD analysis files... [PMD] Searching for all files in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model that match the pattern target/pmd.xml [PMD] Parsing 1 file in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model [PMD] Successfully parsed file /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model/target/pmd.xml with 2 unique warnings and 0 duplicates. Skipping warnings blame since pipelines do not have an SCM link.%n [PMD] Ignore new warnings since this is the first valid build [PMD] Plug-in Result: Unstable - <a href="pmdResult">2 warnings</a> exceed the threshold of 0 by 2

Assignee:: Ulli Hafner

Reporter:: Matthias Fuchs

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Created:: 2019-01-07 18:43

Updated:: 2019-03-03 12:29

Resolved:: 2019-02-14 23:19

Jenkins

Details

Description

Attachments

Issue Links

Activity

Collapse comment: Ulli Hafner added a comment - 2019-01-07 21:29

Expand comment: Ulli Hafner added a comment - 2019-01-07 21:29

Collapse comment: Matthias Fuchs added a comment - 2019-01-08 16:23

Expand comment: Matthias Fuchs added a comment - 2019-01-08 16:23

Collapse comment: Ulli Hafner added a comment - 2019-01-08 22:53

Expand comment: Ulli Hafner added a comment - 2019-01-08 22:53

Collapse comment: Matthias Fuchs added a comment - 2019-01-09 10:44, Edited by Matthias Fuchs - 2019-01-09 10:48

Expand comment: Matthias Fuchs added a comment - 2019-01-09 10:44, Edited by Matthias Fuchs - 2019-01-09 10:48

Collapse comment: Matthias Fuchs added a comment - 2019-01-09 19:05

Expand comment: Matthias Fuchs added a comment - 2019-01-09 19:05

Collapse comment: Ulli Hafner added a comment - 2019-01-09 22:09

Expand comment: Ulli Hafner added a comment - 2019-01-09 22:09

People

Dates