Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55436

Duplicate symbols break declarative pipeline engine

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • warnings-ng-plugin
    • None
    • JENKINS 2.156
      Arch Linux

      Relevant installed plugins:

      warnings-ng 1.0.1
      pmd 4.0.0
      android-lint 2.6
      checkstyle 4.0.0
      workflow-cps 2.61.1
    • 3.0.1

      If two plugins define symbols with the same name then the declarative pipeline engine can't invoke a job anymore.

      Example: warnings-ng plugin and androidLint define the same symbol androidLint (one is for a step in the androidLint plugin, one is for the selected tool in warnings-ng). 

      This is a much bigger problem as in JENKINS-55328 for scripted pipelines. Here one simply can use the additional symbol (androidLintParser) to make the old and new plugin happy. This does not work for declarative pipelines.

      Example (usage of OLD plugins):

      pmd         canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml",        unHealthy: '', unstableTotalAll: '0'
      checkstyle  canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/checkstyle.xml", unHealthy: '', unstableTotalAll: '0'
      androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml",      unHealthy: '', unstableTotalAll: '0'
      

      If the new warnings-ng plugin is installed (but not used) I get the following error:

      org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
      WorkflowScript: 67: Invalid parameter "canComputeNew", did you mean "name"? @ line 67, column 33.
                             pmd         canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/pmd.xml",        unHealthy: '', unstableTotalAll: '0'
                                         ^
      
      WorkflowScript: 67: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 67, column 55.
                 canComputeNew: false, canRunOnFa
                                       ^
      
      WorkflowScript: 67: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 67, column 77.
          false, canRunOnFailed: true, defaultEnc
                                       ^
      
      WorkflowScript: 67: Invalid parameter "healthy", did you mean "name"? @ line 67, column 98.
         d: true, defaultEncoding: '', healthy: '
                                       ^
      
      WorkflowScript: 67: Invalid parameter "unHealthy", did you mean "name"? @ line 67, column 147.
         n: "$reports/pmd.xml",        unHealthy:
                                       ^
      
      WorkflowScript: 67: Invalid parameter "unstableTotalAll", did you mean "name"? @ line 67, column 162.
         d.xml",        unHealthy: '', unstableTo
                                       ^
      
      WorkflowScript: 69: Invalid parameter "canComputeNew", did you mean "name"? @ line 69, column 33.
                             androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml",      unHealthy: '', unstableTotalAll: '0'
                                         ^
      
      WorkflowScript: 69: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 69, column 55.
         oidLint canComputeNew: false, canRunOnFa
                                       ^
      
      WorkflowScript: 69: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 69, column 77.
          false, canRunOnFailed: true, defaultEnc
                                       ^
      
      WorkflowScript: 69: Invalid parameter "healthy", did you mean "name"? @ line 69, column 98.
         d: true, defaultEncoding: '', healthy: '
                                       ^
      
      10 errors
      
      	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
      	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
      	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
      	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
      	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
      	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
      	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
      	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
      	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
      	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
      	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
      	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
      	at hudson.model.ResourceController.execute(ResourceController.java:97)
      	at hudson.model.Executor.run(Executor.java:429)
      Finished: FAILURE
      

      I do not have these issues if I uninstall the warnings-ng plugin.

          [JENKINS-55436] Duplicate symbols break declarative pipeline engine

          Ulli Hafner added a comment -

          warnings-ng is not backwards compatible due to different parameters it accepts

          Yes, this is a design design to provide a clean API. I don't think that it is feasible to provide a general one-to-one mapping of parameters.

          having the Android Lint plugin, the Checkstyle plugin, and the PMD plugin installed alongside warnings-ng breaks builds

          Why is that? This should be no problem at all. Can you please elaborate and create a different issue?

          This means that I have to transition all jobs to warnings-ng which is simply not feasible, given that there are multiple branches that are build and given our policies. Especially I would have to perform the transition at once to avoid failing builds.

          You should use a shared library for your analysis checks. Then you need to change the code in one place only.

          Ulli Hafner added a comment - warnings-ng is not backwards compatible due to different parameters it accepts Yes, this is a design design to provide a clean API. I don't think that it is feasible to provide a general one-to-one mapping of parameters. having the Android Lint plugin, the Checkstyle plugin, and the PMD plugin installed alongside warnings-ng breaks builds Why is that? This should be no problem at all. Can you please elaborate and create a different issue? This means that I have to transition all jobs to warnings-ng which is simply not feasible, given that there are multiple branches that are build and given our policies. Especially I would have to perform the transition at once to avoid failing builds. You should use a shared library for your analysis checks. Then you need to change the code in one place only.

          I updated the story.
          Of course I have no issues with the design decision as long as I can have both the old and new plugins installed and working side-by-side.
          Which unfortunately does not seem to be the case, see my updates.

          Matthias Fuchs added a comment - I updated the story. Of course I have no issues with the design decision as long as I can have both the old and new plugins installed and working side-by-side. Which unfortunately does not seem to be the case, see my updates.

          Ulli Hafner added a comment -

          Interesting, on our CI we have both plugins working in parallel without problems:

          1. https://ci.jenkins.io/job/Plugins/job/warnings-ng-plugin/
          2. https://ci.jenkins.io/job/Plugins/job/analysis-model/job/master/

          Which version of the workflow cps plugin do you have installed?

          Ulli Hafner added a comment - Interesting, on our CI we have both plugins working in parallel without problems: https://ci.jenkins.io/job/Plugins/job/warnings-ng-plugin/ https://ci.jenkins.io/job/Plugins/job/analysis-model/job/master/ Which version of the workflow cps plugin do you have installed?

          Matthias Fuchs added a comment - - edited

          I updated the environment.

          If this is the Jenkinsfile you use then I know why it works for you.
          You use fewer parameters in the calls to pmd etc.

          If I also use fewer parameters then it works as well. I just tried that.
          So the backward incompatibility is for some of the parameters, as also indicated by the parsing error.

          Matthias Fuchs added a comment - - edited I updated the environment. If this is the Jenkinsfile you use then I know why it works for you. You use fewer parameters in the calls to pmd etc. If I also use fewer parameters then it works as well. I just tried that. So the backward incompatibility is for some of the parameters, as also indicated by the parsing error.

          Maybe the reason that checkstyle is working but not the others is the different names.
          In the checkstyle plugin the exposed symbol in the pipeline is check*s*tyle while warning-ng exposes check*S*tyle. Thus there is no clash.
          At the same time the exposed symbol for android lint is androidLint in both the android-lint plugin and the new plugin, respectively for pmd.

          Matthias Fuchs added a comment - Maybe the reason that checkstyle is working but not the others is the different names. In the checkstyle plugin the exposed symbol in the pipeline is check*s*tyle while warning-ng exposes check*S*tyle. Thus there is no clash. At the same time the exposed symbol for android lint is androidLint in both the android-lint plugin and the new plugin, respectively for pmd.

          Ulli Hafner added a comment -

          I tried the following on my local machine:

          node {
              stage ('Checkout') {
                  git branch:'master', url: 'file:///Users/hafner/Development/git/analysis-model'
              }
          
              stage ('Build and Analysis') {
                  def mvnHome = tool 'mvn-default'
          
                  sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean verify revapi:check -Dsurefire.useFile=false -Dmaven.test.failure.ignore"
          
                  junit testResults: '**/target/surefire-reports/TEST-*.xml'
          
                  pmd canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "target/pmd.xml",        unHealthy: '', unstableTotalAll: '0'
              }
          }
          
          

          This works quite well with the PMD plugin 4.0.0 and warnings-ng plugin. There still seems to be something different in your configuration...

          [PMD] Collecting PMD analysis files...
          [PMD] Searching for all files in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model that match the pattern target/pmd.xml
          [PMD] Parsing 1 file in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model
          [PMD] Successfully parsed file /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model/target/pmd.xml with 2 unique warnings and 0 duplicates.
          Skipping warnings blame since pipelines do not have an SCM link.%n
          [PMD] Ignore new warnings since this is the first valid build
          [PMD] Plug-in Result: Unstable - <a href="pmdResult">2 warnings</a> exceed the threshold of 0 by 2
          

          Ulli Hafner added a comment - I tried the following on my local machine: node { stage ( 'Checkout' ) { git branch: 'master' , url: 'file: ///Users/hafner/Development/git/analysis-model' } stage ( 'Build and Analysis' ) { def mvnHome = tool 'mvn- default ' sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean verify revapi:check -Dsurefire.useFile= false -Dmaven.test.failure.ignore" junit testResults: '**/target/surefire-reports/TEST-*.xml' pmd canComputeNew: false , canRunOnFailed: true , defaultEncoding: '', healthy: ' ', pattern: "target/pmd.xml" , unHealthy: ' ', unstableTotalAll: ' 0' } } This works quite well with the PMD plugin 4.0.0 and warnings-ng plugin. There still seems to be something different in your configuration... [PMD] Collecting PMD analysis files... [PMD] Searching for all files in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model that match the pattern target/pmd.xml [PMD] Parsing 1 file in /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model [PMD] Successfully parsed file /Users/hafner/Development/jenkins/workspace/Old - Pipeline Model/target/pmd.xml with 2 unique warnings and 0 duplicates. Skipping warnings blame since pipelines do not have an SCM link.%n [PMD] Ignore new warnings since this is the first valid build [PMD] Plug-in Result: Unstable - <a href="pmdResult">2 warnings</a> exceed the threshold of 0 by 2

          Yeah that worked for me too.
          So I did some further trials.
          And actually androidLint seems to be the culprit. The following declarative script fails for me

          pipeline {
              agent any
          
              stages {
                  stage('Static Analysis') {
                      steps {
                          echo 'Hello World!'
                      }
          
                      post {
                          always {
          // When uncommenting pmd and checkstyle but commenting out androidLint it works                    
          //                    pmd         canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "pmd.xml",        unHealthy: '', unstableTotalAll: '0'
          //                    checkstyle  canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "checkstyle.xml", unHealthy: '', unstableTotalAll: '0'
                              androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "$reports/lint*.xml",      unHealthy: '', unstableTotalAll: '0'
          
                          }
                      }
                  }
              }
          }
           

          Matthias Fuchs added a comment - Yeah that worked for me too. So I did some further trials. And actually androidLint seems to be the culprit. The following declarative script fails for me pipeline { agent any stages { stage( 'Static Analysis' ) { steps { echo 'Hello World!' } post { always { // When uncommenting pmd and checkstyle but commenting out androidLint it works // pmd canComputeNew: false , canRunOnFailed: true , defaultEncoding: '', healthy: ' ', pattern: "pmd.xml" , unHealthy: ' ', unstableTotalAll: ' 0' // checkstyle canComputeNew: false , canRunOnFailed: true , defaultEncoding: '', healthy: ' ', pattern: "checkstyle.xml" , unHealthy: ' ', unstableTotalAll: ' 0' androidLint canComputeNew: false , canRunOnFailed: true , defaultEncoding: '', healthy: ' ', pattern: "$reports/lint*.xml" , unHealthy: ' ', unstableTotalAll: ' 0' } } } } }

          Ulli Hafner added a comment -

          This is really strange. For

          node {
              stage ('Checkout') {
                  git branch:'master', url: 'file:///Users/hafner/Development/git/analysis-model'
              }
          
              stage ('Build and Analysis') {
                  def mvnHome = tool 'mvn-default'
          
                  sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean test-compile -Dsurefire.useFile=false -Dmaven.test.failure.ignore"
          
                  androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "**/lint*.xml",      unHealthy: '', unstableTotalAll: '0'
              }
          }
          
          

          I get the output

          [Pipeline] androidLint
          [android-lint] Collecting Android Lint files...
          [android-lint] Searching for all files in /tmp/node1/workspace/JENKINS-55436 that match the pattern **/lint*.xml
          [android-lint] No files found. Configuration error?
          [android-lint] Ignore new warnings since this is the first valid build
          [android-lint] Plug-in Result: Success - no threshold has been exceeded
          [Pipeline] }
          
          

          Ulli Hafner added a comment - This is really strange. For node { stage ( 'Checkout' ) { git branch: 'master' , url: 'file: ///Users/hafner/Development/git/analysis-model' } stage ( 'Build and Analysis' ) { def mvnHome = tool 'mvn- default ' sh "${mvnHome}/bin/mvn --batch-mode -V -U -e clean test-compile -Dsurefire.useFile= false -Dmaven.test.failure.ignore" androidLint canComputeNew: false , canRunOnFailed: true , defaultEncoding: '', healthy: ' ', pattern: "**/lint*.xml" , unHealthy: ' ', unstableTotalAll: ' 0' } } I get the output [Pipeline] androidLint [android-lint] Collecting Android Lint files... [android-lint] Searching for all files in /tmp/node1/workspace/JENKINS-55436 that match the pattern **/lint*.xml [android-lint] No files found. Configuration error? [android-lint] Ignore new warnings since this is the first valid build [android-lint] Plug-in Result: Success - no threshold has been exceeded [Pipeline] }

          Maybe this is caused by the syntax check performed for declarative pipelines. The actual lookup for which function to call works, while the lookup for the syntax finds the wrong function.

          Matthias Fuchs added a comment - Maybe this is caused by the syntax check performed for declarative pipelines. The actual lookup for which function to call works, while the lookup for the syntax finds the wrong function.

          Ulli Hafner added a comment -

          Ah, this is a declarative pipeline, of course... I'll check this.

          Ulli Hafner added a comment - Ah, this is a declarative pipeline, of course... I'll check this.

          Ulli Hafner added a comment -

          Ok, I can confirm the bug. This definitely is a problem of declarative pipelines only:

          Started by user 
          [Ulli Hafner|http://localhost:8080/user/hafner]
          Replayed 
          [#38|http://localhost:8080/job/New%20-%20Pipeline%20Declarative%20-%20Model/38/]
          Running in Durability level: MAX_SURVIVABILITY
          org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
          WorkflowScript: 26: Invalid parameter "canComputeNew", did you mean "name"? @ line 26, column 25.
                         androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "**/lint*.xml",      unHealthy: '', unstableTotalAll: '0'
                                     ^
          
          WorkflowScript: 26: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 26, column 47.
             oidLint canComputeNew: false, canRunOnFa
                                           ^
          
          WorkflowScript: 26: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 26, column 69.
              false, canRunOnFailed: true, defaultEnc
                                           ^
          
          WorkflowScript: 26: Invalid parameter "healthy", did you mean "name"? @ line 26, column 90.
             d: true, defaultEncoding: '', healthy: '
                                           ^
          
          WorkflowScript: 26: Invalid parameter "unHealthy", did you mean "name"? @ line 26, column 133.
             pattern: "**/lint*.xml",      unHealthy:
                                           ^
          
          WorkflowScript: 26: Invalid parameter "unstableTotalAll", did you mean "name"? @ line 26, column 148.
             nt*.xml",      unHealthy: '', unstableTo
                                           ^
          
          6 errors
          
          	at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
          	at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
          	at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
          	at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
          	at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
          	at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
          	at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
          	at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
          	at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
          	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131)
          	at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560)
          	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521)
          	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320)
          	at hudson.model.ResourceController.execute(ResourceController.java:97)
          	at hudson.model.Executor.run(Executor.java:429)
          Finished: FAILURE
          

          Ulli Hafner added a comment - Ok, I can confirm the bug. This definitely is a problem of declarative pipelines only: Started by user [Ulli Hafner|http://localhost:8080/user/hafner] Replayed [#38|http://localhost:8080/job/New%20-%20Pipeline%20Declarative%20-%20Model/38/] Running in Durability level: MAX_SURVIVABILITY org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed: WorkflowScript: 26: Invalid parameter "canComputeNew", did you mean "name"? @ line 26, column 25. androidLint canComputeNew: false, canRunOnFailed: true, defaultEncoding: '', healthy: '', pattern: "**/lint*.xml", unHealthy: '', unstableTotalAll: '0' ^ WorkflowScript: 26: Invalid parameter "canRunOnFailed", did you mean "name"? @ line 26, column 47. oidLint canComputeNew: false, canRunOnFa ^ WorkflowScript: 26: Invalid parameter "defaultEncoding", did you mean "reportEncoding"? @ line 26, column 69. false, canRunOnFailed: true, defaultEnc ^ WorkflowScript: 26: Invalid parameter "healthy", did you mean "name"? @ line 26, column 90. d: true, defaultEncoding: '', healthy: ' ^ WorkflowScript: 26: Invalid parameter "unHealthy", did you mean "name"? @ line 26, column 133. pattern: "**/lint*.xml", unHealthy: ^ WorkflowScript: 26: Invalid parameter "unstableTotalAll", did you mean "name"? @ line 26, column 148. nt*.xml", unHealthy: '', unstableTo ^ 6 errors at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310) at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085) at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603) at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581) at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558) at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298) at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268) at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688) at groovy.lang.GroovyShell.parse(GroovyShell.java:700) at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:131) at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:125) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:560) at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:521) at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:320) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Finished: FAILURE

          Ulli Hafner added a comment -

          abayer: is there a way to prevent such errors in declarative pipelines? I think we need to add a duplicate-symbol checker in our build infrastructure, otherwise we will fall into this trap very often! 

          Ulli Hafner added a comment - abayer : is there a way to prevent such errors in declarative pipelines? I think we need to add a duplicate-symbol checker in our build infrastructure, otherwise we will fall into this trap very often! 

          Ulli Hafner added a comment -

          Ulli Hafner added a comment - See: https://issues.jenkins-ci.org/browse/INFRA-2012

          Is the issue fixed? Or does it mean that I have to live with the incompatibility?

          Matthias Fuchs added a comment - Is the issue fixed? Or does it mean that I have to live with the incompatibility?

          Ulli Hafner added a comment -

          Should be fixed in 3.0.1

          Ulli Hafner added a comment - Should be fixed in 3.0.1

            drulli Ulli Hafner
            mfuchs Matthias Fuchs
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: