Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37610

Sauce Labs Results Publisher throws Exception in Pipeline

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I'm trying to use the Sauce On Demand Plug in Pipeline. I've been able to get my project to build and run tests using SauceConnect, but I can't get the Sauce-based results publishing step to work.

      There's little to not documentation on this, and what there is seems to apply to the Freestyle job type, not pipeline.

      The Snippet Generator for the publish step generates this output:

      step([$class: 'SauceOnDemandTestPublisher', testDataPublishers: []])
      

      Which throws this exception when run as part of a pipeline:

      java.lang.IllegalArgumentException: argument type mismatch
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:258)
      	at org.jenkinsci.plugins.structs.describable.DescribableModel.coerce(DescribableModel.java:372)
      	at org.jenkinsci.plugins.structs.describable.DescribableModel.buildArguments(DescribableModel.java:313)
      	at org.jenkinsci.plugins.structs.describable.DescribableModel.instantiate(DescribableModel.java:257)
      	at org.jenkinsci.plugins.workflow.steps.StepDescriptor.newInstance(StepDescriptor.java:195)
      	at org.jenkinsci.plugins.workflow.cps.DSL.invokeStep(DSL.java:175)
      	at org.jenkinsci.plugins.workflow.cps.DSL.invokeMethod(DSL.java:124)
      	at org.jenkinsci.plugins.workflow.cps.CpsScript.invokeMethod(CpsScript.java:117)
      	at groovy.lang.GroovyObject$invokeMethod.call(Unknown Source)
      	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
      	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
      	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:151)
      	at org.kohsuke.groovy.sandbox.GroovyInterceptor.onMethodCall(GroovyInterceptor.java:21)
      	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.SandboxInterceptor.onMethodCall(SandboxInterceptor.java:115)
      	at org.kohsuke.groovy.sandbox.impl.Checker$1.call(Checker.java:149)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:146)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
      	at org.kohsuke.groovy.sandbox.impl.Checker.checkedCall(Checker.java:123)
      	at com.cloudbees.groovy.cps.sandbox.SandboxInvoker.methodCall(SandboxInvoker.java:16)
      	at WorkflowScript.run(WorkflowScript:28)
      	at ___cps.transform___(Native Method)
      	at com.cloudbees.groovy.cps.impl.ContinuationGroup.methodCall(ContinuationGroup.java:48)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.dispatchOrArg(FunctionCallBlock.java:109)
      	at com.cloudbees.groovy.cps.impl.FunctionCallBlock$ContinuationImpl.fixArg(FunctionCallBlock.java:82)
      	at sun.reflect.GeneratedMethodAccessor324.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.dispatch(CollectionLiteralBlock.java:55)
      	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.item(CollectionLiteralBlock.java:45)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at com.cloudbees.groovy.cps.impl.ContinuationPtr$ContinuationImpl.receive(ContinuationPtr.java:72)
      	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.dispatch(CollectionLiteralBlock.java:55)
      	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock$ContinuationImpl.access$000(CollectionLiteralBlock.java:31)
      	at com.cloudbees.groovy.cps.impl.CollectionLiteralBlock.eval(CollectionLiteralBlock.java:26)
      	at com.cloudbees.groovy.cps.Next.step(Next.java:58)
      	at com.cloudbees.groovy.cps.Continuable.run0(Continuable.java:154)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.access$001(SandboxContinuable.java:18)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:33)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable$1.call(SandboxContinuable.java:30)
      	at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:108)
      	at org.jenkinsci.plugins.workflow.cps.SandboxContinuable.run0(SandboxContinuable.java:30)
      	at org.jenkinsci.plugins.workflow.cps.CpsThread.runNextChunk(CpsThread.java:164)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:361)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:236)
      	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:226)
      	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:47)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
      	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Activity

          Hide
          halkeye Gavin Mogan added a comment -

          Hi Liam Newman

          Can you update the ticket with the version number? Is it the one we released yesterday? Did you have it working previously?

          https://github.com/saucelabs-sample-test-frameworks/Java-TestNG-Selenium/blob/master/Jenkinsfile#L15-L16 is the test case I've been using

          step([$class: 'JUnitResultArchiver', testResults: '**/target/surefire-reports/TEST-*.xml'])
          step([$class: 'SauceOnDemandTestPublisher'])
          

          The fact that you have

          , testDataPublishers: []

          makes me think its trying to do something weird. How did that get generated? I'll try to look into the bug asap though. For now can you drop that chunk?

          Gavin

          Show
          halkeye Gavin Mogan added a comment - Hi Liam Newman Can you update the ticket with the version number? Is it the one we released yesterday? Did you have it working previously? https://github.com/saucelabs-sample-test-frameworks/Java-TestNG-Selenium/blob/master/Jenkinsfile#L15-L16 is the test case I've been using step([$class: 'JUnitResultArchiver' , testResults: '**/target/surefire-reports/TEST-*.xml' ]) step([$class: 'SauceOnDemandTestPublisher' ]) The fact that you have , testDataPublishers: [] makes me think its trying to do something weird. How did that get generated? I'll try to look into the bug asap though. For now can you drop that chunk? Gavin
          Hide
          bitwiseman Liam Newman added a comment - - edited

          I'm using version 1.154. The one you released yesterday is not on jenkins.io yet.

          I just used the Snippet Generator to get that syntax. There are no parameters in the UI, but the generator adds "testDataPublishers".

          So, I need to use the JUnitArchiver and then use the SaucePublisher afterward? No problem, I was already doing that. I'll removed that extra parameter.

          Show
          bitwiseman Liam Newman added a comment - - edited I'm using version 1.154. The one you released yesterday is not on jenkins.io yet. I just used the Snippet Generator to get that syntax. There are no parameters in the UI, but the generator adds "testDataPublishers". So, I need to use the JUnitArchiver and then use the SaucePublisher afterward? No problem, I was already doing that. I'll removed that extra parameter.
          Hide
          halkeye Gavin Mogan added a comment -

          okay, I'll look into why the snippet generator creating that snippet. It shouldn't need to at all.

          SaucePublisher looks at your log, and junit tests, so its better to do it afterwards.

          Does that fix things?

          Show
          halkeye Gavin Mogan added a comment - okay, I'll look into why the snippet generator creating that snippet. It shouldn't need to at all. SaucePublisher looks at your log, and junit tests, so its better to do it afterwards. Does that fix things?
          Hide
          bitwiseman Liam Newman added a comment -

          I did as you suggested. I no longer get an exception, but I still don't get any results in the Sauce Results section.

          I've added this to my capabilities (java):

          desiredCapabilities.setCapability("build", System.getenv("JOB_NAME") + "__" + System.getenv("BUILD_NUMBER"));
          

          I've been exclusively using pipeline for this and it is basically working, but have been unable to figure out the reporting.
          At the very least there's a bug here that the Snippet generator output is incorrect, but more generally it seems like the reporting/publishing integration is really easy to get wrong.

          Show
          bitwiseman Liam Newman added a comment - I did as you suggested. I no longer get an exception, but I still don't get any results in the Sauce Results section. I've added this to my capabilities (java): desiredCapabilities.setCapability( "build" , System .getenv( "JOB_NAME" ) + "__" + System .getenv( "BUILD_NUMBER" )); I've been exclusively using pipeline for this and it is basically working, but have been unable to figure out the reporting. At the very least there's a bug here that the Snippet generator output is incorrect, but more generally it seems like the reporting/publishing integration is really easy to get wrong.
          Hide
          halkeye Gavin Mogan added a comment -

          If you get me a sessionId i'll take a look

          The idea is we pull jobs where build=env.JENKINS_BUILD_NUMBER && anything that is outputted to tests/stdout
          i'm not sure how well the second half works as it predates me, but I can certainly take a look.

          Show
          halkeye Gavin Mogan added a comment - If you get me a sessionId i'll take a look The idea is we pull jobs where build=env.JENKINS_BUILD_NUMBER && anything that is outputted to tests/stdout i'm not sure how well the second half works as it predates me, but I can certainly take a look.
          Hide
          bitwiseman Liam Newman added a comment - - edited

          Okay, I got it working finally.
          The docs at https://wiki.saucelabs.com/display/DOCS/Setting+Up+Reporting+between+Sauce+Labs+and+Jenkins are correct but not great.

          Now the projects in https://github.com/saucelabs-sample-test-frameworks/ are much more helpful.

          I spun up each of these projects and could immediately understand what values were supposed to be:
          https://github.com/saucelabs-sample-test-frameworks/Java-Junit-Selenium
          https://github.com/saucelabs-sample-test-frameworks/JS-Nightwatch.js

          Wish I'd found these earlier. Any way to get the docs to point to these (pretty much at the top of every page of the docs)? They are great.

          Show
          bitwiseman Liam Newman added a comment - - edited Okay, I got it working finally. The docs at https://wiki.saucelabs.com/display/DOCS/Setting+Up+Reporting+between+Sauce+Labs+and+Jenkins are correct but not great. Now the projects in https://github.com/saucelabs-sample-test-frameworks/ are much more helpful. I spun up each of these projects and could immediately understand what values were supposed to be: https://github.com/saucelabs-sample-test-frameworks/Java-Junit-Selenium https://github.com/saucelabs-sample-test-frameworks/JS-Nightwatch.js Wish I'd found these earlier. Any way to get the docs to point to these (pretty much at the top of every page of the docs)? They are great.
          Hide
          halkeye Gavin Mogan added a comment -

          My goal is to get those examples all updated an official, they are just not yet

          I'll get it noted as a feature you want.

          Show
          halkeye Gavin Mogan added a comment - My goal is to get those examples all updated an official, they are just not yet I'll get it noted as a feature you want.
          Hide
          mehmetg Mehmet Gerceker added a comment - - edited

          The issue with the pipeline syntax tool is still there:

          http://localhost:8080/jenkins/job/testing/pipeline-syntax/

          It'd be helpful id this was on the Sauce Wiki.

          Show
          mehmetg Mehmet Gerceker added a comment - - edited The issue with the pipeline syntax tool is still there: http://localhost:8080/jenkins/job/testing/pipeline-syntax/ It'd be helpful id this was on the Sauce Wiki.
          Hide
          halkeye Gavin Mogan added a comment -

          Okay, I had time today, so did a bit of refactoring. Should be released tomorrow, monday at the latest.

          Also got it to not include that extra field, and made a `saucePublisher` shortcut

          Show
          halkeye Gavin Mogan added a comment - Okay, I had time today, so did a bit of refactoring. Should be released tomorrow, monday at the latest. Also got it to not include that extra field, and made a `saucePublisher` shortcut
          Hide
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Gavin Mogan
          Path:
          pom.xml
          src/main/java/hudson/plugins/sauce_ondemand/SauceOnDemandTestPublisher.java
          src/main/resources/hudson/plugins/sauce_ondemand/SauceOnDemandTestPublisher/config.jelly
          src/main/resources/hudson/plugins/sauce_ondemand/SauceOnDemandTestPublisher/help.html
          http://jenkins-ci.org/commit/sauce-ondemand-plugin/cc61f90fe98e7dfc03c453155d9187befd2a6966
          Log:
          Make parameters for publish sauce labs test results pipeline step optional - JENKINS-37610

          Also make a helper symbol of saucePublisher()

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Gavin Mogan Path: pom.xml src/main/java/hudson/plugins/sauce_ondemand/SauceOnDemandTestPublisher.java src/main/resources/hudson/plugins/sauce_ondemand/SauceOnDemandTestPublisher/config.jelly src/main/resources/hudson/plugins/sauce_ondemand/SauceOnDemandTestPublisher/help.html http://jenkins-ci.org/commit/sauce-ondemand-plugin/cc61f90fe98e7dfc03c453155d9187befd2a6966 Log: Make parameters for publish sauce labs test results pipeline step optional - JENKINS-37610 Also make a helper symbol of saucePublisher()
          Hide
          halkeye Gavin Mogan added a comment -

          Liam Newman
          this should now be fixed in 1.160

          https://github.com/saucelabs/jenkins-sauce-ondemand-plugin/blob/master/CHANGELOG.md#jenkins-1160---2017-01-27

          I'll mark it as resolved, reopen if necessary

          Show
          halkeye Gavin Mogan added a comment - Liam Newman this should now be fixed in 1.160 https://github.com/saucelabs/jenkins-sauce-ondemand-plugin/blob/master/CHANGELOG.md#jenkins-1160---2017-01-27 I'll mark it as resolved, reopen if necessary

            People

            Assignee:
            halkeye Gavin Mogan
            Reporter:
            bitwiseman Liam Newman
            Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: