Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-30778

Sauce Labs Test Publisher Configuration

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      I'm using TestNG (also tried JUnit) to run tests in Sauce Labs cloud. In order to update the pass/fail status of those tests in Sauce Labs once they done, I added the "Run Sauce Labs Test Publisher" post-build action to my Jenkins build.

      First of all, the documentation isn't even clear about how this integration must be configured. This is what it says: "In addition, you will need to output the following line to your for each test that is run using Sauce Labs: SauceOnDemandSessionID=YOUR_SESSION_ID job-name=YOUR_JOB_NAME." What does that mean? To your what?

      Based on source code, I figured there has to be <system-out> tag in the Surefire XML Report, the content of which must be that line. But that doesn't make much sense, because those tags are only generated when an error happens. Also no matter what I've tried, I just couldn't make Surefire/TestNG output that tag into the report at all. Not even when test failed.

      I'm either not understanding something (probably) or this way of configuring things is messed up.

      Last few lines of my log:
      Starting post-build for Sauce Labs plugin
      Finished post-build for Sauce Labs plugin
      Recording test results
      Starting Sauce Labs test publisher
      Finished Sauce Labs test publisher
      Starting Sauce Labs test publisher
      Finished Sauce Labs test publisher

        Attachments

          Issue Links

            Activity

            Hide
            svenstyla Sven Ehlert added a comment -

            I have the exact same problem. Tests results are not shown on Jenkins. I get the same output on the log as the original reporter

            Show
            svenstyla Sven Ehlert added a comment - I have the exact same problem. Tests results are not shown on Jenkins. I get the same output on the log as the original reporter
            Hide
            svenstyla Sven Ehlert added a comment -

            No report printed, but with different testing framworks

            Show
            svenstyla Sven Ehlert added a comment - No report printed, but with different testing framworks
            Hide
            thecampbellcash Andrew Campbell added a comment -

            Hi Sven Ehlert y Alex G., regarding this line from the documentation....

            First of all, the documentation isn't even clear about how this integration must be configured. This is what it says: "In addition, you will need to output the following line to your for each test that is run using Sauce Labs: SauceOnDemandSessionID=YOUR_SESSION_ID job-name=YOUR_JOB_NAME." What does that mean? To your what?

            It is confusing and we will work to alter the docs to be more clear in the future. In the meantime I can clarify. In this case, intended destination of the session ID text is the standard output console or Jenkins.

            However, there could be an easier way to accomplish what you are looking for and a code example can be found here. The SauceOnDemandTestWatcher class should listen for test completion and output the results to console for you. I would advise having a look as I think it could be helpful for both of you.

            I hope this helps and please do let me know if there are any questions.

            Show
            thecampbellcash Andrew Campbell added a comment - Hi Sven Ehlert y Alex G. , regarding this line from the documentation.... First of all, the documentation isn't even clear about how this integration must be configured. This is what it says: "In addition, you will need to output the following line to your for each test that is run using Sauce Labs: SauceOnDemandSessionID=YOUR_SESSION_ID job-name=YOUR_JOB_NAME." What does that mean? To your what? It is confusing and we will work to alter the docs to be more clear in the future. In the meantime I can clarify. In this case, intended destination of the session ID text is the standard output console or Jenkins. However, there could be an easier way to accomplish what you are looking for and a code example can be found here . The SauceOnDemandTestWatcher class should listen for test completion and output the results to console for you. I would advise having a look as I think it could be helpful for both of you. I hope this helps and please do let me know if there are any questions.
            Hide
            mark_han Mark Han added a comment -

            2019 and the same documentation still exists :\ the SauceOnDemandTestWatcher.java class helped.

            Show
            mark_han Mark Han added a comment - 2019 and the same documentation still exists :\ the SauceOnDemandTestWatcher.java class helped.
            Hide
            danzhang dan zhang added a comment -

            based on the online doc, you also need this:

            capabilities.setCapability("build", System.getenv("JOB_NAME")  "__"  System.getenv("BUILD_NUMBER"));

            {{}}

            {{}}In my jenkins environment: "BUILD_TAG" is correct.

             

             

            Show
            danzhang dan zhang added a comment - based on the online doc, you also need this: capabilities.setCapability( "build" , System.getenv( "JOB_NAME" )   "__"   System.getenv( "BUILD_NUMBER" )); {{}} {{}}In my jenkins environment: "BUILD_TAG" is correct.    

              People

              Assignee:
              thecampbellcash Andrew Campbell
              Reporter:
              grulex Alex G.
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated: