Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-37739

Support multiple output.xml's to support parallel test executions in pipeline

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open (View Workflow)
    • Priority: Trivial
    • Resolution: Unresolved
    • Component/s: robot-plugin
    • Labels:
      None
    • Similar Issues:

      Description

      I am building running my robot framework test case parallel in Jenkins pipeline. However the Robot Test Summary generated post build is not correct, it displaying the count/summary of last finished in parallelism, however it should show the count of individual job. It was overwriting the report also earlier but that I saved as different name & it worked fine, but still the Robot Test Summary is problem(please not my both suite below have different number of test cases).

      suite1:{
      try

      { mvn -s /settings.xml install robotframework:run -Dexcludes=inprogress -Dsuites=test1 -DLogFileName=suite1_log.html -DoutputFileName=suite1_output.xml -DreportFileName=suite1_report.html' currentBuild.result = 'SUCCESS' }

      catch(any)

      { currentBuild.result = 'FAILURE' throw any }

      finally

      { step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite1_log.html', outputFileName: 'suite1_output.xml', reportFileName: 'suite1_report.html']) }

      },
      suite2:{
      try

      { mvn -s /settings.xml robotframework:run -Dexcludes=inprogress -Dsuites=test2 -DLogFileName=suite2_log.html -DoutputFileName=suite2_output.xml -DreportFileName=suite2_report.html' currentBuild.result = 'SUCCESS' }

      catch(any)

      { currentBuild.result = 'FAILURE' //sendMail() throw any }

      finally

      { step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports/', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite2_log.html', outputFileName: 'suite2_output.xml', reportFileName: 'suite2_report.html']) }

        Attachments

          Activity

          singhshyam shyam singh created issue -
          singhshyam shyam singh made changes -
          Field Original Value New Value
          Attachment Robot Test Summary.png [ 33749 ]
          singhshyam shyam singh made changes -
          Attachment Robot Test Summary.png [ 33750 ]
          singhshyam shyam singh made changes -
          Description I am building running my robot framework test case parallel in Jenkins pipeline. However the Robot Test Summary generated post build is not correct, it displaying the count/summary of last finished in parallelism, however it should show the count of individual job. It was merging the report also earlier but that I saved as different name & it worked fine, but still the Robot Test Summary is problem(please not my both suite below have different number of test cases).

          suite1:{
                              try {
          mvn -s /settings.xml install robotframework:run -Dexcludes=inprogress -Dsuites=test1 -DLogFileName=suite1_log.html -DoutputFileName=suite1_output.xml -DreportFileName=suite1_report.html'
                              
                                   currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite1_log.html', outputFileName: 'suite1_output.xml', reportFileName: 'suite1_report.html'])
                              }
          },
          suite2:{
                              try {
          mvn -s /settings.xml robotframework:run -Dexcludes=inprogress -Dsuites=test2 -DLogFileName=suite2_log.html -DoutputFileName=suite2_output.xml -DreportFileName=suite2_report.html'
                              
                                   currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  //sendMail()
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports/', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite2_log.html', outputFileName: 'suite2_output.xml', reportFileName: 'suite2_report.html'])
                              }
          I am building running my robot framework test case parallel in Jenkins pipeline. However the Robot Test Summary generated post build is not correct, it displaying the count/summary of last finished in parallelism, however it should show the count of individual job. It was merging the report also earlier but that I saved as different name & it worked fine, but still the Robot Test Summary is problem(please not my both suite below have different number of test cases).

          suite1:{
                              try {
          mvn -s /settings.xml install robotframework:run -Dexcludes=inprogress -Dsuites=test1 -DLogFileName=suite1_log.html -DoutputFileName=suite1_output.xml -DreportFileName=suite1_report.html'
                              
                                   
          currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite1_log.html', outputFileName: 'suite1_output.xml', reportFileName: 'suite1_report.html'])
                              }
          },
          suite2:{
                              try {
          mvn -s /settings.xml robotframework:run -Dexcludes=inprogress -Dsuites=test2 -DLogFileName=suite2_log.html -DoutputFileName=suite2_output.xml -DreportFileName=suite2_report.html'
                              
                                   currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  //sendMail()
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports/', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite2_log.html', outputFileName: 'suite2_output.xml', reportFileName: 'suite2_report.html'])
                              }
          singhshyam shyam singh made changes -
          Description I am building running my robot framework test case parallel in Jenkins pipeline. However the Robot Test Summary generated post build is not correct, it displaying the count/summary of last finished in parallelism, however it should show the count of individual job. It was merging the report also earlier but that I saved as different name & it worked fine, but still the Robot Test Summary is problem(please not my both suite below have different number of test cases).

          suite1:{
                              try {
          mvn -s /settings.xml install robotframework:run -Dexcludes=inprogress -Dsuites=test1 -DLogFileName=suite1_log.html -DoutputFileName=suite1_output.xml -DreportFileName=suite1_report.html'
                              
                                   
          currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite1_log.html', outputFileName: 'suite1_output.xml', reportFileName: 'suite1_report.html'])
                              }
          },
          suite2:{
                              try {
          mvn -s /settings.xml robotframework:run -Dexcludes=inprogress -Dsuites=test2 -DLogFileName=suite2_log.html -DoutputFileName=suite2_output.xml -DreportFileName=suite2_report.html'
                              
                                   currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  //sendMail()
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports/', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite2_log.html', outputFileName: 'suite2_output.xml', reportFileName: 'suite2_report.html'])
                              }
          I am building running my robot framework test case parallel in Jenkins pipeline. However the Robot Test Summary generated post build is not correct, it displaying the count/summary of last finished in parallelism, however it should show the count of individual job. It was overwriting the report also earlier but that I saved as different name & it worked fine, but still the Robot Test Summary is problem(please not my both suite below have different number of test cases).

          suite1:{
                              try {
          mvn -s /settings.xml install robotframework:run -Dexcludes=inprogress -Dsuites=test1 -DLogFileName=suite1_log.html -DoutputFileName=suite1_output.xml -DreportFileName=suite1_report.html'
                              
                                   
          currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite1_log.html', outputFileName: 'suite1_output.xml', reportFileName: 'suite1_report.html'])
                              }
          },
          suite2:{
                              try {
          mvn -s /settings.xml robotframework:run -Dexcludes=inprogress -Dsuites=test2 -DLogFileName=suite2_log.html -DoutputFileName=suite2_output.xml -DreportFileName=suite2_report.html'
                              
                                   currentBuild.result = 'SUCCESS'
                              }catch(any){
                                  currentBuild.result = 'FAILURE'
                                  //sendMail()
                                  throw any
                              }finally{
                                  step([$class: 'RobotPublisher', outputPath:'/source/xxxxxx/target/robotframework-reports/', passThreshold: 0, unstableThreshold: 0, otherFiles: "", logFileName: 'suite2_log.html', outputFileName: 'suite2_output.xml', reportFileName: 'suite2_report.html'])
                              }
          iwelch Ian Welch made changes -
          Attachment MultipleRobotResults-Summary.png [ 34362 ]
          dirkrichter Dirk Richter made changes -
          Priority Critical [ 2 ] Trivial [ 5 ]
          adir_atias Adir Atias made changes -
          Assignee jpiironen [ jpiironen ] Adir Atias [ adir_atias ]
          adir_atias Adir Atias made changes -
          Assignee Adir Atias [ adir_atias ] jpiironen [ jpiironen ]
          jpiironen jpiironen made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          hifi Juho Saarinen made changes -
          Assignee jpiironen [ jpiironen ] Juho Saarinen [ hifi ]
          hifi Juho Saarinen made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          singhshyam shyam singh made changes -
          Attachment RobotFramework_testTable.PNG [ 46025 ]
          astaruch Andrej Staruch made changes -
          Status In Progress [ 3 ] Open [ 1 ]
          tattoo Tatu Kairi made changes -
          Summary Plugin not displaying Robot Test Summary correctly if robot framework task running in parallel in pipeline Support multiple output.xml's to support parallel test executions in pipeline
          singhshyam shyam singh made changes -
          Attachment image-2021-03-02-16-20-43-422.png [ 54088 ]

            People

            Assignee:
            hifi Juho Saarinen
            Reporter:
            singhshyam shyam singh
            Votes:
            10 Vote for this issue
            Watchers:
            20 Start watching this issue

              Dates

              Created:
              Updated: