Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-71697

Test Result Aggregator is always showing Total Tests as 1

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker

      • Test Result Aggregator is always showing Total Tests as 1 even when test count is > 1
      • We have done both the global and pipeline configuration needed for integrating Test Result Aggregator

          [JENKINS-71697] Test Result Aggregator is always showing Total Tests as 1

          Piyush created issue -
          Socrates Sidereas made changes -
          Attachment New: image-2023-07-24-21-57-07-827.png [ 60871 ]

          Hi Piyush, can you please share a part from the JSON response that you'll find if you open in a browser the URL of the job and then add 

          /lastBuild/api/json?depth=1

          You should see a part of the JSON like the following : 

           

          Socrates Sidereas added a comment - Hi Piyush, can you please share a part from the JSON response that you'll find if you open in a browser the URL of the job and then add  /lastBuild/api/json?depth=1 You should see a part of the JSON like the following :   
          Socrates Sidereas made changes -
          Status Original: Open [ 1 ] New: In Progress [ 3 ]
          Piyush made changes -
          Attachment New: image-2023-07-25-09-09-42-629.png [ 60872 ]

          Piyush added a comment -

          Hi Socrates, Thanks for replying. Please find the required json response screenshot below:

          Piyush added a comment - Hi Socrates, Thanks for replying. Please find the required json response screenshot below:

          Piyush added a comment - - edited

          Hi Socrates,
          we have observed one more thing:
          When we provide values for parameters recipientsList and subject in testResultsAggregator step, the testResultsAggregator step fails with below error

           

          Generate Email Report
          Error Occurred: Couldn't connect to host, port: localhost, 25; timeout 60000Sent message successfully to : 
          Error Occurred : com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 25; timeout 60000;
            nested exception is:
          	java.net.ConnectException: Connection refused com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 25; timeout 60000;
            nested exception is:
          	java.net.ConnectException: Connection refused
          	at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2210)
          	at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:722)
          	at jakarta.mail.Service.connect(Service.java:342)
          	at jakarta.mail.Service.connect(Service.java:222)
          	at jakarta.mail.Service.connect(Service.java:171)
          	at jakarta.mail.Transport.send0(Transport.java:230)
          	at jakarta.mail.Transport.send(Transport.java:100)
          	at com.jenkins.testresultsaggregator.reporter.MailNotification.sendMessage(MailNotification.java:211)
          	at com.jenkins.testresultsaggregator.reporter.MailNotification.send(MailNotification.java:139)
          	at com.jenkins.testresultsaggregator.reporter.Reporter.publishResuts(Reporter.java:88)
          	at com.jenkins.testresultsaggregator.TestResultsAggregator.perform(TestResultsAggregator.java:204)
          	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101)
          	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71)
          	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
          	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
          	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
          	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
          	at java.base/java.lang.Thread.run(Thread.java:833)
          Caused by: java.net.ConnectException: Connection refused
          	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
          	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
          	at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
          	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597)
          	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
          	at java.base/java.net.Socket.connect(Socket.java:633)
          	at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:333)
          	at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:214)
          	at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2160)
          	... 18 more 

          But when I trigger the next build after removing parameters  recipientsList and subject in testResultsAggregator step, we get correct value for Total Tests and Success/Fail Tests, but these values are of the last build not the current one.
          For example, if we have executed 5 tests in the last build, out of which 3 passed and 2 failed.
          And in the current build we have run 10 tests, out of which 7 passed and  3 failed.
          On "Job Results Aggregated" page, we see the summary as follows:

          Total Tests : 5(+5)
          Failed Tests : 2(+2)
          Aborted Tests : 0
          Success Tests : 3(+3) 

          But, the expected summary is as follows:

          Total Tests : 10(+10)
          Failed Tests : 3(+3)
          Aborted Tests : 0
          Success Tests : 7(+7)  

           

          Piyush added a comment - - edited Hi Socrates, we have observed one more thing: When we provide values for parameters recipientsList and subject in testResultsAggregator step, the testResultsAggregator step fails with below error   Generate Email Report Error Occurred: Couldn't connect to host, port: localhost, 25; timeout 60000Sent message successfully to : Error Occurred : com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 25; timeout 60000; nested exception is: java.net.ConnectException: Connection refused com.sun.mail.util.MailConnectException: Couldn't connect to host, port: localhost, 25; timeout 60000; nested exception is: java.net.ConnectException: Connection refused at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2210) at com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:722) at jakarta.mail.Service.connect(Service.java:342) at jakarta.mail.Service.connect(Service.java:222) at jakarta.mail.Service.connect(Service.java:171) at jakarta.mail.Transport.send0(Transport.java:230) at jakarta.mail.Transport.send(Transport.java:100) at com.jenkins.testresultsaggregator.reporter.MailNotification.sendMessage(MailNotification.java:211) at com.jenkins.testresultsaggregator.reporter.MailNotification.send(MailNotification.java:139) at com.jenkins.testresultsaggregator.reporter.Reporter.publishResuts(Reporter.java:88) at com.jenkins.testresultsaggregator.TestResultsAggregator.perform(TestResultsAggregator.java:204) at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:101) at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:71) at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang. Thread .run( Thread .java:833) Caused by: java.net.ConnectException: Connection refused at java.base/sun.nio.ch.Net.pollConnect(Native Method) at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) at java.base/java.net.Socket.connect(Socket.java:633) at com.sun.mail.util.SocketFetcher.createSocket(SocketFetcher.java:333) at com.sun.mail.util.SocketFetcher.getSocket(SocketFetcher.java:214) at com.sun.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2160) ... 18 more But when I trigger the next build after removing parameters   recipientsList and subject in testResultsAggregator step, we get correct value for Total Tests and Success/Fail Tests, but these values are of the last build not the current one. For example, if we have executed 5 tests in the last build, out of which 3 passed and 2 failed. And in the current build we have run 10 tests, out of which 7 passed and  3 failed. On "Job Results Aggregated" page, we see the summary as follows: Total Tests : 5(+5) Failed Tests : 2(+2) Aborted Tests : 0 Success Tests : 3(+3) But, the expected summary is as follows: Total Tests : 10(+10) Failed Tests : 3(+3) Aborted Tests : 0 Success Tests : 7(+7)  
          Socrates Sidereas made changes -
          Attachment New: image-2023-07-25-19-30-43-737.png [ 60877 ]

          Socrates Sidereas added a comment - - edited

          This one is strange : 

          Why there are tests into a class named com,jenkins.testresultsaggregator. etc ?

          Probably has to do with the fact that the execution of the tests and the call to aggregator are running from the same groovy file and sharing the same workspace. 

          This is not the purpose of this plugin , meaning that since you are having only one job running tests why do you need the aggregator plugin ? 

           

          Socrates Sidereas added a comment - - edited This one is strange :  Why there are tests into a class named com,jenkins.testresultsaggregator. etc ? Probably has to do with the fact that the execution of the tests and the call to aggregator are running from the same groovy file and sharing the same workspace.  This is not the purpose of this plugin , meaning that since you are having only one job running tests why do you need the aggregator plugin ?   
          Socrates Sidereas made changes -
          Attachment New: image-2023-07-25-19-40-55-024.png [ 60878 ]

            socrates Socrates Sidereas
            pkcodes Piyush
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: