-
Bug
-
Resolution: Not A Defect
-
Blocker
-
None
-
Powered by SuggestiMate
We have a main job running the gradle test command to run some test cases and archive the artifacts at the end. the main job calls multiple downstream jobs as a post-build success step, which in turn runs some gradle test cases and archives the artifacts just like the main job. The main job is supposed to wait for all the downstream jobs to finish and aggregate the test results of all the jobs and display the results in the common view.
We are using the latest Jenkins version: 2.375. and have the latest version of the Test Results Aggregator plugin installed: 1.2.12. We are following the correct syntax to configure the main job as per this doc: https://github.com/sdrss/test-results-aggregator-plugin
Have configured the Plugin in the Jenkins global setting also correctly, But still the plugin is not fetching the test results of individual downstream jobs and we are not seeing any errors in the console logs. Attaching the console logs of the main job in the screenshots
NOTE: The plugin was working fine just 4 weeks back and was able to aggregate all the downstream jobs test results in the main job under the Job Results Aggregator view but is not showing any results now.
17:15:05 BUILD SUCCESSFUL in 1m 56s 17:15:05 93 actionable tasks: 78 executed, 15 up-to-date 17:15:05 + mkdir -p build/fingerprint 17:15:05 + echo Care_QA/qa-01-critical-response--build-all-78 17:15:05 + ls build 17:15:05 fingerprint 17:15:05 kotlin 17:15:05 reports 17:15:05 resources 17:15:05 test-results 17:15:05 version [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (archive and aggregate) [Pipeline] archiveArtifacts 17:15:05 Archiving artifacts 17:15:05 Recording fingerprints [Pipeline] junit 17:15:06 Recording test results 17:15:06 [Checks API] No suitable checks publisher found. [Pipeline] script [Pipeline] { [Pipeline] echo 17:15:06 Care_QA/qa-01-critical-response-AdminIntegrationsSuite [Pipeline] build (Building Care QA » qa-01-critical-response-AdminIntegrationsSuite) 17:15:06 Scheduling project: Care QA » qa-01-critical-response-AdminIntegrationsSuite 17:15:14 Starting building: Care QA » qa-01-critical-response-AdminIntegrationsSuite #37 [Pipeline] copyArtifacts 17:19:02 Copied 1 artifact from "Care QA » qa-01-critical-response-AdminIntegrationsSuite" build number 37 [Pipeline] testResultsAggregator (hide) 17:19:02 Starting Aggregate Test Results Action 17:19:10 Not found results for Care_QA/qa-01-critical-response-AdminIntegrationsSuite 17:19:10 Analyze ...Finished 17:19:10 Generate XML Report...Finished XML Report 17:19:10 Generate HTML Report...Finished HTML Report 17:19:10 Generate HTML Report for all Ignored Jobs...Finished HTML Report 17:19:10 Generate Email Report 17:19:10 No mail will be Send since all Jobs are having status 'NOT_FOUND'. 17:19:10 Generate Email Report for all Ignored Jobs 17:19:10 No mail will be Send since all Jobs are having status 'NOT_FOUND'. 17:19:10 Finished Aggregate Test Results Action [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS
[JENKINS-71586] Test Results Aggregator plugin not fetching the test results of downstream jobs
Hi, The URL which is working for us is: https://care-jenkins.qa.????/job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite/ not the http one:
http://care-jenkins.????:8080/job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite , But since the Plugin doc shared above shows it needs to be configured using the http url. What should we use in place of base URL in Jenkins global configuration.
And also If I use the https base URL:https://care-jenkins.qa.????/ in the Jenkins global configuration, We get the connection error screenshot attached below.
Sorry the ??? means replace it with what you have in Screenshot 2023-07-04 at 2.08.35 PM.png
Your jenkins base URL and then job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite
Forget anything about "?"
So using your base URL and then /job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite/api/json it works? you can see the job's json response?
yes, I am replacing it, but still it shows a connection issue. Just hiding the rest of the URL. But If I give http://care-jenkins.????:8080/ in the config for the plugin. It connects.
To answer your above question, Yes I can see the JSON response using this URL:
https://care-jenkins.qa.????.com/job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite/
This is our Jenkins base URL: https://care-jenkins.qa./????.com/ hiding the rest of the URL. It works for us and also if I append api/json to it. The only doubt I am having is Do we have to keep the same URL in the Jenkins Global setting for the plugin part? But it won't show connection success as shared above and also I am using my Jenkins username and password to connect which was working 4 weeks back.
To add more info I am using the following lines of code in the Jenkins pipeline to test as shared as file in the attachment:
Looking back to the code and trying to reproduce it, I have posted something that was not true
The message :
Not found results for Care_QA/qa-01-critical-response-AdminIntegrationsSuite
means that the job has been found but there are no results available.
I am seeing a logical issue in the plugin and has to do with the 'ignore running jobs'. Something that changed into the last version
I am reproducing your case, you really help me especially with the :
I am going to have a release as soon as possible , thanks again
socrates Really appreciate your efforts! Hoping to get a resolution from your side soon!
I uninstalled the old version, cleaned up all the plugin-related files from our Jenkins master node home and plugins folder, and then restarted and installed the latest plugin version. However, We are still seeing the same thing:
21:20:14 Archiving artifacts 21:20:14 Recording fingerprints [Pipeline] junit 21:20:14 Recording test results 21:20:14 [Checks API] No suitable checks publisher found. [Pipeline] script [Pipeline] { [Pipeline] echo 21:20:14 Care_QA/qa-01-critical-response-AdminIntegrationsSuite [Pipeline] build (Building Care QA » qa-01-critical-response-AdminIntegrationsSuite) 21:20:14 Scheduling project: Care QA » qa-01-critical-response-AdminIntegrationsSuite 21:20:22 Starting building: Care QA » qa-01-critical-response-AdminIntegrationsSuite #44 [Pipeline] copyArtifacts 21:22:32 Copied 1 artifact from "Care QA » qa-01-critical-response-AdminIntegrationsSuite" build number 44 [Pipeline] testResultsAggregator 21:22:32 Starting Aggregate Test Results Action 21:22:40 Not found results for Care_QA/qa-01-critical-response-AdminIntegrationsSuite 21:22:40 Analyze ...Finished 21:22:40 Generate XML Report...Finished XML Report 21:22:40 Generate HTML Report...Finished HTML Report 21:22:40 Generate HTML Report for all Ignored Jobs...Finished HTML Report 21:22:40 Generate Email Report 21:22:40 No mail will be Send since all Jobs are having status 'NOT_FOUND'. 21:22:40 Generate Email Report for all Ignored Jobs 21:22:40 No mail will be Send since all Jobs are having status 'NOT_FOUND'. 21:22:40 Finished Aggregate Test Results Action [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS
21:52:53 BUILD SUCCESSFUL in 3m 39s 21:52:53 93 actionable tasks: 78 executed, 15 up-to-date 21:52:53 + mkdir -p build/fingerprint 21:52:53 + echo Care_QA/qa-01-critical-response--build-all-88 21:52:53 + ls build 21:52:53 fingerprint 21:52:53 kotlin 21:52:53 reports 21:52:53 resources 21:52:53 test-results 21:52:53 version [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (archive and aggregate) [Pipeline] archiveArtifacts 21:52:53 Archiving artifacts 21:52:53 Recording fingerprints [Pipeline] junit 21:52:53 Recording test results 21:52:54 [Checks API] No suitable checks publisher found. [Pipeline] script [Pipeline] { [Pipeline] echo 21:52:54 Care_QA/qa-01-critical-response-AdminIntegrationsSuite [Pipeline] build (Building Care QA » qa-01-critical-response-AdminIntegrationsSuite) 21:52:54 Scheduling project: Care QA » qa-01-critical-response-AdminIntegrationsSuite 21:53:02 Starting building: Care QA » qa-01-critical-response-AdminIntegrationsSuite #45 [Pipeline] testResultsAggregator 21:55:03 Starting Aggregate Test Results Action 21:55:10 Not found results for Care_QA/qa-01-critical-response-AdminIntegrationsSuite 21:55:10 Analyze ...Finished 21:55:10 Generate XML Report...Finished XML Report 21:55:10 Generate HTML Report...Finished HTML Report 21:55:10 Generate HTML Report for all Ignored Jobs...Finished HTML Report 21:55:10 Generate Email Report 21:55:10 No mail will be Send since all Jobs are having status 'NOT_FOUND'. 21:55:10 Generate Email Report for all Ignored Jobs 21:55:10 No mail will be Send since all Jobs are having status 'NOT_FOUND'. 21:55:10 Finished Aggregate Test Results Action [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS
Hi one more question please, is the last build available?
or better is the URL myJenkinsbaseUrl /job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite/lastbuild/api/json?depth=1 available , its displaying data ?
Yes, This URL is giving the results: myJenkinsbaseUrl/job/Care_QA/view/response-critical-01/job/qa-01-critical-response-AdminIntegrationsSuite//lastBuild/api/json?depth=1
myJenkinsbaseURL/job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite/lastBuild/api/json
NOTE: we have the mater job and all downstream jobs linked to a view: response-critical-01.
Sorry i didn't have the time to check it , hopefully this weekend
socrates I tested today, but unfortunately we are getting the following error
21:18:17 Starting Aggregate Test Results Action 21:18:24 Collecting Data from Jenkins Job with name 'Care_QA/BNR/downstreamtest2' ...Finished with no data for last build url https://care-jenkins.?????.com/job/Care_QA/job/BNR/job/downstreamtest2/lastBuild/api/json?depth=1 has no data 21:18:24 Not found results for Care_QA/BNR/downstreamtest2 21:18:24 Analyze ...Finished
even though we are getting the results when I click the link:
https://care-jenkins.?????.com/job/Care_QA/job/BNR/job/downstreamtest2/lastBuild/api/json?depth=1
Not sure why its showing last build does not have data.
Yes, the new release now prints out the URL in case there are no data to read.
By the way in your last sample URL I am seeing that the job (downstreamtest2) is inside a folder (BNR) and this folder is in another one (Care_QA) , right?
Can I have a response from the JSON that you are seeing when you click the URL?
Please feel free to delete anything that is confidential and please send it to me via mail. Looks like there is something special there.
socrates yes This is my test job in Care_QA/BNR folder in Jenkins. Have sent you the JSON response of the downstream job:
https://care-jenkins./?????.com/job/Care_QA/job/BNR/job/downstreamtest2/lastBuild/api/json?depth=1
So the JSON has no tests, so there are no results to display..... Normally a Jenkins job that runs tests and publishes the results should have into "lastBuild/api/json?depth=1" for junits something like that ->
for testNG something like that
but there is nothing for the downstreamtest2 job, ( at least there is nothing in the Json you send me )
So is the job downstreamtest2 publishing its own results?
socrates Basically, we are running a gradle command to run the test suite and then archive the test results. I have sent you the details on the last email.
socrates I even removed the plugin version 1.2.14 and installed the same version where it was working i.e 1.2.11 but it is still showing the same result. not found results for the job.
V 2,0 has a new integration with https://github.com/jenkinsci/java-client-api , this should resolve the https issue
socrates After installing the latest plugin version, I am seeing a different error now:
14:41:45 Recording test results 14:41:45 [Checks API] No suitable checks publisher found. [Pipeline] script [Pipeline] { [Pipeline] build (Building criticaltestchild) 14:41:46 Scheduling project: criticaltestchild 14:41:56 Starting building: criticaltestchild #6 [Pipeline] testResultsAggregator 14:46:17 Starting Aggregate Test Results Action 14:46:17 Job 'criticaltestchild' found NOT_FOUND (0s) 14:46:17 Analyze ...Finished 14:46:17 Generate XML Report...Finished XML Report 14:46:18 Generate HTML Report...Finished HTML Report 14:46:18 Generate HTML Report for all Ignored Jobs...Finished HTML Report 14:46:18 Generate Email Report 14:46:18 No mail will be Send since the Recipients List is empty. 14:46:18 Generate Email Report for all Ignored Jobs 14:46:18 No mail will be Send since the Recipients List is empty. 14:46:18 Finished Aggregate Test Results Action [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // timestamps [Pipeline] } [Pipeline] // node [Pipeline] End of Pipeline Finished: SUCCESS
In the above test, I have one main job and only one downstream job. both the jobs have test results.
socrates Could you please provide some updates? This is currently a priority issue for us.
Hello, I am seeing the same error with latest plugin version. My aggregator job is not able to find the child jobs. But when I rolled back to previous version, it is working fine and my aggregator job is able to find other jobs in the same folder.
Job 'childjob' found NOT_FOUND (19ss)
Hi, i am seeing somewhat of the same issue. Recently upgraded jenkins alongside with the plugin.
Job 'FoldernameXXX/JobnameXXX' found build number 0 with statusJob 'FoldernameXXX/JobnameXXX' found NOT_FOUNDwith error : null (3s)
Test Result Aggregator Plugin v2.0
Jenkins 2.414.3
Note: i was unable to test the older version as i am not the jenkins admin. Just a user.
esidhu I was seeing the issue with the previous versions as well but the error was a little different. Since the previous version is working for you, I just needed to ask, Are you using the http or https url for the Jenkins? With the earlier version of the plugin installed, I am getting the following error:
Starting Aggregate Test Results Action 14:31:06 Collecting Data from Jenkins Job with name 'downstreamtest1' ...Finished with no data for last build url https://care-jenkins.########.com/job/downstreamtest1/lastBuild/api/json?depth=1 has no data
The latest release 2.1 is fixing a bug for jobs inside folders, so this version should fix the issues mentioned by syamil and esidhu
socrates We are not able to upgrade to version 2.1 as it needs Jenkins version above 2.375 which we are currently using. It's not possible for us to upgrade to the above Jenkins as it will break the whole Jenkins. Please suggest.
The next latest release is fixing another one issue for jenkins under https , its tested with several jenkins under https , if you can also check it I would appreciate , thanks in advance
socrates
I hope you are doing great! I had one ticket regarding the test-results-aggregator plugin opened a long time ago. We have recently upgraded our Jenkins to 2.440 version and installed the latest versions of plugin 2.2 and 2.1, but we are seeing a connection refused error while the aggregator plugin runs.
Below is the log from my test job.
[Pipeline] script [Pipeline] { [Pipeline] build (Building criticaltestchild) 13:28:24 Scheduling project: criticaltestchild 13:28:30 Starting building: criticaltestchild #23 13:28:35 Build criticaltestchild #23 completed: SUCCESS [Pipeline] copyArtifacts 13:28:35 Copied 1 artifact from "criticaltestchild" build number 23 [Pipeline] } [Pipeline] // script [Pipeline] testResultsAggregator 13:28:35 Starting Aggregate Test Results Action 13:28:37 Total Jenkins jobs found 14 and Aggregator has resolved successfully 1/1 13:28:37 Collect data 13:28:38 No last build details for job criticaltestchild Connect to care-jenkins.qa.######.com:443 [care-jenkins.qa.######.com/10.244.61.175] failed: Connection refused (Connection refused) 13:28:38 No last build details for job criticaltestchild Connect to care-jenkins.qa.######.com:443 [care-jenkins.qa.######.com/10.244.61.175] failed: Connection refused (Connection refused) 13:28:38 No last build details for job criticaltestchild Connect to care-jenkins.qa.######.com:443 [care-jenkins.qa.######.com/10.244.61.175] failed: Connection refused (Connection refused) 13:28:38 Job 'criticaltestchild' found NOT_FOUND with error : null (1s) 13:28:38 Collect data ...Finished 13:28:38 Analyze ...Finished 13:28:38 Generate XML Report...Finished XML Report 13:28:38 Generate HTML Report...Finished HTML Report 13:28:38 Generate HTML Report for all Ignored Jobs...Finished HTML Report 13:28:38 Generate Email Report 13:28:38 No mail will be Send since the Recipients List is empty. 13:28:38 Generate Email Report for all Ignored Jobs 13:28:38 No mail will be Send since the Recipients List is empty. 13:28:38 Finished Aggregate Test Results Action [Pipeline] } [Pipeline] // script [Pipeline] } [Pipeline] // stage [Pipeline] stage [Pipeline] { (Declarative: Post Actions) [Pipeline] archiveArtifacts 13:28:38 Archiving artifacts
Could you please help us in resolving the error. This is certainly the different error than the ones we have been seeing earlier. Have also configured the plugin correctly in the configure settings with the URL: http://jenkinsurl:8080 the connection is happening fine there. This is critical for us.
socrates I have been waiting for your response on this issue. It would greatly be appreciated if we could please get some help specifically on the error we are getting as mentioned above. I assume the plugin is not able to connect to the Jenkins master instance to fetch the test result of the downstream sample job, so the result is a connection error. But this is my guess.
socrates We have fixed the issue at our end causing Test Results Aggregator plugin to fail while fetching the test results. This ticket can be closed. Thank you for all of your help!
Hi,
In general a message like the
Not found results for Care_QA/qa-01-critical-response-AdminIntegrationsSuite
means that the plugin is not able to find the job...
Checking also the screenshot [Screenshot 2023-07-04 at 2.36.33 PM.png
] , it looks like 3 jobs are inside a folder named Care_QA , right?
if you open it in a browser the URL :
http://care-jenkins.????:8080/job/Care_QA/job/qa-01-critical-response-AdminIntegrationsSuite (please replace the Jenkins base URL with the correct one)
Is it accessible ?
... and if you place at the end of the URL the /api/json , do you see the JSON response?