Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59308

Cobertura Code coverage all reports are showing in similar

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Blocker Blocker
    • cobertura-plugin
    • None

      Hi ,  I have created cobertura plugin using code cover reports for my unit test stages (I have  parallel running 04 unit-test stages in my pipeline)  but my issue is I'm getting all code-coverage test results as similarly in all 04 code-coverage reports.   Please see the below attachment of my report results. 

       

       So as you can see where I highlighted in red all 04 reports have same percentage of results.  What will be the problem causing this?   If you want further information I will give you.  If you want my declarative pipeline syntax to look at? 

          [JENKINS-59308] Cobertura Code coverage all reports are showing in similar

          Sure I saw you were doing 4 of them, that's what triggers the problem in the first place

           

          So the:

          step([$class: 'CoberturaPublisher', coberturaReportFile: '**/build/Tests/**/Cobertura.xml'])

          Is correct as it will search for all of them, since the second asterisk-asterisk pattern will iterate through all of them.

           

          Of course if you prefer you can also be more explicit with:

          step([$class: 'CoberturaPublisher', coberturaReportFile: '**/build/Tests/NunitFast/Cobertura.xml,**/build/Tests/NUnitTestsUpdateCws/Cobertura.xml,**/build/Tests/NUnitTestsSlow/Cobertura.xml,**/build/Tests/PerformanceTests/Cobertura.xml'])

           

          Federico Pellegrin added a comment - Sure I saw you were doing 4 of them, that's what triggers the problem in the first place   So the: step([$class: 'CoberturaPublisher', coberturaReportFile: '**/build/Tests/**/Cobertura.xml']) Is correct as it will search for all of them, since the second asterisk-asterisk pattern will iterate through all of them.   Of course if you prefer you can also be more explicit with: step([$class: 'CoberturaPublisher', coberturaReportFile: '**/build/Tests/NunitFast/Cobertura.xml,**/build/Tests/NUnitTestsUpdateCws/Cobertura.xml,**/build/Tests/NUnitTestsSlow/Cobertura.xml,**/build/Tests/PerformanceTests/Cobertura.xml'])  

          Amila Gunathilake added a comment - - edited

          fedepell  Hi mate how are you?

          I have tested this your second solution, by using code-coverage-api-plugin I able to generate 04 different reports. 

          So I renamed my Cobertura.xml file by 04 different names in each stage's Post build section in my Jenkins file.  Like below, 

          publishCoverage adapters: [coberturaAdapter(path: '**/build/Tests/NunitFast/NUnitTestsSlow_Test_Report.xml')]

           

          *  Below is the my output:

           

           

          It's really better output than my previous reports.  Thank you for that.  So do you know how to rename the Title of the reports ?  Because it showing my .xml file name as the report heading.

          eg:

           

           

           

          Amila Gunathilake added a comment - - edited fedepell   Hi mate how are you? I have tested this your second solution, by using  code-coverage-api-plugin  I able to generate 04 different reports.  So I renamed my Cobertura.xml file by 04 different names in each stage's Post build section in my Jenkins file.  Like below,  publishCoverage adapters: [coberturaAdapter(path: '**/build/Tests/NunitFast/NUnitTestsSlow_Test_Report.xml' )]   *  Below is the my output:     It's really better output than my previous reports.  Thank you for that.  So do you know how to rename the Title of the reports ?  Because it showing my .xml file name as the report heading. eg:      

          As of now there is no possibility to customize that name label, it will just pick the filename (with some minor filtering of characters), sorry!

          Federico Pellegrin added a comment - As of now there is no possibility to customize that name label, it will just pick the filename (with some minor filtering of characters), sorry!

          Ulli Hafner added a comment - - edited

          amila_devops You can create a feature request in Jira for the code coverage plugin. Maybe they can reuse the same concepts as the warnings plugin: using an id and name property.

          See JENKINS-51367.

          Ulli Hafner added a comment - - edited amila_devops You can create a feature request in Jira for the code coverage plugin. Maybe they can reuse the same concepts as the warnings plugin: using an id and name property. See JENKINS-51367 .

          hi fedepell it's okay no worries.  And one thing do you guys have any Tutorial or any Doc with information to operate these reports and how to read these reports ? 

          Thanks again.

          Amila Gunathilake added a comment - hi fedepell it's okay no worries.  And one thing do you guys have any Tutorial or any Doc with information to operate these reports and how to read these reports ?  Thanks again.

          Amila Gunathilake added a comment - - edited

          FYI fedepell, stephenconnolly 

          Hi fedepell hope you doing good?

          I just want one thing to make sure. I wanted to exclude some dlls (eg: dapper.dll) from including to this code-coverage reports; Such as in my source code their are dlls I don't want to use for use as packages for Code-Coverage reports. So when generating my unit tests Code-Coverage reports how can I exclude those dll files?

          Is their any code to exclude dll files in here?  Such as when I running my RunTests.ps1 script for unit testing can I add some code to exclude dapper.dll file in below code ?  This example is not worked. .

             Existing Code:

          .$Opencover\OpenCover.Console.exe -target:"$NUNIT\nunit3-console.exe" -targetargs:".\Tests\NunitFast\NUnitTestsFast\$CONFIGURATION\net47\ServiceTestFast.dll --result=.\Tests\NunitFast\upstream-corewebservice-nunit-result.xml" -output:".\Tests\NunitFast\opencover-upstream-corewebservice-nunit-result.xml" -register:"user" 	
          

           

           

          example: Code I want for Exclude dapper.dll:

          .$Opencover\OpenCover.Console.exe -target:"$NUNIT\nunit3-console.exe" -targetargs:".\Tests\NunitFast\NUnitTestsFast\$CONFIGURATION\net47\ServiceTestFast.dll --exclude dapper.dll --result=.\Tests\NunitFast\upstream-corewebservice-nunit-result.xml" -output:".\Tests\NunitFast\opencover-upstream-corewebservice-nunit-result.xml" -register:"user" 	
          

          Amila Gunathilake added a comment - - edited FYI  fedepell , stephenconnolly   Hi fedepell hope you doing good? I just want one thing to make sure. I wanted to exclude some dlls (eg: dapper.dll) from including to this code-coverage reports; Such as in my source code their are dlls I don't want to use for use as packages for Code-Coverage reports. So when generating my unit tests Code-Coverage reports how can I exclude those dll files? Is their any code to exclude dll files in here?  Such as when I running my RunTests.ps1 script for unit testing can I add some code to exclude dapper.dll file in below code ?  This example is not worked.  .    Existing Code: .$Opencover\OpenCover.Console.exe -target: "$NUNIT\nunit3-console.exe" -targetargs: ".\Tests\NunitFast\NUnitTestsFast\$CONFIGURATION\net47\ServiceTestFast.dll --result=.\Tests\NunitFast\upstream-corewebservice-nunit-result.xml" -output: ".\Tests\NunitFast\opencover-upstream-corewebservice-nunit-result.xml" -register: "user"     example: Code I want for Exclude dapper.dll: .$Opencover\OpenCover.Console.exe -target: "$NUNIT\nunit3-console.exe" -targetargs: ".\Tests\NunitFast\NUnitTestsFast\$CONFIGURATION\net47\ServiceTestFast.dll --exclude dapper.dll --result=.\Tests\NunitFast\upstream-corewebservice-nunit-result.xml" -output: ".\Tests\NunitFast\opencover-upstream-corewebservice-nunit-result.xml" -register: "user"

          Federico Pellegrin added a comment - - edited

          What you are trying to do is to tell your tool (OpenCover ?) to filter them and that totally depends on your tool. I sincerely don't know much about OpenCover and never used it, but looking at the usage page I don't see an

          --exclude

          option as by your example but see some

          --excludebyfile

          . Maybe that is what you are looking for!

           

          On the other side, on Jenkins coverage plugin side, such a filtering feature right now is not present. You may in case try to request it at the plugin page, but given that usually the tools generating the reports can do this it is maybe not something very interesting for the plugin itself.

          Federico Pellegrin added a comment - - edited What you are trying to do is to tell your tool (OpenCover ?) to filter them and that totally depends on your tool. I sincerely don't know much about OpenCover and never used it, but looking at the usage page I don't see an --exclude option as by your example but see some --excludebyfile . Maybe that is what you are looking for!   On the other side, on Jenkins coverage plugin side, such a filtering feature right now is not present. You may in case try to request it at the plugin page, but given that usually the tools generating the reports can do this it is maybe not something very interesting for the plugin itself.

          hi fedepell  actually here what I want is exclude this Dapper Packages (which is Dapper.dll)  from generating my  NunitFast Code coverage reports.  eg:

          So I changed my code as you suggested.  But after I include  -excludebyfile or -filter like below to exclude Dapper packages I'm getting zero Code coverage output in NunitFast unit test reports.

          my new code is below:

          .$Opencover\OpenCover.Console.exe -target:"$NUNIT\nunit3-console.exe" -targetargs:".\Tests\NunitFast\NUnitTestsFast\$CONFIGURATION\net47\ServiceTestFast.dll -excludebyfile:-[Dapper*] --result=.\Tests\NunitFast\upstream-corewebservice-nunit-result.xml" -output:".\Tests\NunitFast\opencover-upstream-corewebservice-nunit-result.xml" -register:"user"
          

          even I tried with -filter option as well but I'm still get the below results which is zero output, I think these filtering options are only for exclude class or methods, but here we going to exclude an entire package (Dapper.dll),  so is that why I'm getting below crazy out-put ? 

          Could you suggest something new fedepell ?

          Thanks  ! 

          Amila Gunathilake added a comment - hi fedepell   actually here what I want is exclude this Dapper Packages (which is Dapper.dll)  from generating my  NunitFast Code coverage reports.  eg: So I changed my code as you suggested.  But after I include  -excludebyfile or -filter like below to exclude Dapper packages I'm getting zero Code coverage output in NunitFast unit test reports. my new code is below: .$Opencover\OpenCover.Console.exe -target: "$NUNIT\nunit3-console.exe" -targetargs: ".\Tests\NunitFast\NUnitTestsFast\$CONFIGURATION\net47\ServiceTestFast.dll -excludebyfile:-[Dapper*] --result=.\Tests\NunitFast\upstream-corewebservice-nunit-result.xml" -output: ".\Tests\NunitFast\opencover-upstream-corewebservice-nunit-result.xml" -register: "user" even I tried with -filter option as well but I'm still get the below results which is zero output, I think these filtering options are only for exclude class or methods, but here we going to exclude an entire package (Dapper.dll),  so is that why I'm getting below crazy out-put ?  Could you suggest something new fedepell ? Thanks  ! 

          hi guys any idea about my above last comment ?  I just want to exclude Dapper dll/package  from the report.  So far its not working and when I exclude as you suggested it shows no any data in the report. 

          fedepell, stephenconnolly  redsoloabayer   Please help me on this. I got no reply since months.

          Thanks
          Amila
          Jenkins Customer

          Amila Gunathilake added a comment - hi guys any idea about my above last comment ?  I just want to exclude Dapper dll/package  from the report.  So far its not working and when I exclude as you suggested it shows no any data in the report.  fedepell , stephenconnolly   redsolo ,  abayer    Please help me on this. I got no reply since months. Thanks Amila Jenkins Customer

          Ulli Hafner added a comment -

          You are asking the wrong persons, please try to get help from the NUnit team. Jenkins just reports the results: you need to configure your coverage tool accordingly. (And an issue tracker probably is the wrong place for a discussion about tool configurations as well)

          Ulli Hafner added a comment - You are asking the wrong persons, please try to get help from the NUnit team. Jenkins just reports the results: you need to configure your coverage tool accordingly. (And an issue tracker probably is the wrong place for a discussion about tool configurations as well)

            fedepell Federico Pellegrin
            amila_devops Amila Gunathilake
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: