The Jenkins TAP plugin is awesome, but could be designed to scale a bit better. As an example:

My client's test-suite has ~60,000 test cases in ~1,300 test scripts. After running the test suite with a Shell build step that ultimately runs:

prove --archive $WORKSPACE/tap-output t

And passing that on to Jenkins TAP plugin, we see:

• $WORKSPACE/tap-output is 172M
• The last build dir is 789M. Breakdown:

  452M	build.xml
  4.0K	changelog.xml
  167M	log
  8.0K	revision.txt
  171M	tap-master-files
  

• The job's summary pages all load quickly, e.g.:
  • http://jenkins/job/ProjectName
  • http://jenkins/job/ProjectName/36/
• The last build's 'tapResults' page: http://jenkins/job/ProjectName/36/tapResults/
  • load time: 46s
  • size: 16.2 MB
• The last build's 'tapTestReport' page: http://jenkins/job/ProjectName/36/tapTestReport/
  • load time: 53s
  • size: 29.1 MB

Its difficult to navigate both tapResults & tapTestReport pages because they are so big.

I'd suggest setting these goals:

• minimise on-disk storage of TAP output & parsed TAP results
• minimise load-time of TAP Results & TAP Test Report pages
• enhance usability of these pages

Here are a few ideas:

• Don't pre-render TAP results
  • At a glance, I don't think the Jenkins TAP plugin is, but included just in case
  • Avoids: large files on disk of pre-rendered content
• Render results client-side
  • Introduce server-side APIs for: test-set summaries, test-set details, original TAP output
  • Render test-set summaries by default
  • Only render test-set details on request
    • Avoids: a high up-front load time for large result sets
  • Gracefully degrade to individual pages
• Introduce a filter for failed tests
• Stop storing parsed TAP results in build.xml
  • Move to an sqlite db?
  • At least: remove extra whitespace
• Compress everything in tap-master-files
  • deliver it to client compressed to minimise over-the-wire transfer size?

See also:

• JENKINS-13451 - I ran into a similar scaling issue with TAP::Formatter::HTML, though different because of the nature of the tool - static output was a requirement. Still, some ideas such as 'minimise size of html & CSS used in report', and 'remove unnecessary whitespace' may apply here.