We have several projects which run TAP tests. One such project is getting hundreds of new tests each week, and we're finding Jenkins is getting slower and slower each time we add some new tests.

It seems that the TAP plugin reads a lot of files and their contents every time you click around inside a project (it may be due to the trend chart, or to report "Latest test result" on the project home page - or both, or something else entirely). I did an 'strace' of the Java process, and found lots of this sort of thing:

[pid 9323] read(155, " <org.tap4j.model.TestResult re"..., 8192) = 8192
[pid 9323] fstat(155,

{st_mode=S_IFREG|0644, st_size=17020120, ...}) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15171584
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15171584, SEEK_SET) = 15171584
[pid 9323] read(155, "<org.tap4j.model.TestResult refe"..., 8192) = 8192
[pid 9323] fstat(155, {st_mode=S_IFREG|0644, st_size=17020120, ...}

) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15179776
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15179776, SEEK_SET) = 15179776
[pid 9323] read(155, "ce=\"../../tapLines/org.tap4j.mod"..., 8192) = 8192
[pid 9323] fstat(155,

{st_mode=S_IFREG|0644, st_size=17020120, ...}) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15187968
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15187968, SEEK_SET) = 15187968
[pid 9323] read(155, ".tap4j.model.Comment[2]\"/>\n "..., 8192) = 8192
[pid 9323] fstat(155, {st_mode=S_IFREG|0644, st_size=17020120, ...}

) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15196160
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15196160, SEEK_SET) = 15196160
[pid 9323] read(155, "mment reference=\"../../tapLines/"..., 8192) = 8192
[pid 9323] fstat(155,

{st_mode=S_IFREG|0644, st_size=17020120, ...}) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15204352
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15204352, SEEK_SET) = 15204352
[pid 9323] read(155, "stResult[106]/comments/org.tap4j"..., 8192) = 8192
[pid 9323] fstat(155, {st_mode=S_IFREG|0644, st_size=17020120, ...}

) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15212544
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15212544, SEEK_SET) = 15212544
[pid 9323] read(155, "Result[136]/comments/org.tap4j.m"..., 8192) = 8192
[pid 9323] fstat(155,

{st_mode=S_IFREG|0644, st_size=17020120, ...}

) = 0
[pid 9323] lseek(155, 0, SEEK_CUR) = 15220736
[pid 9323] lseek(155, 0, SEEK_END) = 17020120
[pid 9323] lseek(155, 15220736, SEEK_SET) = 15220736

(occasionally, I see a bit of text that looks like the TAP test text in there too).

The good news is that when Jenkins is working on a slow request, it doesn't seem to affect the rest of the server - clicking about in other projects seems to work fine (maybe a bit slower than usual, but acceptable).

We now have >10000 tests in one particular project, and Jenkins pretty much never responds to clicks. We've had to remove blocks of tests that we think work so that Jenkins runs properly again. Disabling the TAP plugin also works, although means we can't see our test results

What can we do about this? What can I/we do to help find a solution?