-
Bug
-
Resolution: Duplicate
-
Critical
-
None
-
Jenkins 1.510 + gerrit 2.5.2 on Ubuntu 12.04. See attached systeminfo file for details as reported by Jenkins.
I just updated from Jenkins 1.505 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages.
My main page has quite a few jobs in the main view, but that didn't have an adverse effect before.
It did have some trouble connecting to the slave, right after the update, but that appears to be working properly now.
At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins.
Downgraded to 1.505 + git-client 1.03 until this is resolved or explained.
Note 1: After the downgrade things are back to being speedy. The dashboard loads quickly on opening it or on switching to it, and any CPU load is very brief (versus waiting several seconds for the page to display, or much longer if it was refreshed a few times). I hit reload several times in succession... it was reloaded in a second with no major load on the CPU (versus waiting around 30 seconds for the page to finally appear on the newer version).
Note 2: Even in 1.505 you end up with one Jenkins-bearing CPU that stubbornly spikes at 100% after a few quick reloads... and appears to stay that way (if you reload a few more times you still end up with this condition, though it might be on another CPU in the system). So the persistent CPU spike may or may not be related to this issue.
- duplicates
-
-
- Resolved
-
-
-
- Resolved
-
[JENKINS-17571] The main Jenkins dashboard page is slow to reload, seems to cause unusually high CPU load in the process
| Description |
Original:
I just updated from Jenkins 1.504 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages. My main page has quite a few jobs in the main view, but that didn't have an adverse effect before. It *did* have some trouble connecting to the slave, right after the update, but that appears to be working properly now. At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins. Downgrading to 1.504 until this is resolved or explained. |
New:
I just updated from Jenkins 1.504 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages. My main page has quite a few jobs in the main view, but that didn't have an adverse effect before. It *did* have some trouble connecting to the slave, right after the update, but that appears to be working properly now. At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins. Downgraded to 1.505 + git-client 1.03 until this is resolved or explained. Note: after the downgrade things are back to being speedy. Dashboard loads qickly on opening it or on switching to it, and any CPU load is very brief (versus waiting several seconds for the page to display, or much longer if it was refreshed a few times). I hit reload several times in succession... it was reloaded in a second with no major load on the CPU (versus waiting around 30 seconds for the page to finally appear on the newer version). Note2: even in 1.505 you end up with one Jenkins-bearing CPU that stubbornly spikes at 100% after a few quick reloads... and appears to stay that way. So that persistent spike may not be part of the same issue but is certainly interesting. |
| Description |
Original:
I just updated from Jenkins 1.504 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages. My main page has quite a few jobs in the main view, but that didn't have an adverse effect before. It *did* have some trouble connecting to the slave, right after the update, but that appears to be working properly now. At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins. Downgraded to 1.505 + git-client 1.03 until this is resolved or explained. Note: after the downgrade things are back to being speedy. Dashboard loads qickly on opening it or on switching to it, and any CPU load is very brief (versus waiting several seconds for the page to display, or much longer if it was refreshed a few times). I hit reload several times in succession... it was reloaded in a second with no major load on the CPU (versus waiting around 30 seconds for the page to finally appear on the newer version). Note2: even in 1.505 you end up with one Jenkins-bearing CPU that stubbornly spikes at 100% after a few quick reloads... and appears to stay that way. So that persistent spike may not be part of the same issue but is certainly interesting. |
New:
I just updated from Jenkins 1.504 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages. My main page has quite a few jobs in the main view, but that didn't have an adverse effect before. It *did* have some trouble connecting to the slave, right after the update, but that appears to be working properly now. At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins. Downgraded to 1.505 + git-client 1.03 until this is resolved or explained. Note 1: After the downgrade things are back to being speedy. The dashboard loads quickly on opening it or on switching to it, and any CPU load is very brief (versus waiting several seconds for the page to display, or much longer if it was refreshed a few times). I hit reload several times in succession... it was reloaded in a second with no major load on the CPU (versus waiting around 30 seconds for the page to finally appear on the newer version). Note 2: Even in 1.505 you end up with one Jenkins-bearing CPU that stubbornly spikes at 100% after a few quick reloads... and appears to stay that way (if you reload a few more times you still end up with this condition, though it might be on another CPU in the system). So the persistent CPU spike may or may not be related to this issue. |
| Description |
Original:
I just updated from Jenkins 1.504 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages. My main page has quite a few jobs in the main view, but that didn't have an adverse effect before. It *did* have some trouble connecting to the slave, right after the update, but that appears to be working properly now. At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins. Downgraded to 1.505 + git-client 1.03 until this is resolved or explained. Note 1: After the downgrade things are back to being speedy. The dashboard loads quickly on opening it or on switching to it, and any CPU load is very brief (versus waiting several seconds for the page to display, or much longer if it was refreshed a few times). I hit reload several times in succession... it was reloaded in a second with no major load on the CPU (versus waiting around 30 seconds for the page to finally appear on the newer version). Note 2: Even in 1.505 you end up with one Jenkins-bearing CPU that stubbornly spikes at 100% after a few quick reloads... and appears to stay that way (if you reload a few more times you still end up with this condition, though it might be on another CPU in the system). So the persistent CPU spike may or may not be related to this issue. |
New:
I just updated from Jenkins 1.505 today as part of resolving another outstanding issue. I notice that the main dashboard page is much slower to load than it was before, and also notice that the server gets high CPU load in the process (it gets super-bad if you try and reload it a few times, as one might do when a page seems frozen). This is not the case on the individual job pages, the "manage Jenkins" page or other such ancillary pages. My main page has quite a few jobs in the main view, but that didn't have an adverse effect before. It *did* have some trouble connecting to the slave, right after the update, but that appears to be working properly now. At the moment, several minutes after it seemed to have normalized a bit, I see one CPU is now completely spiked by Jenkins. Downgraded to 1.505 + git-client 1.03 until this is resolved or explained. Note 1: After the downgrade things are back to being speedy. The dashboard loads quickly on opening it or on switching to it, and any CPU load is very brief (versus waiting several seconds for the page to display, or much longer if it was refreshed a few times). I hit reload several times in succession... it was reloaded in a second with no major load on the CPU (versus waiting around 30 seconds for the page to finally appear on the newer version). Note 2: Even in 1.505 you end up with one Jenkins-bearing CPU that stubbornly spikes at 100% after a few quick reloads... and appears to stay that way (if you reload a few more times you still end up with this condition, though it might be on another CPU in the system). So the persistent CPU spike may or may not be related to this issue. |
dave smolin
added a comment - I just updated jenkins from 1.505 to 1.511 and immediately encountered this issue. Jenkins in this state is unusable, and we will be downgrading back to 1.505. We're using jenkins as a native package on ubuntu 10.10 on a beefy box.
dave smolin
added a comment - I just updated jenkins from 1.505 to 1.511 and immediately encountered this issue. Jenkins in this state is unusable, and we will be downgrading back to 1.505. We're using jenkins as a native package on ubuntu 10.10 on a beefy box. 
Justin Slattery
added a comment - Another confirmation here. We upgraded to 1.511 and immediately encountered the issue. I lost more time than I care to admit troubleshooting. Downgrading to 1.505 fixed it right away.
Justin Slattery
added a comment - Another confirmation here. We upgraded to 1.511 and immediately encountered the issue. I lost more time than I care to admit troubleshooting. Downgrading to 1.505 fixed it right away. To anyone suffering from this: it is impossible to even begin diagnosis without a thread dump (e.g. /threadDump in the web UI, or jstack from a shell on the master).
Forme this issue duplicates this one JENKINS-15858 which is related to JENKINS-15929
| Link |
New:
This issue duplicates |
On that persistent CPU thing: several full refreshes of the Jenkins dashboard page (shift-ctrl-R in Windows) is quite effective at spiking not just one but several CPUs at once. I'm curious if this is a known issue or how to track down where the problem is.
As relates to this present 1.510 issue, even with almost all the cores maxed out because of this weird persistent CPU thing, 1.505 still loads and presents the dashboard quickly. 1.510 would just get worse and worse the more you hammered it.
Now, on the prudence of actually refreshing this much: At this point I'm testing the boundaries of this issue. Obviously this isn't a great thing to do, but the tool shouldn't have persistent adverse side-effects because of it, and performance completely tanks in 1.510 if you do this even a little (it was the lag on switching back to the dashboard view that caused me to refresh and trigger this in the first place, and to notice how much extra time even a couple of loads/refreshes added to an already extended load time).
A few users accessing the dashboard at once would have a similar effect in terms of simultaneous load, with no doubt similar unpleasant results.