-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Jenkins 2.89.4
The hudson.security.csrf.CrumbFilter generates so many log entries it causes parts of Jenkins to stall until the rate of log messages slows down.
2018-02-24 05:17:10.406+0000 [id=20011] WARNING hudson.security.csrf.CrumbFilter#doFilter: Found invalid crumb 418axxxx20cb74b577eaae393aa8ac0e. Will check remaining parameters for a valid one... 2018-02-24 05:17:10.406+0000 [id=20011] WARNING hudson.security.csrf.CrumbFilter#doFilter: No valid crumb was included in request for /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getOnlineSlaves by joecool. Returning 403.
The amount of these logs was causing my Jenkins to stop working: The executors were not being released by jobs (even after they were done running) until the log entry could be written.
I checked through the logs and all the entries I have are for these URLs (there could be more, due to the logs rolling so quick):
- /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getDisconnectedSlaves
- /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getOfflineSlaves
- /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getOnlineSlaves
- /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getRunningJobs
- /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getSlaves
- /$stapler/bound/419618ba-22aa-4afb-8528-b112a604cce9/getTasksInQueue
I would suggest that either these logs shouldn't be generated at all, or only as FINE or FINER entries.
- relates to
-
-
- Resolved
-
[JENKINS-49745] Too many CrumbFilter log entries per second
| Labels | New: newbie-friendly |
There were some fixes in the recent core for not showing these warning for requests coming from old open browser pages after restart in 2.82: JENKINS-40344 . Maybe the fix was not complete. Seems docwhat has a discussion there as well. Also CC danielbeck
| Link |
New:
This issue relates to |
| Labels | Original: newbie-friendly |
| Component/s | New: dashboard-view-plugin [ 15679 ] | |
| Component/s | Original: core [ 15593 ] |
I consider this to be a bug in Dashboard View Plugin that seems to provide these URLs and doesn't seem to handle 403 well (https://github.com/jenkinsci/dashboard-view-plugin/blob/392aaa7c4631a459ce7af801a66ee0fdd09cab08/src/main/java/hudson/plugins/view/dashboard/stats/StatSlaves.java).
https://github.com/jenkinsci/dashboard-view-plugin/blob/392aaa7c4631a459ce7af801a66ee0fdd09cab08/src/main/resources/hudson/plugins/view/dashboard/stats/StatSlaves/statslaves.jelly#L32...L47 seems to be overly simplistic.
I'm glad my extra research helped identify the culprit (dashboard-view).
I'm not familiar enough with Jelly to know exactly what's going on but it looks like some of the problems are:
- It doesn't check the return value (or try/catch if done that way) in case there is an error (e.g. a 403). This means there is nothing to bounce the page to the login page (if it requires auth) or refetch a CSRF token.
- It fetches everything, even if only some items are needed.
- This stuff, if everything is needed, could be returned as a single struct.
- Did I miss anything? I'm not familiar with how the CSRF stuff is implemented in Jenkins.
It also seems wrong that if the permission for Jenkins are such that anyone (anonymous) can view a URL, then GET requests for that URL shouldn't need a CSRF token and should never raise a 403.
CC wfollonier