Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-39084

PROD: cbwf-controller component is causing Jenkins's CPU to spike

      Within the Jenkin Pipeline View, any build that contains

      cbwf-controller

      , is causing a HTTP GET request to https://example.com/job/project/job/BRANCH_NAME/3/wfapi/changesets?_=1476824899808.

      Each branch of a project on initial load will make a request to that endpoint. That will then cause the Jenkins process spike because it is trying to fulfill all of the requests.
      An example of the element which has that controller is

      <div class="changeset-box cbwf-widget cbwf-controller-applied run-changesets" cbwf-controller="run-changesets" objecturl="/job/ssw/job/theater-client/job/SSW-416/4/wfapi/changesets"><div><div class="run-changesets">
      

      Build #10 will cause a GET request while build #9 wouldn't

          [JENKINS-39084] PROD: cbwf-controller component is causing Jenkins's CPU to spike

          Sam Van Oort added a comment -

          dchau The cause is a Jenkins core change to fix SECURITY-243, which makes user lookup extremely expensive – this is now fixed, please see JENKINS-35493 and upgrade to Jenkins 2.7.3 or later.

          There's a workaround for people not yet able to update, and documented on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Stage+View+Plugin#PipelineStageViewPlugin-ConfigurableProperties%3A

          Set property com.cloudbees.workflow.rest.external.ChangeSetExt.resolveCommitAuthors to "false" to disable user lookup for changesets. This can be set at runtime to test the impact and should make your problem go away nearly immediately.

          Sam Van Oort added a comment - dchau The cause is a Jenkins core change to fix SECURITY-243, which makes user lookup extremely expensive – this is now fixed, please see JENKINS-35493 and upgrade to Jenkins 2.7.3 or later. There's a workaround for people not yet able to update, and documented on the wiki: https://wiki.jenkins-ci.org/display/JENKINS/Pipeline+Stage+View+Plugin#PipelineStageViewPlugin-ConfigurableProperties%3A Set property com.cloudbees.workflow.rest.external.ChangeSetExt.resolveCommitAuthors to "false" to disable user lookup for changesets. This can be set at runtime to test the impact and should make your problem go away nearly immediately.

          Sam Van Oort added a comment -

          Closing as duplicate of existing issue with documented workaround and sustainable fix.

          Sam Van Oort added a comment - Closing as duplicate of existing issue with documented workaround and sustainable fix.

          David Chau added a comment -

          The work around fixes the error but we are in Jenkins Version 2.7.4

          David Chau added a comment - The work around fixes the error but we are in Jenkins Version 2.7.4

          Sam Van Oort added a comment -

          davidschou Which would suggest that user lookup is still proving to be a bottleneck in your setup (haven't seen that happen before with the UserDetails cache on though). That's tied to your security realm – setting hudson.model.User.SECURITY_243_FULL_DEFENSE=false may also resolve the issue (and would suggest the details cache needs more work).

          Unfortunately there's not really anything to be done about this from the stage view end of things, aside from using one of the overrides in your Jenkins config file to make it permanent: the problematic API is deeply embedded in Jenkins core, and the override I put in here simply avoids invoking it. No rabbits to pull out from my hat this time, I'm afraid.

          Sam Van Oort added a comment - davidschou Which would suggest that user lookup is still proving to be a bottleneck in your setup (haven't seen that happen before with the UserDetails cache on though). That's tied to your security realm – setting hudson.model.User.SECURITY_243_FULL_DEFENSE=false may also resolve the issue (and would suggest the details cache needs more work). Unfortunately there's not really anything to be done about this from the stage view end of things, aside from using one of the overrides in your Jenkins config file to make it permanent: the problematic API is deeply embedded in Jenkins core, and the override I put in here simply avoids invoking it. No rabbits to pull out from my hat this time, I'm afraid.

            svanoort Sam Van Oort
            dchau David Chau
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: