Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-61107

Excessive querying of change information

    XMLWordPrintable

Details

    Description

      While debugging JENKINS-60833 I also noticed an extreme number of identical requests was made to the Gerrit server for each build. It turns out that for each sh step Gerrit Code Review Plugin makes four requests to the same url:

      /gerrit/a/changes/123456?o=LABELS&o=DETAILED_LABELS&o=CURRENT_REVISION&o=ALL_REVISIONS&o=CURRENT_COMMIT&o=ALL_COMMITS&o=CURRENT_FILES&o=ALL_FILES&o=DETAILED_ACCOUNTS&o=MESSAGES&o=CURRENT_ACTIONS&o=REVIEWED&o=DRAFT_COMMENTS&o=DOWNLOAD_COMMANDS&o=WEB_LINKS&o=CHANGE_ACTIONS&o=COMMIT_FOOTERS&o=PUSH_CERTIFICATES&o=REVIEWER_UPDATES&o=SUBMITTABLE&o=TRACKING_IDS&o=SKIP_MERGEABLE
      

      So, in a Jenkinsfile with a handfull sh steps this quickly becomes quite a lot of requests. E.g. the following will make 81 identical requests to the Gerrit server:

      node {
          for (int i = 0; i < 20; i++) {
              sh "echo Loop ${i}"
          }
      }
      

      The root cause of this is that GerritEnvironmentContributor is called repeatedly each time the Pipeline execution needs to execute long duration steps (or something like that). Worth noting in the documentation for EnvironmentContributor#buildEnvironmentFor(Job, ...):

      This method can be called repeatedly for the same {@link Job}, thus the computation of this method needs to be efficient.

      ... which (in my opinion) doesn't play well with the repeated HTTP requests to the Gerrit server.

      One possible fix is to instead implement EnvironmentContributor#buildEnvironmentFor(Run, ...) and then compute once and store as InvisibleAction (as per JavaDoc comments for the method). Now, the question is whether there is a special reason that the Job version is implemented instead of Run version? lucamilanesio, maybe you have som insights to why this is? If not it should be a fairly easy fix.

      Attachments

        Activity

          jons Jon Sten created issue -
          jons Jon Sten made changes -
          Field Original Value New Value
          Description While debugging JENKINS-60833 I also noticed an extreme number of identical requests was made to the Gerrit server for each build. It turns out that for each {{sh}} step Gerrit Code Review Plugin makes four requests to the same url:

          {noformat}
          /gerrit/a/changes/123456?o=LABELS&o=DETAILED_LABELS&o=CURRENT_REVISION&o=ALL_REVISIONS&o=CURRENT_COMMIT&o=ALL_COMMITS&o=CURRENT_FILES&o=ALL_FILES&o=DETAILED_ACCOUNTS&o=MESSAGES&o=CURRENT_ACTIONS&o=REVIEWED&o=DRAFT_COMMENTS&o=DOWNLOAD_COMMANDS&o=WEB_LINKS&o=CHANGE_ACTIONS&o=COMMIT_FOOTERS&o=PUSH_CERTIFICATES&o=REVIEWER_UPDATES&o=SUBMITTABLE&o=TRACKING_IDS&o=SKIP_MERGEABLE
          {noformat}

          So, in a Jenkinsfile with a handfull sh steps this quickly becomes quite a lot of requests. E.g. the following will make 81 identical requests to the Gerrit server:

          {noformat}
          node {
              for (int i = 0; i < 20; i++) {
                  sh "echo Loop ${i}"
              }
          }
          {noformat}

          The root cause of this is that {{GerritEnvironmentContributor}} is called repeatedly each time the Pipeline execution needs to execute long duration steps (or something like that). Worth noting in the documentation for {{EnvironmentContributor#buildEnvironmentFor(Job, ...)}}:

          bq. This method can be called repeatedly for the same {@link Job}, thus the computation of this method needs to be efficient.

          ... which (in my opinion) doesn't play well with the repeated HTTP requests to the Gerrit server.

          One possible fix is to instead implement {{EnvironmentContributor#buildEnvironmentFor(Run, ...)}} and then compute once and store as {{InvisibleAction}} (as per JavaDoc comments for the method). Now, the question is whether there is a special reason that the Job version is implemented instead of Run version? If not it should be a fairly easy fix.
          While debugging JENKINS-60833 I also noticed an extreme number of identical requests was made to the Gerrit server for each build. It turns out that for each {{sh}} step Gerrit Code Review Plugin makes four requests to the same url:

          {noformat}
          /gerrit/a/changes/123456?o=LABELS&o=DETAILED_LABELS&o=CURRENT_REVISION&o=ALL_REVISIONS&o=CURRENT_COMMIT&o=ALL_COMMITS&o=CURRENT_FILES&o=ALL_FILES&o=DETAILED_ACCOUNTS&o=MESSAGES&o=CURRENT_ACTIONS&o=REVIEWED&o=DRAFT_COMMENTS&o=DOWNLOAD_COMMANDS&o=WEB_LINKS&o=CHANGE_ACTIONS&o=COMMIT_FOOTERS&o=PUSH_CERTIFICATES&o=REVIEWER_UPDATES&o=SUBMITTABLE&o=TRACKING_IDS&o=SKIP_MERGEABLE
          {noformat}

          So, in a Jenkinsfile with a handfull sh steps this quickly becomes quite a lot of requests. E.g. the following will make 81 identical requests to the Gerrit server:

          {noformat}
          node {
              for (int i = 0; i < 20; i++) {
                  sh "echo Loop ${i}"
              }
          }
          {noformat}

          The root cause of this is that {{GerritEnvironmentContributor}} is called repeatedly each time the Pipeline execution needs to execute long duration steps (or something like that). Worth noting in the documentation for {{EnvironmentContributor#buildEnvironmentFor(Job, ...)}}:

          bq. This method can be called repeatedly for the same {@link Job}, thus the computation of this method needs to be efficient.

          ... which (in my opinion) doesn't play well with the repeated HTTP requests to the Gerrit server.

          One possible fix is to instead implement {{EnvironmentContributor#buildEnvironmentFor(Run, ...)}} and then compute once and store as {{InvisibleAction}} (as per JavaDoc comments for the method). Now, the question is whether there is a special reason that the Job version is implemented instead of Run version? [~lucamilanesio], maybe you have som insights to why this is? If not it should be a fairly easy fix.

          jons thanks for the suggestion: let me try to use the `Run` version. of the EnvironmentContributor.

          lucamilanesio Luca Domenico Milanesio added a comment - jons thanks for the suggestion: let me try to use the `Run` version. of the EnvironmentContributor.
          lucamilanesio Luca Domenico Milanesio made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          lucamilanesio Luca Domenico Milanesio added a comment - The fix is ready for review at:  https://review.gerrithub.io/c/jenkinsci/gerrit-code-review-plugin/+/499652
          lucamilanesio Luca Domenico Milanesio made changes -
          Status In Progress [ 3 ] In Review [ 10005 ]

          People

            lucamilanesio Luca Domenico Milanesio
            jons Jon Sten
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: