Channel already keeps some rudimentary statistics about class loading (classLoadingCount and so on), which can give insight into slow initial builds. But there are many cases where we suspect that a performance or reliability problem is ultimately due to excessive traffic on the channel, yet cannot prove this except via the crude tool of a network traffic analyzer, and have no visibility into which features (plugins) are responsible and whether the traffic is really necessary.

      Propose that we add some more statistics (counts, mean, median, standard deviation) maintained per Channel for each of the three types of Request/Response.

      • For ProxyInputStream/ProxyOutputStream, record number of pipes opened and closed in each direction, and bytes sent on them. Ideally we would like to get some information about what code opened the stream, though I am not sure how to aggregate this well, since stack traces are too variable.
      • For RPCRequest, record the implementation class name of the proxied object, perhaps the method name/signature being called, the number of calls, their duration, the byte size of the request, the byte size of the response.
      • For UserRequest, record the implementation class name of the Callable (or delegate of a FileCallableWrapper), number of calls, duration, byte sizes of request and response.

      These statistics could be reported in various places:

      • Slave system information page.
      • support-core bundles. (Of course aggregate data across channels should be recorded, so it is still available after slave disconnections.)
      • metrics data.

          [JENKINS-27035] Request/response statistics

          Jesse Glick added a comment -

          Wireshark plugin or other detailed diagnostics would be good too.

          Jesse Glick added a comment - Wireshark plugin or other detailed diagnostics would be good too.

          Oleg Nenashev added a comment -

          +1 for the Wireshark plugin, but we cannot easily do it with Encrypted protocols like JNLP4. BTW we still can extract some data

          Oleg Nenashev added a comment - +1 for the Wireshark plugin, but we cannot easily do it with Encrypted protocols like JNLP4. BTW we still can extract some data

          Jesse Glick added a comment -

          Implemented raw event gathering infrastructure; initially the information is only available as a logger.

          Jesse Glick added a comment - Implemented raw event gathering infrastructure; initially the information is only available as a logger.

          Jesse Glick added a comment -

          Might also be useful to introduce a utility in jenkins-test-harness which would allow you to assert that a given (little-W) workflow run on an agent transfers at most a given number of packets, at most a given total size, includes only the specified packet types, etc. For example, proof of the effectiveness of JENKINS-38381 could include a test that no packets are being transmitted during the course of a long-running sh step.

          Jesse Glick added a comment - Might also be useful to introduce a utility in jenkins-test-harness which would allow you to assert that a given (little-W) workflow run on an agent transfers at most a given number of packets, at most a given total size, includes only the specified packet types, etc. For example, proof of the effectiveness of JENKINS-38381 could include a test that no packets are being transmitted during the course of a long-running sh step.

          Jesse Glick added a comment -

          Have a proposed integration in support-core which seems interesting enough to justify the API. The listener could certainly tally more detailed information, and/or present the results more compellingly, but I wanted to keep the scope small for now; other people may want to refine it now that the plumbing is in place.

          Jesse Glick added a comment - Have a proposed integration in support-core which seems interesting enough to justify the API. The listener could certainly tally more detailed information, and/or present the results more compellingly, but I wanted to keep the scope small for now; other people may want to refine it now that the plumbing is in place.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/java/hudson/FilePath.java
          core/src/main/java/hudson/slaves/SlaveComputer.java
          pom.xml
          test/src/test/java/hudson/bugs/JnlpAccessWithSecuredHudsonTest.java
          test/src/test/java/jenkins/security/Security218Test.java
          test/src/test/java/jenkins/security/s2m/DefaultFilePathFilterTest.java
          http://jenkins-ci.org/commit/jenkins/f4b78921e87c0a265c633674bc3ca3ec0458c19a
          Log:
          JENKINS-27035 Gathering command read/write events (#3071)

          • Slave.JnlpJar.getURL did not work in some modes when core had a snapshot dependency on Remoting.
          • Starting to implement Channel.Listener.read/write.
          • Now obtaining response timing statistics.
          • For now, avoiding timestamped snapshots, as it caused problems for #3120 which I have asked for help from @stephenc diagnosing.
          • Simplified logging a bit.
          • onResponse
          • hudson.FilePath$Mkdirs is a lot more readable than hudson.FilePath$13.
          • Specific snapshot.
          • onJar
          • LoggingChannelListener
          • remoting.version=3.17

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/java/hudson/FilePath.java core/src/main/java/hudson/slaves/SlaveComputer.java pom.xml test/src/test/java/hudson/bugs/JnlpAccessWithSecuredHudsonTest.java test/src/test/java/jenkins/security/Security218Test.java test/src/test/java/jenkins/security/s2m/DefaultFilePathFilterTest.java http://jenkins-ci.org/commit/jenkins/f4b78921e87c0a265c633674bc3ca3ec0458c19a Log: JENKINS-27035 Gathering command read/write events (#3071) Slave.JnlpJar.getURL did not work in some modes when core had a snapshot dependency on Remoting. Starting to implement Channel.Listener.read/write. Now obtaining response timing statistics. For now, avoiding timestamped snapshots, as it caused problems for #3120 which I have asked for help from @stephenc diagnosing. Simplified logging a bit. onResponse hudson.FilePath$Mkdirs is a lot more readable than hudson.FilePath$13. Specific snapshot. onJar LoggingChannelListener remoting.version=3.17 Making a few test assertions more lenient to adapt to https://github.com/jenkinsci/remoting/pull/247 .

          Oleg Nenashev added a comment -

          Merged towards 2.106

          I will update the changelog entries

          Oleg Nenashev added a comment - Merged towards 2.106 I will update the changelog entries

          Oleg Nenashev added a comment - - edited

          It has been released in Remoting 3.17 and Jenkins 2.106

          Oleg Nenashev added a comment - - edited It has been released in Remoting 3.17 and Jenkins 2.106

          Code changed in jenkins
          User: Owen Wood
          Path:
          Jenkinsfile
          pom.xml
          src/main/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatistics.java
          src/test/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatisticsTest.java
          http://jenkins-ci.org/commit/support-core-plugin/299ed0983c66c315ff99f6c8917599f16d9577c8
          Log:
          Merge pull request #128 from jglick/CommandListener-JENKINS-27035

          JENKINS-27035 Tallying command read/write events

          Compare: https://github.com/jenkinsci/support-core-plugin/compare/5eb81933d6aa...299ed0983c66

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Owen Wood Path: Jenkinsfile pom.xml src/main/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatistics.java src/test/java/com/cloudbees/jenkins/support/impl/SlaveCommandStatisticsTest.java http://jenkins-ci.org/commit/support-core-plugin/299ed0983c66c315ff99f6c8917599f16d9577c8 Log: Merge pull request #128 from jglick/CommandListener- JENKINS-27035 JENKINS-27035 Tallying command read/write events Compare: https://github.com/jenkinsci/support-core-plugin/compare/5eb81933d6aa...299ed0983c66

          Released in support-core:2.46

          Emilio Escobar added a comment - Released in support-core:2.46

            jglick Jesse Glick
            jglick Jesse Glick
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: