Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-23471

Channel reader thread consumes all CPUs

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      On ci.jenkins-ci.org I noticed that there are a lot of channel reader threads claiming to be connected to remote-slave-6.

      ps output indicates that these threads are eating up CPUs, even though the method names suggest they should be blocking.

      "Channel reader thread: remote-slave-6" prio=10 tid=0x0000000004be5800 nid=0x1f59 runnable [0x00007fb6722e1000]
   java.lang.Thread.State: RUNNABLE
        at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:77)
        at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
        at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:68)
        at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:93)
        at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33)
        at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
        at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: remote-slave-6" prio=10 tid=0x0000000004be4000 nid=0x71d8 runnable [0x00007fb673ffe000]
   java.lang.Thread.State: RUNNABLE
        at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:79)
        at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
        at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:68)
        at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:93)
        at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33)
        at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
        at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: remote-slave-6" prio=10 tid=0x0000000004e48800 nid=0x4edc runnable [0x00007fb6719d8000]
   java.lang.Thread.State: RUNNABLE
        at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:79)
        at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
        at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:67)
        at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:93)
        at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33)
        at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
        at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

"Channel reader thread: remote-slave-6" prio=10 tid=0x0000000004df2000 nid=0x406d runnable [0x00007fb67b432000]
   java.lang.Thread.State: RUNNABLE
        at com.trilead.ssh2.channel.ChannelInputStream.read(ChannelInputStream.java:79)
        at hudson.remoting.FlightRecorderInputStream.read(FlightRecorderInputStream.java:82)
        at hudson.remoting.ChunkedInputStream.readHeader(ChunkedInputStream.java:68)
        at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:93)
        at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:33)
        at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:34)
        at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

        Attachments

          Activity

          Ascending order - Click to sort in descending order
          5 older comments
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          src/main/java/hudson/remoting/ChunkedInputStream.java
          http://jenkins-ci.org/commit/remoting/83da718ab06b2f881aa08434684a02ea95ff5135
          Log:
          [FIXED JENKINS-23471]

          if readUntilBreak hits stream EOF, it would end up hanging in a busy loop. EOF is a break boundary,
          so it should cause the method to return gracefully.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: src/main/java/hudson/remoting/ChunkedInputStream.java http://jenkins-ci.org/commit/remoting/83da718ab06b2f881aa08434684a02ea95ff5135 Log: [FIXED JENKINS-23471] if readUntilBreak hits stream EOF, it would end up hanging in a busy loop. EOF is a break boundary, so it should cause the method to return gracefully.
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          pom.xml
          http://jenkins-ci.org/commit/jenkins/8fc609fe0952b285d5b26a59fd5ff4c29704d33d
          Log:
          [JENKINS-23471 JENKINS-24050]

          Integrated the fix in remoting to Jenkins 1.580.

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/8fc609fe0952b285d5b26a59fd5ff4c29704d33d Log: [JENKINS-23471 JENKINS-24050] Integrated the fix in remoting to Jenkins 1.580.
          Hide
          Permalink
          olenz Olaf Lenz added a comment -

          Will the patch also be included into the LTS?

          Show
          olenz Olaf Lenz added a comment - Will the patch also be included into the LTS?
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          pom.xml
          http://jenkins-ci.org/commit/jenkins/9c82fc42eb08b89047c544aaa586291ad1485472
          Log:
          [JENKINS-23471 JENKINS-24050]

          Integrated the fix in remoting to Jenkins 1.580.

          (cherry picked from commit 8fc609fe0952b285d5b26a59fd5ff4c29704d33d)

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/9c82fc42eb08b89047c544aaa586291ad1485472 Log: [JENKINS-23471 JENKINS-24050] Integrated the fix in remoting to Jenkins 1.580. (cherry picked from commit 8fc609fe0952b285d5b26a59fd5ff4c29704d33d)
          Hide
          Permalink
          scm_issue_link SCM/JIRA link daemon added a comment -

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          pom.xml
          http://jenkins-ci.org/commit/jenkins/91c5551d4c7682d4adba28fe591fa7772eee62e0
          Log:
          [JENKINS-23471 JENKINS-24050]

          Integrated the fix in remoting to Jenkins 1.580.

          (cherry picked from commit 8fc609fe0952b285d5b26a59fd5ff4c29704d33d)

          Show
          scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html pom.xml http://jenkins-ci.org/commit/jenkins/91c5551d4c7682d4adba28fe591fa7772eee62e0 Log: [JENKINS-23471 JENKINS-24050] Integrated the fix in remoting to Jenkins 1.580. (cherry picked from commit 8fc609fe0952b285d5b26a59fd5ff4c29704d33d)

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            kohsuke Kohsuke Kawaguchi
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: