• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • core
    • Jenkins 2.102
      OpenJDK 1.8.0_151
      Amazon Linux

      I have Jenkins 2.101 running perfectly well. When yum upgrades to 2.102 it fails to start with a "java.nio.charset.MalformedInputException: Input length = 1" error.
      I can downgrade back to 101 and it starts fine.

      I'd guess at something in a config file somewhere it doesn't like, but 101 is happy with it, and it isn't telling me which config file. We're unlikely to be using any international characters beyond ASCII, but we could be, somewhere...
      (I've included the error messages about slaves but I expect they're just a consequence of the main app failing to start. It does run a web server, and I can get the same error about charset in my browser, framed by a normal Jenkins page.)

      Jenkins home directory: /var/lib/jenkins found at: SystemProperties.getProperty("JENKINS_HOME")
      Jan 21, 2018 12:43:40 PM org.eclipse.jetty.server.handler.ContextHandler doStart
      INFO: Started w.@798162bc{/,file:///var/cache/jenkins/war/,AVAILABLE}{/var/cache/jenkins/war}
      Jan 21, 2018 12:43:40 PM org.eclipse.jetty.server.AbstractConnector doStart
      INFO: Started ServerConnector@13526e59{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
      Jan 21, 2018 12:43:40 PM org.eclipse.jetty.server.Server doStart
      INFO: Started @2973ms
      Jan 21, 2018 12:43:40 PM winstone.Logger logInternal
      INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
      Jan 21, 2018 12:43:40 PM hudson.util.BootFailure publish
      SEVERE: Failed to initialize Jenkins
      hudson.util.HudsonFailedToLoad: java.nio.charset.MalformedInputException: Input length = 1
      at hudson.WebAppMain$3.run(WebAppMain.java:250)
      Caused by: java.nio.charset.MalformedInputException: Input length = 1
      at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
      at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
      at java.io.InputStreamReader.read(InputStreamReader.java:184)
      at java.io.BufferedReader.fill(BufferedReader.java:161)
      at java.io.BufferedReader.readLine(BufferedReader.java:324)
      at java.io.BufferedReader.readLine(BufferedReader.java:389)
      at hudson.util.TextFile.read(TextFile.java:76)
      at hudson.util.TextFile.readTrim(TextFile.java:204)
      at jenkins.model.Jenkins.<init>(Jenkins.java:869)
      at hudson.model.Hudson.<init>(Hudson.java:85)
      at hudson.model.Hudson.<init>(Hudson.java:81)
      at hudson.WebAppMain$3.run(WebAppMain.java:233)
      
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins cleanUp
      INFO: Stopping Jenkins
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onAttained
      INFO: Started termination
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onTaskFailed
      SEVERE: Failed IOHubProvider.cleanUp
      java.lang.IllegalArgumentException: Unable to inject class jenkins.slaves.IOHubProvider
      at hudson.init.TaskMethodFinder.lookUp(TaskMethodFinder.java:125)
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
      at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
      at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:263)
      at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3271)
      at jenkins.model.Jenkins.cleanUp(Jenkins.java:3192)
      at hudson.WebAppMain$3.run(WebAppMain.java:254)
      
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onTaskFailed
      SEVERE: Failed NioChannelSelector.cleanUp
      java.lang.IllegalArgumentException: Unable to inject class jenkins.slaves.NioChannelSelector
      at hudson.init.TaskMethodFinder.lookUp(TaskMethodFinder.java:125)
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
      at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
      at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:263)
      at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3271)
      at jenkins.model.Jenkins.cleanUp(Jenkins.java:3192)
      at hudson.WebAppMain$3.run(WebAppMain.java:254)
      
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onAttained
      INFO: Completed termination
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins _cleanUpDisconnectComputers
      INFO: Starting node disconnection
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins _cleanUpPersistQueue
      INFO: Persisting build queue
      Jan 21, 2018 12:43:41 PM jenkins.model.Jenkins cleanUp
      INFO: Jenkins stopped
      
      

          [JENKINS-49060] Jenkins fails to start

          Oleg Nenashev added a comment -

          The line points to "secretKey = secretFile.readTrim();" which tries to read a secret from the disk (Jenkins:869). The code itself has not changed since 2012, but in 2.102 there was a change in the TextFile implementation: https://github.com/jenkinsci/jenkins/commit/67076834a24f03b832ab03b1bf9b96e05a5da81e#diff-7bec82f0a647b1592b682a66e7cc6bcf

          CC dtrebbien

          Oleg Nenashev added a comment - The line points to "secretKey = secretFile.readTrim();" which tries to read a secret from the disk (Jenkins:869). The code itself has not changed since 2012, but in 2.102 there was a change in the TextFile implementation: https://github.com/jenkinsci/jenkins/commit/67076834a24f03b832ab03b1bf9b96e05a5da81e#diff-7bec82f0a647b1592b682a66e7cc6bcf CC dtrebbien

          Oleg Nenashev added a comment -

          I would assume that the file is corrupted somehow, but it needs to be diagnosed. And I doubt I can ask to attach the secret file to JIRA

          Oleg Nenashev added a comment - I would assume that the file is corrupted somehow, but it needs to be diagnosed. And I doubt I can ask to attach the secret file to JIRA

          max allan added a comment -

          Thanks oleg_nenashev
          Looking way back in time I can find a copy of the secret on an old server :

           

          ls -la secret.key 
          -rw-r--r-- 1 jenkins jenkins 64 Apr 27  2012 secret.key
          [root@jenkins jenkins]# more secret.key 
          d86b15...REDACTED...12de09
          

           

           

          Looking on the current server :

          ls -la secret.key
          -rw-r--r-- 1 jenkins jenkins 61440 May 18 2017 secret.key
          [root@jenkins]# more secret.key
          secret.key.not-so-secret
          #
          # This file is for Jenkins core developers to list plugin callables which we believe are safe.
          #
          # To whitelist other names, place *.conf files by other names into this folder.
          # This file gets overwritten every time Jenkins starts.
          #
          # See https://jenkins.io/redirect/security-144 for more details.
          etc.......
          more text, config files, random non-ASCII, etc...

           

          It's been like this since May last year!! And nothing broke.

          max allan added a comment - Thanks oleg_nenashev Looking way back in time I can find a copy of the secret on an old server :   ls -la secret.key -rw-r--r-- 1 jenkins jenkins 64 Apr 27  2012 secret.key [root@jenkins jenkins]# more secret.key d86b15...REDACTED...12de09     Looking on the current server : ls -la secret.key -rw-r--r-- 1 jenkins jenkins 61440 May 18 2017 secret.key [root@jenkins]# more secret.key secret.key.not-so-secret # # This file is for Jenkins core developers to list plugin callables which we believe are safe. # # To whitelist other names, place *.conf files by other names into this folder. # This file gets overwritten every time Jenkins starts. # # See https: //jenkins.io/redirect/security-144 for more details. etc....... more text, config files, random non-ASCII, etc...   It's been like this since May last year!! And nothing broke.

          The file probably has non-UTF-8 content.

          Investigating, I found that there is a slight difference between:

          BufferedReader in = new BufferedReader(new InputStreamReader(Files.newInputStream(file.toPath()), "UTF-8")
          

          and

          BufferedReader in = Files.newBufferedReader(file.toPath(), StandardCharsets.UTF_8)
          

          Whereas the former uses the InputStreamReader constructor taking a charset name, the latter uses the InputStreamReader taking a CharsetDecoder. CharsetDecoder is apparently more picky about not accepting malformed bytes.

          Daniel Trebbien added a comment - The file probably has non-UTF-8 content. Investigating, I found that there is a slight difference between: BufferedReader in = new BufferedReader( new InputStreamReader(Files.newInputStream(file.toPath()), "UTF-8" ) and BufferedReader in = Files.newBufferedReader(file.toPath(), StandardCharsets.UTF_8) Whereas the former uses the InputStreamReader constructor taking a charset name, the latter uses the InputStreamReader taking a CharsetDecoder . CharsetDecoder is apparently more picky about not accepting malformed bytes.

          Here is a test I wrote showing the difference:

          import java.io.*;
          import java.nio.charset.*;
          
          public class CharsetsTest {
              public static void main(String[] args) throws Exception {
                  String str = "\uD83D\uDE0E";
                  byte[] bytes = str.getBytes(StandardCharsets.UTF_8);
                  for (byte b : bytes) {
                      System.out.printf("%02x ", b);
                  }
                  System.out.printf("(%d bytes)\n", bytes.length);
                  byte[] malformedBytes = new byte[bytes.length + 1];
                  System.arraycopy(bytes, 0, malformedBytes, 1, bytes.length);
                  malformedBytes[0] = malformedBytes[1];
                  for (byte b : malformedBytes) {
                      System.out.printf("%02x ", b);
                  }
                  System.out.printf("(%d bytes)\n", malformedBytes.length);
          
                  {
                      System.out.println("Reading via `new InputStreamReader(is, \"UTF-8\")'");
                      ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes);
                      BufferedReader br = new BufferedReader(new InputStreamReader(is, "UTF-8"));
                      br.readLine();
                  }
                  {
                      System.out.println("Reading via `new InputStreamReader(is, StandardCharsets.UTF_8)'");
                      ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes);
                      BufferedReader br = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
                      br.readLine();
                  }
                  {
                      System.out.println("Reading via `new InputStreamReader(is, d)'");
                      ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes);
                      CharsetDecoder d = StandardCharsets.UTF_8.newDecoder();
                      BufferedReader br = new BufferedReader(new InputStreamReader(is, d));
                      br.readLine();
                  }
              }
          }
          

          This outputs:

          f0 9f 98 8e (4 bytes)
          f0 f0 9f 98 8e (5 bytes)
          Reading via `new InputStreamReader(is, "UTF-8")'
          Reading via `new InputStreamReader(is, StandardCharsets.UTF_8)'
          Reading via `new InputStreamReader(is, d)'
          Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
          	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
          	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
          	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
          	at java.io.InputStreamReader.read(InputStreamReader.java:184)
          	at java.io.BufferedReader.fill(BufferedReader.java:161)
          	at java.io.BufferedReader.readLine(BufferedReader.java:324)
          	at java.io.BufferedReader.readLine(BufferedReader.java:389)
          	at CharsetsTest.main(CharsetsTest.java:121)
          

          Daniel Trebbien added a comment - Here is a test I wrote showing the difference: import java.io.*; import java.nio.charset.*; public class CharsetsTest { public static void main( String [] args) throws Exception { String str = "\uD83D\uDE0E" ; byte [] bytes = str.getBytes(StandardCharsets.UTF_8); for ( byte b : bytes) { System .out.printf( "%02x " , b); } System .out.printf( "(%d bytes)\n" , bytes.length); byte [] malformedBytes = new byte [bytes.length + 1]; System .arraycopy(bytes, 0, malformedBytes, 1, bytes.length); malformedBytes[0] = malformedBytes[1]; for ( byte b : malformedBytes) { System .out.printf( "%02x " , b); } System .out.printf( "(%d bytes)\n" , malformedBytes.length); { System .out.println( "Reading via ` new InputStreamReader(is, \" UTF-8\ ")'" ); ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes); BufferedReader br = new BufferedReader( new InputStreamReader(is, "UTF-8" )); br.readLine(); } { System .out.println( "Reading via ` new InputStreamReader(is, StandardCharsets.UTF_8)'" ); ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes); BufferedReader br = new BufferedReader( new InputStreamReader(is, StandardCharsets.UTF_8)); br.readLine(); } { System .out.println( "Reading via ` new InputStreamReader(is, d)'" ); ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes); CharsetDecoder d = StandardCharsets.UTF_8.newDecoder(); BufferedReader br = new BufferedReader( new InputStreamReader(is, d)); br.readLine(); } } } This outputs: f0 9f 98 8e (4 bytes) f0 f0 9f 98 8e (5 bytes) Reading via `new InputStreamReader(is, "UTF-8")' Reading via `new InputStreamReader(is, StandardCharsets.UTF_8)' Reading via `new InputStreamReader(is, d)' Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:281) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) at java.io.BufferedReader.readLine(BufferedReader.java:389) at CharsetsTest.main(CharsetsTest.java:121)

          Is the secret.key file on the current server a default file? Wouldn't that be a security configuration issue?

          Daniel Trebbien added a comment - Is the secret.key file on the current server a default file? Wouldn't that be a security configuration issue?

          max allan added a comment -

          It's totally corrupted. Bits of Jenkins config, bits of binary from god knows where, bits of other files.... So, I removed it. Restarted Jenkins and all seems well. No idea how the file got corrupted so badly... Depending on how worried the project are about bad files causing weird  problems this can probably be closed. Hopefully anyone else googling the same error will find this solution!

          max allan added a comment - It's totally corrupted. Bits of Jenkins config, bits of binary from god knows where, bits of other files.... So, I removed it. Restarted Jenkins and all seems well. No idea how the file got corrupted so badly... Depending on how worried the project are about bad files causing weird  problems this can probably be closed. Hopefully anyone else googling the same error will find this solution!

          Code changed in jenkins
          User: Daniel Trebbien
          Path:
          core/src/main/java/hudson/util/TextFile.java
          http://jenkins-ci.org/commit/jenkins/0f1c35291d38a7b5700a4cfa47eb04d2cee3a0a2
          Log:
          JENKINS-49060 Include the filename of the unreadable file in the exception message

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Daniel Trebbien Path: core/src/main/java/hudson/util/TextFile.java http://jenkins-ci.org/commit/jenkins/0f1c35291d38a7b5700a4cfa47eb04d2cee3a0a2 Log: JENKINS-49060 Include the filename of the unreadable file in the exception message

          Code changed in jenkins
          User: Oleg Nenashev
          Path:
          core/src/main/java/hudson/util/TextFile.java
          http://jenkins-ci.org/commit/jenkins/66bb3dc6fd2a7ef456b3b7dba732ac2336943236
          Log:
          Merge pull request #3255 from dtrebbien/JENKINS-49060-include-filename-of-unreadable-file

          JENKINS-49060 Include the filename of the unreadable file in the exception message

          Compare: https://github.com/jenkinsci/jenkins/compare/839b003261e8...66bb3dc6fd2a

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Oleg Nenashev Path: core/src/main/java/hudson/util/TextFile.java http://jenkins-ci.org/commit/jenkins/66bb3dc6fd2a7ef456b3b7dba732ac2336943236 Log: Merge pull request #3255 from dtrebbien/ JENKINS-49060 -include-filename-of-unreadable-file JENKINS-49060 Include the filename of the unreadable file in the exception message Compare: https://github.com/jenkinsci/jenkins/compare/839b003261e8...66bb3dc6fd2a

          Daniel Beck added a comment -

          Resolved towards 2.104.

          Daniel Beck added a comment - Resolved towards 2.104.

            dtrebbien Daniel Trebbien
            max_allan max allan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: