• Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • core
    • Jenkins 2.102
      OpenJDK 1.8.0_151
      Amazon Linux

      I have Jenkins 2.101 running perfectly well. When yum upgrades to 2.102 it fails to start with a "java.nio.charset.MalformedInputException: Input length = 1" error.
      I can downgrade back to 101 and it starts fine.

      I'd guess at something in a config file somewhere it doesn't like, but 101 is happy with it, and it isn't telling me which config file. We're unlikely to be using any international characters beyond ASCII, but we could be, somewhere...
      (I've included the error messages about slaves but I expect they're just a consequence of the main app failing to start. It does run a web server, and I can get the same error about charset in my browser, framed by a normal Jenkins page.)

      Jenkins home directory: /var/lib/jenkins found at: SystemProperties.getProperty("JENKINS_HOME")
      Jan 21, 2018 12:43:40 PM org.eclipse.jetty.server.handler.ContextHandler doStart
      INFO: Started w.@798162bc{/,file:///var/cache/jenkins/war/,AVAILABLE}{/var/cache/jenkins/war}
      Jan 21, 2018 12:43:40 PM org.eclipse.jetty.server.AbstractConnector doStart
      INFO: Started ServerConnector@13526e59{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
      Jan 21, 2018 12:43:40 PM org.eclipse.jetty.server.Server doStart
      INFO: Started @2973ms
      Jan 21, 2018 12:43:40 PM winstone.Logger logInternal
      INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
      Jan 21, 2018 12:43:40 PM hudson.util.BootFailure publish
      SEVERE: Failed to initialize Jenkins
      hudson.util.HudsonFailedToLoad: java.nio.charset.MalformedInputException: Input length = 1
      at hudson.WebAppMain$3.run(WebAppMain.java:250)
      Caused by: java.nio.charset.MalformedInputException: Input length = 1
      at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
      at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
      at java.io.InputStreamReader.read(InputStreamReader.java:184)
      at java.io.BufferedReader.fill(BufferedReader.java:161)
      at java.io.BufferedReader.readLine(BufferedReader.java:324)
      at java.io.BufferedReader.readLine(BufferedReader.java:389)
      at hudson.util.TextFile.read(TextFile.java:76)
      at hudson.util.TextFile.readTrim(TextFile.java:204)
      at jenkins.model.Jenkins.<init>(Jenkins.java:869)
      at hudson.model.Hudson.<init>(Hudson.java:85)
      at hudson.model.Hudson.<init>(Hudson.java:81)
      at hudson.WebAppMain$3.run(WebAppMain.java:233)
      
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins cleanUp
      INFO: Stopping Jenkins
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onAttained
      INFO: Started termination
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onTaskFailed
      SEVERE: Failed IOHubProvider.cleanUp
      java.lang.IllegalArgumentException: Unable to inject class jenkins.slaves.IOHubProvider
      at hudson.init.TaskMethodFinder.lookUp(TaskMethodFinder.java:125)
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
      at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
      at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:263)
      at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3271)
      at jenkins.model.Jenkins.cleanUp(Jenkins.java:3192)
      at hudson.WebAppMain$3.run(WebAppMain.java:254)
      
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onTaskFailed
      SEVERE: Failed NioChannelSelector.cleanUp
      java.lang.IllegalArgumentException: Unable to inject class jenkins.slaves.NioChannelSelector
      at hudson.init.TaskMethodFinder.lookUp(TaskMethodFinder.java:125)
      at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
      at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
      at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:128)
      at jenkins.model.Jenkins$18.execute(Jenkins.java:3274)
      at org.jvnet.hudson.reactor.Reactor$Node.runIfPossible(Reactor.java:139)
      at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:263)
      at jenkins.model.Jenkins._cleanUpRunTerminators(Jenkins.java:3271)
      at jenkins.model.Jenkins.cleanUp(Jenkins.java:3192)
      at hudson.WebAppMain$3.run(WebAppMain.java:254)
      
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins$19 onAttained
      INFO: Completed termination
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins _cleanUpDisconnectComputers
      INFO: Starting node disconnection
      Jan 21, 2018 12:43:40 PM jenkins.model.Jenkins _cleanUpPersistQueue
      INFO: Persisting build queue
      Jan 21, 2018 12:43:41 PM jenkins.model.Jenkins cleanUp
      INFO: Jenkins stopped
      
      

          [JENKINS-49060] Jenkins fails to start

          max allan created issue -

          Oleg Nenashev added a comment -

          The line points to "secretKey = secretFile.readTrim();" which tries to read a secret from the disk (Jenkins:869). The code itself has not changed since 2012, but in 2.102 there was a change in the TextFile implementation: https://github.com/jenkinsci/jenkins/commit/67076834a24f03b832ab03b1bf9b96e05a5da81e#diff-7bec82f0a647b1592b682a66e7cc6bcf

          CC dtrebbien

          Oleg Nenashev added a comment - The line points to "secretKey = secretFile.readTrim();" which tries to read a secret from the disk (Jenkins:869). The code itself has not changed since 2012, but in 2.102 there was a change in the TextFile implementation: https://github.com/jenkinsci/jenkins/commit/67076834a24f03b832ab03b1bf9b96e05a5da81e#diff-7bec82f0a647b1592b682a66e7cc6bcf CC dtrebbien
          Oleg Nenashev made changes -
          Assignee New: Daniel Trebbien [ dtrebbien ]
          Oleg Nenashev made changes -
          Priority Original: Minor [ 4 ] New: Critical [ 2 ]

          Oleg Nenashev added a comment -

          I would assume that the file is corrupted somehow, but it needs to be diagnosed. And I doubt I can ask to attach the secret file to JIRA

          Oleg Nenashev added a comment - I would assume that the file is corrupted somehow, but it needs to be diagnosed. And I doubt I can ask to attach the secret file to JIRA

          max allan added a comment -

          Thanks oleg_nenashev
          Looking way back in time I can find a copy of the secret on an old server :

           

          ls -la secret.key 
          -rw-r--r-- 1 jenkins jenkins 64 Apr 27  2012 secret.key
          [root@jenkins jenkins]# more secret.key 
          d86b15...REDACTED...12de09
          

           

           

          Looking on the current server :

          ls -la secret.key
          -rw-r--r-- 1 jenkins jenkins 61440 May 18 2017 secret.key
          [root@jenkins]# more secret.key
          secret.key.not-so-secret
          #
          # This file is for Jenkins core developers to list plugin callables which we believe are safe.
          #
          # To whitelist other names, place *.conf files by other names into this folder.
          # This file gets overwritten every time Jenkins starts.
          #
          # See https://jenkins.io/redirect/security-144 for more details.
          etc.......
          more text, config files, random non-ASCII, etc...

           

          It's been like this since May last year!! And nothing broke.

          max allan added a comment - Thanks oleg_nenashev Looking way back in time I can find a copy of the secret on an old server :   ls -la secret.key -rw-r--r-- 1 jenkins jenkins 64 Apr 27  2012 secret.key [root@jenkins jenkins]# more secret.key d86b15...REDACTED...12de09     Looking on the current server : ls -la secret.key -rw-r--r-- 1 jenkins jenkins 61440 May 18 2017 secret.key [root@jenkins]# more secret.key secret.key.not-so-secret # # This file is for Jenkins core developers to list plugin callables which we believe are safe. # # To whitelist other names, place *.conf files by other names into this folder. # This file gets overwritten every time Jenkins starts. # # See https: //jenkins.io/redirect/security-144 for more details. etc....... more text, config files, random non-ASCII, etc...   It's been like this since May last year!! And nothing broke.

          The file probably has non-UTF-8 content.

          Investigating, I found that there is a slight difference between:

          BufferedReader in = new BufferedReader(new InputStreamReader(Files.newInputStream(file.toPath()), "UTF-8")
          

          and

          BufferedReader in = Files.newBufferedReader(file.toPath(), StandardCharsets.UTF_8)
          

          Whereas the former uses the InputStreamReader constructor taking a charset name, the latter uses the InputStreamReader taking a CharsetDecoder. CharsetDecoder is apparently more picky about not accepting malformed bytes.

          Daniel Trebbien added a comment - The file probably has non-UTF-8 content. Investigating, I found that there is a slight difference between: BufferedReader in = new BufferedReader( new InputStreamReader(Files.newInputStream(file.toPath()), "UTF-8" ) and BufferedReader in = Files.newBufferedReader(file.toPath(), StandardCharsets.UTF_8) Whereas the former uses the InputStreamReader constructor taking a charset name, the latter uses the InputStreamReader taking a CharsetDecoder . CharsetDecoder is apparently more picky about not accepting malformed bytes.

          Here is a test I wrote showing the difference:

          import java.io.*;
          import java.nio.charset.*;
          
          public class CharsetsTest {
              public static void main(String[] args) throws Exception {
                  String str = "\uD83D\uDE0E";
                  byte[] bytes = str.getBytes(StandardCharsets.UTF_8);
                  for (byte b : bytes) {
                      System.out.printf("%02x ", b);
                  }
                  System.out.printf("(%d bytes)\n", bytes.length);
                  byte[] malformedBytes = new byte[bytes.length + 1];
                  System.arraycopy(bytes, 0, malformedBytes, 1, bytes.length);
                  malformedBytes[0] = malformedBytes[1];
                  for (byte b : malformedBytes) {
                      System.out.printf("%02x ", b);
                  }
                  System.out.printf("(%d bytes)\n", malformedBytes.length);
          
                  {
                      System.out.println("Reading via `new InputStreamReader(is, \"UTF-8\")'");
                      ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes);
                      BufferedReader br = new BufferedReader(new InputStreamReader(is, "UTF-8"));
                      br.readLine();
                  }
                  {
                      System.out.println("Reading via `new InputStreamReader(is, StandardCharsets.UTF_8)'");
                      ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes);
                      BufferedReader br = new BufferedReader(new InputStreamReader(is, StandardCharsets.UTF_8));
                      br.readLine();
                  }
                  {
                      System.out.println("Reading via `new InputStreamReader(is, d)'");
                      ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes);
                      CharsetDecoder d = StandardCharsets.UTF_8.newDecoder();
                      BufferedReader br = new BufferedReader(new InputStreamReader(is, d));
                      br.readLine();
                  }
              }
          }
          

          This outputs:

          f0 9f 98 8e (4 bytes)
          f0 f0 9f 98 8e (5 bytes)
          Reading via `new InputStreamReader(is, "UTF-8")'
          Reading via `new InputStreamReader(is, StandardCharsets.UTF_8)'
          Reading via `new InputStreamReader(is, d)'
          Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1
          	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
          	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
          	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
          	at java.io.InputStreamReader.read(InputStreamReader.java:184)
          	at java.io.BufferedReader.fill(BufferedReader.java:161)
          	at java.io.BufferedReader.readLine(BufferedReader.java:324)
          	at java.io.BufferedReader.readLine(BufferedReader.java:389)
          	at CharsetsTest.main(CharsetsTest.java:121)
          

          Daniel Trebbien added a comment - Here is a test I wrote showing the difference: import java.io.*; import java.nio.charset.*; public class CharsetsTest { public static void main( String [] args) throws Exception { String str = "\uD83D\uDE0E" ; byte [] bytes = str.getBytes(StandardCharsets.UTF_8); for ( byte b : bytes) { System .out.printf( "%02x " , b); } System .out.printf( "(%d bytes)\n" , bytes.length); byte [] malformedBytes = new byte [bytes.length + 1]; System .arraycopy(bytes, 0, malformedBytes, 1, bytes.length); malformedBytes[0] = malformedBytes[1]; for ( byte b : malformedBytes) { System .out.printf( "%02x " , b); } System .out.printf( "(%d bytes)\n" , malformedBytes.length); { System .out.println( "Reading via ` new InputStreamReader(is, \" UTF-8\ ")'" ); ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes); BufferedReader br = new BufferedReader( new InputStreamReader(is, "UTF-8" )); br.readLine(); } { System .out.println( "Reading via ` new InputStreamReader(is, StandardCharsets.UTF_8)'" ); ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes); BufferedReader br = new BufferedReader( new InputStreamReader(is, StandardCharsets.UTF_8)); br.readLine(); } { System .out.println( "Reading via ` new InputStreamReader(is, d)'" ); ByteArrayInputStream is = new ByteArrayInputStream(malformedBytes); CharsetDecoder d = StandardCharsets.UTF_8.newDecoder(); BufferedReader br = new BufferedReader( new InputStreamReader(is, d)); br.readLine(); } } } This outputs: f0 9f 98 8e (4 bytes) f0 f0 9f 98 8e (5 bytes) Reading via `new InputStreamReader(is, "UTF-8")' Reading via `new InputStreamReader(is, StandardCharsets.UTF_8)' Reading via `new InputStreamReader(is, d)' Exception in thread "main" java.nio.charset.MalformedInputException: Input length = 1 at java.nio.charset.CoderResult.throwException(CoderResult.java:281) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) at java.io.BufferedReader.readLine(BufferedReader.java:389) at CharsetsTest.main(CharsetsTest.java:121)

          Is the secret.key file on the current server a default file? Wouldn't that be a security configuration issue?

          Daniel Trebbien added a comment - Is the secret.key file on the current server a default file? Wouldn't that be a security configuration issue?
          Daniel Beck made changes -
          Labels New: regression

            dtrebbien Daniel Trebbien
            max_allan max allan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: