Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-49025

SecurityException: Rejected: java.lang.String$CaseInsensitiveComparator

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Critical Critical
    • pipeline-aws-plugin
    • Jenkins 2.102

      Pipeline: 2.5
      Pipeline: AWS Steps 1.20
      Plain Credentials Plugin 1.4
      Credentials Plugin 2.1.16
      Credentials Binding Plugin 1.14

      Ubuntu 14.04.5 LTS
      OpenJDK 1.8.0_141

      Creating my first pipeline build, trying to upload output to S3. Getting a SecurityException: Rejected: java.lang.String$CaseInsensitiveComparator

      Cause in my configuration:

      withAWS(credentials:'aws-softwareops') {
          s3Upload acl: 'Private', bucket: 'my-bucket', file: 'my-file.zip', path: 'master'
      }

      Remove this and the build works.

      Comment out s3Upload and the build works - proving the cause is definitely s3Upload, not withAWS.

      Change the credentials key ("aws-softwareops" above) to something different, and the build fails again, proving the credentials are being picked up from Jenkins correctly.

      This is in the logs:

      Jan 18, 2018 3:06:58 PM jenkins.security.ClassFilterImpl lambda$isBlacklisted$1
      WARNING: java.lang.String$CaseInsensitiveComparator in JRE might be dangerous, so rejecting; see https://jenkins.io/redirect/class-filter/

      This doesn't work:

      -Dhudson.remoting.ClassFilter=java.lang.String$CaseInsensitiveComparator

      (there is no mention of ClassFilter in the Jenkins log at startup - should there be?)

      This is a blocking issue for me.

      Stack trace:

      java.lang.SecurityException: Rejected: java.lang.String$CaseInsensitiveComparator
      	at hudson.remoting.ClassFilter.check(ClassFilter.java:75)
      	at hudson.remoting.MultiClassLoaderSerializer$Input.resolveClass(MultiClassLoaderSerializer.java:129)
      	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1826)
      	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1713)
      	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2000)
      	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
      	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
      	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:552)
      	at java.util.TreeMap.readObject(TreeMap.java:2449)
      	at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1058)
      	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2136)
      	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
      	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
      	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2245)
      	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2169)
      	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2027)
      	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1535)
      	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:422)
      	at hudson.remoting.UserRequest.deserialize(UserRequest.java:277)
      	at hudson.remoting.UserResponse.retrieve(UserRequest.java:310)
      	at hudson.remoting.Channel.call(Channel.java:909)
      	at hudson.FilePath.act(FilePath.java:998)
      	at hudson.FilePath.act(FilePath.java:987)
      	at de.taimos.pipeline.aws.S3UploadStep$Execution$1.run(S3UploadStep.java:259)

          [JENKINS-49025] SecurityException: Rejected: java.lang.String$CaseInsensitiveComparator

          Oleg Nenashev added a comment -

          I think that common comparators should be whiitelisted on the core's side

          Oleg Nenashev added a comment - I think that common comparators should be whiitelisted on the core's side

          Tom Fanning added a comment -

          Any idea why the workaround doesn't work?

          (-Dhudson.remoting.ClassFilter=...)

          Tom Fanning added a comment - Any idea why the workaround doesn't work? (-Dhudson.remoting.ClassFilter=...)

          Oleg Nenashev added a comment -

          Oleg Nenashev added a comment - It happens, because EnvVars get serialized over the Remoting channel: https://github.com/jenkinsci/jenkins/blob/master/core/src/main/java/hudson/EnvVars.java . It Happens in RemoteUploder: https://github.com/jenkinsci/pipeline-aws-plugin/blob/de2a2abf961971f6dcaa5354f66dd306250f1331/src/main/java/de/taimos/pipeline/aws/S3UploadStep.java#L288 The fix should be on the core's side

          Oleg Nenashev added a comment -

          tomfanning Probably you specify the option in the wrong place. It should be "java -D.... -jar jenkins.war"

          Oleg Nenashev added a comment - tomfanning Probably you specify the option in the wrong place. It should be "java -D.... -jar jenkins.war"

          Oleg Nenashev added a comment -

          CC jglick

          Oleg Nenashev added a comment - CC jglick

          Tom Fanning added a comment -

          oleg_nenashev No, it is specified alongside other -Dhudson arguments in my /etc/default/jenkins:

          JAVA_ARGS="-Djava.awt.headless=true -Dsvnkit.http.methods=Basic,NTLM,Negotiate,Digest -Dhudson.model.ParametersAction.keepUndefinedParameters=true -Dhudson.remoting.ClassFilter=java.lang.String\$CaseInsensitiveComparator"

          /etc/init.d/jenkins:

          [ -r /etc/default/$NAME ] && . /etc/default/$NAME

          ...

          $SU -l $JENKINS_USER --shell=/bin/bash -c "$DAEMON $DAEMON_ARGS – $JAVA $JAVA_ARGS -jar $JENKINS_WAR $JENKINS_ARGS" || return 2

          Tom Fanning added a comment - oleg_nenashev No, it is specified alongside other -Dhudson arguments in my /etc/default/jenkins: JAVA_ARGS="-Djava.awt.headless=true -Dsvnkit.http.methods=Basic,NTLM,Negotiate,Digest -Dhudson.model.ParametersAction.keepUndefinedParameters=true -Dhudson.remoting.ClassFilter=java.lang.String\$CaseInsensitiveComparator" /etc/init.d/jenkins: [ -r /etc/default/$NAME ] && . /etc/default/$NAME ... $SU -l $JENKINS_USER --shell=/bin/bash -c "$DAEMON $DAEMON_ARGS – $JAVA $JAVA_ARGS -jar $JENKINS_WAR $JENKINS_ARGS" || return 2

          Jesse Glick added a comment -

           Probably you specify the option in the wrong place.

          Or whatever script you are adding this option to is expanding $ as a metacharacter.

          Jesse Glick added a comment -  Probably you specify the option in the wrong place. Or whatever script you are adding this option to is expanding $ as a metacharacter.

          Jesse Glick added a comment -

          Is there a particular stack trace associated with the build failure? I see for example this field but that is sent from the master → slave so should not involve any check.

          Now there is this which would wind up calling this which does ask to receive an EnvVars from the agent. But that is called for any node block, so we should have noticed any problem long ago. In fact the only comparator we use, so far as I can tell, is hudson.util.CaseInsensitiveComparator, which is part of Jenkins core and thus implicitly whitelisted. In other words, EnvVars cannot be the problem at all.

          Jesse Glick added a comment - Is there a particular stack trace associated with the build failure? I see for example this field  but that is sent from the master → slave so should not involve any check. Now there is this  which would wind up calling this  which does ask to receive an EnvVars from the agent. But that is called for any node block, so we should have noticed any problem long ago. In fact the only comparator we use, so far as I can tell, is hudson.util.CaseInsensitiveComparator , which is part of Jenkins core and thus implicitly whitelisted. In other words, EnvVars cannot be the problem at all.

          Jesse Glick added a comment -

          From code inspection I have not found any place in this plugin or its dependencies which would be using String.CASE_INSENSITIVE_ORDER, so I have no idea what is triggering this issue. Would need to inspect the failing build log to get a clue.

          Jesse Glick added a comment - From code inspection I have not found any place in this plugin or its dependencies which would be using String.CASE_INSENSITIVE_ORDER , so I have no idea what is triggering this issue. Would need to inspect the failing build log to get a clue.

          Tom Fanning added a comment - - edited

          jglick Added the stack trace. So sorry - no idea why I failed to paste this in to the original report.

          Tom Fanning added a comment - - edited jglick Added the stack trace. So sorry - no idea why I failed to paste this in to the original report.

          Jesse Glick added a comment -

          That at least narrows it down to this line and suggests that a TreeMap is involved somewhere. From code inspection I have no clues, since this is a FileCallable<Void> which returns nothing, so there is no apparent way that deserializing the return value could cause any error.

          Jesse Glick added a comment - That at least narrows it down to this line  and suggests that a TreeMap is involved somewhere. From code inspection I have no clues, since this is a FileCallable<Void> which returns nothing, so there is no apparent way that deserializing the return value could cause any error.

          Jesse Glick added a comment -

          I do not have the setup required to actually connect to S3, and the plugin includes no tests with a mock service. Miraculously, I was able to reproduce the exception in a test that tries to just upload to a deliberately bogus bucket with bogus credentials; whatever goes wrong, it happens whether or not S3 accepts the upload.

          Jesse Glick added a comment - I do not have the setup required to actually connect to S3, and the plugin includes no tests with a mock service. Miraculously, I was able to reproduce the exception in a test that tries to just upload to a deliberately bogus bucket with bogus credentials; whatever goes wrong, it happens whether or not S3 accepts the upload.

          Jesse Glick added a comment -

          I think I see the issue. The deserialization is not of the return type, it is of an error, such as this one:

          com.amazonaws.services.s3.model.AmazonS3Exception: The specified bucket is not valid. …
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1638)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1303)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1055)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
          	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
          	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
          	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4247)
          	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4194)
          	at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1738)
          	at com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadInOneChunk(UploadCallable.java:133)
          	at com.amazonaws.services.s3.transfer.internal.UploadCallable.call(UploadCallable.java:125)
          	at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:143)
          	at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:48)
          

          And AmazonS3Exception.additionalDetails is a Map-valued field; presumably something in the SDK is setting it to a TreeMap using CaseInsensitiveComparator, a bit oddly.

          JEP-200 does give a blanket pass to all Throwable subtypes, since there are endless varieties throwable by Remoting calls, but this does not cover nested fields of those exceptions.

          I will file a PR to fix the plugin, then a core PR to allow this class for any plugin, then look into whether diagnostics could be improved here in Remoting.

          Jesse Glick added a comment - I think I see the issue. The deserialization is not of the return type, it is of an error, such as this one: com.amazonaws.services.s3.model.AmazonS3Exception: The specified bucket is not valid. … at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1638) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1303) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1055) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4247) at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4194) at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1738) at com.amazonaws.services.s3.transfer.internal.UploadCallable.uploadInOneChunk(UploadCallable.java:133) at com.amazonaws.services.s3.transfer.internal.UploadCallable.call(UploadCallable.java:125) at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:143) at com.amazonaws.services.s3.transfer.internal.UploadMonitor.call(UploadMonitor.java:48) And AmazonS3Exception.additionalDetails is a Map -valued field; presumably something in the SDK is setting it to a TreeMap using CaseInsensitiveComparator , a bit oddly. JEP-200 does give a blanket pass to all Throwable subtypes, since there are endless varieties throwable by Remoting calls, but this does not cover nested fields of those exceptions. I will file a PR to fix the plugin, then a core PR to allow this class for any plugin, then look into whether diagnostics could be improved here in Remoting.

          Oleg Nenashev added a comment -

          jglick FYI I have created a ticket for better diagnostics: JENKINS-49027 . Sorry for misleading by EnvVars, got confused by same-name types

          Oleg Nenashev added a comment - jglick FYI I have created a ticket for better diagnostics: JENKINS-49027 . Sorry for misleading by EnvVars, got confused by same-name types

          Jesse Glick added a comment -

          In case it was not clear from the above, it seems that the problem “only” affects cases where the build would have failed anyway, though with a meaningful exception.

          Jesse Glick added a comment - In case it was not clear from the above, it seems that the problem “only” affects cases where the build would have failed anyway, though with a meaningful exception.

          Tom Fanning added a comment -

          This is no longer a blocking issue for me personally - through blind luck I have managed to work out what my error was and have successfully fixed it.

          Thanks for your work resolving it more generally though.

          Tom Fanning added a comment - This is no longer a blocking issue for me personally - through blind luck I have managed to work out what my error was and have successfully fixed it. Thanks for your work resolving it more generally though.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/resources/jenkins/security/whitelisted-classes.txt
          http://jenkins-ci.org/commit/jenkins/d1e278b6528aae9b966d724182cbbc2a18489976
          Log:
          JENKINS-49025 Add java.lang.String$CaseInsensitiveComparator.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/resources/jenkins/security/whitelisted-classes.txt http://jenkins-ci.org/commit/jenkins/d1e278b6528aae9b966d724182cbbc2a18489976 Log: JENKINS-49025 Add java.lang.String$CaseInsensitiveComparator.

          Code changed in jenkins
          User: Jesse Glick
          Path:
          core/src/main/resources/jenkins/security/whitelisted-classes.txt
          http://jenkins-ci.org/commit/jenkins/2696e9c865c58779c5115ff3fb8bb527cb788678
          Log:
          Merge pull request #3245 from jglick/JENKINS-49025

          JENKINS-49025 Add java.lang.String$CaseInsensitiveComparator

          Compare: https://github.com/jenkinsci/jenkins/compare/4f4af4e6ab8c...2696e9c865c5

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: core/src/main/resources/jenkins/security/whitelisted-classes.txt http://jenkins-ci.org/commit/jenkins/2696e9c865c58779c5115ff3fb8bb527cb788678 Log: Merge pull request #3245 from jglick/ JENKINS-49025 JENKINS-49025 Add java.lang.String$CaseInsensitiveComparator Compare: https://github.com/jenkinsci/jenkins/compare/4f4af4e6ab8c...2696e9c865c5

          Code changed in jenkins
          User: Jesse Glick
          Path:
          Jenkinsfile
          README.md
          pom.xml
          src/main/java/de/taimos/pipeline/aws/S3DownloadStep.java
          src/main/java/de/taimos/pipeline/aws/S3UploadStep.java
          src/main/resources/META-INF/hudson.remoting.ClassFilter
          src/test/java/de/taimos/pipeline/aws/S3UploadStepIntegrationTest.java
          http://jenkins-ci.org/commit/pipeline-aws-plugin/fbffe2bc03384857b24262b1630b2fa2f1799b2a
          Log:
          JENKINS-49025 Adding a necessary whitelist entry.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: Jenkinsfile README.md pom.xml src/main/java/de/taimos/pipeline/aws/S3DownloadStep.java src/main/java/de/taimos/pipeline/aws/S3UploadStep.java src/main/resources/META-INF/hudson.remoting.ClassFilter src/test/java/de/taimos/pipeline/aws/S3UploadStepIntegrationTest.java http://jenkins-ci.org/commit/pipeline-aws-plugin/fbffe2bc03384857b24262b1630b2fa2f1799b2a Log: JENKINS-49025 Adding a necessary whitelist entry.

          Code changed in jenkins
          User: Thorsten Hoeger
          Path:
          Jenkinsfile
          README.md
          pom.xml
          src/main/java/de/taimos/pipeline/aws/S3DownloadStep.java
          src/main/java/de/taimos/pipeline/aws/S3UploadStep.java
          src/main/resources/META-INF/hudson.remoting.ClassFilter
          src/test/java/de/taimos/pipeline/aws/S3UploadStepIntegrationTest.java
          http://jenkins-ci.org/commit/pipeline-aws-plugin/e36797dae5a3f5e09faf14ee5722bad2aab8b400
          Log:
          Merge pull request #37 from jglick/JENKINS-49025

          JENKINS-49025 Adding a necessary whitelist entry

          Compare: https://github.com/jenkinsci/pipeline-aws-plugin/compare/0ed1e9b04ff9...e36797dae5a3

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Thorsten Hoeger Path: Jenkinsfile README.md pom.xml src/main/java/de/taimos/pipeline/aws/S3DownloadStep.java src/main/java/de/taimos/pipeline/aws/S3UploadStep.java src/main/resources/META-INF/hudson.remoting.ClassFilter src/test/java/de/taimos/pipeline/aws/S3UploadStepIntegrationTest.java http://jenkins-ci.org/commit/pipeline-aws-plugin/e36797dae5a3f5e09faf14ee5722bad2aab8b400 Log: Merge pull request #37 from jglick/ JENKINS-49025 JENKINS-49025 Adding a necessary whitelist entry Compare: https://github.com/jenkinsci/pipeline-aws-plugin/compare/0ed1e9b04ff9...e36797dae5a3

          Code changed in jenkins
          User: Jesse Glick
          Path:
          src/main/java/org/jenkinsci/plugins/workflow/actions/ErrorAction.java
          src/test/java/org/jenkinsci/plugins/workflow/actions/ErrorActionTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/a48460ebdf7d03bedd82bb9e3e9de41481d58a4a
          Log:
          JENKINS-49025 Extending JENKINS-34488 fix yet again; JENKINS-39346 only considered causes, but other Throwable fields could also be unserializable.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: src/main/java/org/jenkinsci/plugins/workflow/actions/ErrorAction.java src/test/java/org/jenkinsci/plugins/workflow/actions/ErrorActionTest.java http://jenkins-ci.org/commit/workflow-api-plugin/a48460ebdf7d03bedd82bb9e3e9de41481d58a4a Log: JENKINS-49025 Extending JENKINS-34488 fix yet again; JENKINS-39346 only considered causes, but other Throwable fields could also be unserializable.

          Code changed in jenkins
          User: Andrew Bayer
          Path:
          src/main/java/org/jenkinsci/plugins/workflow/actions/ErrorAction.java
          src/test/java/org/jenkinsci/plugins/workflow/actions/ErrorActionTest.java
          http://jenkins-ci.org/commit/workflow-api-plugin/43fc2f234bdddbdbdbf72c1844cc5750c88618c0
          Log:
          Merge pull request #64 from jglick/nested-error-fields-JENKINS-49025

          JENKINS-49025 Extending JENKINS-34488 fix yet again

          Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/1f5461aa99df...43fc2f234bdd

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Andrew Bayer Path: src/main/java/org/jenkinsci/plugins/workflow/actions/ErrorAction.java src/test/java/org/jenkinsci/plugins/workflow/actions/ErrorActionTest.java http://jenkins-ci.org/commit/workflow-api-plugin/43fc2f234bdddbdbdbf72c1844cc5750c88618c0 Log: Merge pull request #64 from jglick/nested-error-fields- JENKINS-49025 JENKINS-49025 Extending JENKINS-34488 fix yet again Compare: https://github.com/jenkinsci/workflow-api-plugin/compare/1f5461aa99df...43fc2f234bdd

            jglick Jesse Glick
            tomfanning Tom Fanning
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: