Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-67718

InfluxDB Plugin broken since 3.1 update on docker: `java.nio.file.AccessDeniedException: etc/ssl/private`

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Blocker Blocker
    • influxdb-plugin
    • None
    • Jenkins Official Helm Chart 3.11.3, Jenkins 2.319.2, InfluxDB Plugin 3.1, OKD 4.9
    • 3.1.1

      We run Jenkins in Kubernetes installed from the official helm chart with the official jenkins docker image for version 2.319.2. After upgrading to the influxdb 3.1 plugin none of our jobs can report metrics to influxdb. This was tested with a simple pipeline job

      echo "foobar"

      Console output:

      [Pipeline] Start of Pipeline
[Pipeline] echo (hide)
Foobar
[Pipeline] End of Pipeline
[InfluxDB Plugin] Collecting data...
Finished: SUCCESS

      Jenkins Log:

      2022-02-02 07:52:27.782+0000 [id=600]	WARNING	h.model.listeners.RunListener#report: RunListener failed
java.nio.file.AccessDeniedException: etc/ssl/private
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:432)
	at java.base/java.nio.file.Files.newDirectoryStream(Files.java:472)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:300)
	at java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:373)
	at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:83)
Caused: java.io.UncheckedIOException
	at java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87)
	at java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103)
	at java.base/java.util.Iterator.forEachRemaining(Iterator.java:132)
	at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
	at jenkinsci.plugins.influxdb.generators.SonarQubePointGenerator.findReportByFileName(SonarQubePointGenerator.java:317)
	at jenkinsci.plugins.influxdb.generators.SonarQubePointGenerator.getSonarProjectFromBuildReport(SonarQubePointGenerator.java:258)
	at jenkinsci.plugins.influxdb.generators.SonarQubePointGenerator.hasReport(SonarQubePointGenerator.java:109)
	at jenkinsci.plugins.influxdb.InfluxDbPublicationService.perform(InfluxDbPublicationService.java:246)
	at jenkinsci.plugins.influxdb.global.GlobalRunListener.onCompleted(GlobalRunListener.java:75)
	at hudson.model.listeners.RunListener.fireCompleted(RunListener.java:208)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:625)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$800(WorkflowRun.java:138)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:1045)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1473)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:491)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:38)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:139)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

      It looks like the new SonarQube integration is at fault and there is no way to turn it off.

      A downgrade is not a possible solution for us, since we run InfluxDB 2.1.1 where authentication is broken with older plugin versions.

          [JENKINS-67718] InfluxDB Plugin broken since 3.1 update on docker: `java.nio.file.AccessDeniedException: etc/ssl/private`

          Philipp Moeller added a comment -

          We also enable the plugin option "Expose Exceptions" which does not seem to have an effect anymore.

          Philipp Moeller added a comment - We also enable the plugin option "Expose Exceptions" which does not seem to have an effect anymore.

          Philipp Moeller added a comment -

          After having a look at the code I suspect that this is going to require some rework. The code tries to read report from the master file system. This is not going to work for all agent use-cases.

          Philipp Moeller added a comment - After having a look at the code I suspect that this is going to require some rework. The code tries to read report from the master file system. This is not going to work for all agent use-cases.

          Aleksi Simell added a comment - - edited

          Hi pmr, I just pushed a potential fix to https://github.com/jenkinsci/influxdb-plugin/tree/JENKINS-67718.

          Is it possible for you to compile this manually and test it? I'm not quite sure how to reproduce this issue myself.

          Aleksi Simell added a comment - - edited Hi pmr , I just pushed a potential fix to https://github.com/jenkinsci/influxdb-plugin/tree/JENKINS-67718 . Is it possible for you to compile this manually and test it? I'm not quite sure how to reproduce this issue myself.

          Philipp Moeller added a comment -

          Hi aleksisimell,

          I don't think your change is going to help. This will still traverse the entire filesystem on the master which we absolutely do not want. This could possibly even find a sonarqube report in some completely unrelated workspace (if we had builds running on master). I think the plugin should just do nothing for now when `env.get("WORKSPACE", "") == ""` or you should revert https://github.com/jenkinsci/influxdb-plugin/pull/123 and rethink the approach to the problem.

          Philipp Moeller added a comment - Hi aleksisimell , I don't think your change is going to help. This will still traverse the entire filesystem on the master which we absolutely do not want. This could possibly even find a sonarqube report in some completely unrelated workspace (if we had builds running on master). I think the plugin should just do nothing for now when `env.get("WORKSPACE", "") == ""` or you should revert https://github.com/jenkinsci/influxdb-plugin/pull/123 and rethink the approach to the problem.

          Philipp Moeller added a comment -

          Do make another point why this behaviour is not good: CPU usage of our very small jenkins instance has doubled since the plugin update. This is going to be much worse with high-throughput instances affected by this bug.

          Philipp Moeller added a comment - Do make another point why this behaviour is not good: CPU usage of our very small jenkins instance has doubled since the plugin update. This is going to be much worse with high-throughput instances affected by this bug.

          Aleksi Simell added a comment -

          Hi pmr,

          I managed to reproduce this. Your suggestion actually sounds like it could work and I already implemented a path check. Hopefully that is enough to make this work. I'm not too keen on reverting the whole PR, since the SonarQube integration was indeed broken. The PR fixes that, but then again... introduced this. I don't want to end up in a situation where I need to jump back into a situation that doesn't work.

          Aleksi Simell added a comment - Hi pmr , I managed to reproduce this. Your suggestion actually sounds like it could work and I already implemented a path check. Hopefully that is enough to make this work. I'm not too keen on reverting the whole PR, since the SonarQube integration was indeed broken. The PR fixes that, but then again... introduced this. I don't want to end up in a situation where I need to jump back into a situation that doesn't work.

          Philipp Moeller added a comment -

          aleksisimell I fully understand and appreciate the effort you are putting into the plugin. If you have a fix and release it, we will test it as soon as possible.

          Philipp Moeller added a comment - aleksisimell I fully understand and appreciate the effort you are putting into the plugin. If you have a fix and release it, we will test it as soon as possible.

          Aleksi Simell added a comment -

          Pull request 128 created to fix this issue. Waiting for the Jenkins checks to pass before merging. I will make the release immediately after successfully merged.

          Aleksi Simell added a comment - Pull request 128 created to fix this issue. Waiting for the Jenkins checks to pass before merging. I will make the release immediately after successfully merged.

          Aleksi Simell added a comment -

          InfluxDB 3.1.1 provides a fix for this. Only the job workspace is now searched when trying to find SonarQube report, so it's a lot faster and shouldn't give the error anymore and instead just skip SonarQube.

          Aleksi Simell added a comment - InfluxDB 3.1.1 provides a fix for this. Only the job workspace is now searched when trying to find SonarQube report, so it's a lot faster and shouldn't give the error anymore and instead just skip SonarQube.

          Max Jung added a comment -

          aleksisimell thanks, now it works!

          Max Jung added a comment - aleksisimell  thanks, now it works!

            aleksisimell Aleksi Simell
            pmr Philipp Moeller
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: