Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-70122

Jenkins log full with rundeck exceptions

XMLWordPrintable

    • Icon: New Feature New Feature
    • Resolution: Unresolved
    • Icon: Critical Critical
    • rundeck-plugin
    • None

      Our jenkins has only 1 active rundeck job which is triggered many times during each day.
      The issue is our jenkins.log file is full with below exceptions:

      java.io.InterruptedIOException: interrupted
	at okio.Timeout.throwIfReached(Timeout.java:146)
	at okio.Okio$1.write(Okio.java:76)
	at okio.AsyncTimeout$1.write(AsyncTimeout.java:180)
	at okio.RealBufferedSink.flush(RealBufferedSink.java:224)
	at okhttp3.internal.http1.Http1ExchangeCodec.finishRequest(Http1ExchangeCodec.java:190)
	at okhttp3.internal.connection.Exchange.finishRequest(Exchange.java:101)
	at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:86)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at org.rundeck.client.util.StaticHeaderInterceptor.intercept(StaticHeaderInterceptor.java:38)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at org.rundeck.client.util.StaticHeaderInterceptor.intercept(StaticHeaderInterceptor.java:38)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
	at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
	at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
	at okhttp3.RealCall.execute(RealCall.java:81)
	at retrofit2.OkHttpCall.execute(OkHttpCall.java:204)
	at org.jenkinsci.plugins.rundeck.client.RundeckClientManager.getOutput(RundeckClientManager.java:89)
	at org.jenkinsci.plugins.rundeck.client.RundeckClientManager.getOutput(RundeckClientManager.java:83)
	at org.jenkinsci.plugins.rundeck.RunDeckLogTail$RunDeckLogTailIterator.hasNext(RunDeckLogTail.java:99)
	at org.jenkinsci.plugins.rundeck.RundeckNotifier.waitTailingRundeckLogsAndReturnExecution(RundeckNotifier.java:314)
	at org.jenkinsci.plugins.rundeck.RundeckNotifier.notifyRundeck(RundeckNotifier.java:279)
	at org.jenkinsci.plugins.rundeck.RundeckNotifier.perform(RundeckNotifier.java:164)
	at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:100)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:70)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

      This seems to be emanating from:
      https://github.com/jenkinsci/rundeck-plugin/blob/master/src/main/java/org/jenkinsci/plugins/rundeck/RunDeckLogTail.java#L120
      The thing is our jobs run perfectly fine, despite these exceptions and my opinion is that when tailing the log fails occassionally, next iterations resolve it which means its an intermittent issue.
      It would be better to have these stacktraces logged via logger than send them directly to console, so we can isloate these errors from jenkins.log file.

          [JENKINS-70122] Jenkins log full with rundeck exceptions

          AB S created issue -
          AB S made changes -
          Description Original: Our jenkins has only 1 active rundeck job which is triggered many times during each day.
          The issue is our jenkins.log file is full with below exceptions:

          {code:java}
          java.io.InterruptedIOException: interrupted
          at okio.Timeout.throwIfReached(Timeout.java:146)
          at okio.Okio$1.write(Okio.java:76)
          at okio.AsyncTimeout$1.write(AsyncTimeout.java:180)
          at okio.RealBufferedSink.flush(RealBufferedSink.java:224)
          at okhttp3.internal.http1.Http1ExchangeCodec.finishRequest(Http1ExchangeCodec.java:190)
          at okhttp3.internal.connection.Exchange.finishRequest(Exchange.java:101)
          at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:86)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at org.rundeck.client.util.StaticHeaderInterceptor.intercept(StaticHeaderInterceptor.java:38)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at org.rundeck.client.util.StaticHeaderInterceptor.intercept(StaticHeaderInterceptor.java:38)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
          at okhttp3.RealCall.execute(RealCall.java:81)
          at retrofit2.OkHttpCall.execute(OkHttpCall.java:204)
          at org.jenkinsci.plugins.rundeck.client.RundeckClientManager.getOutput(RundeckClientManager.java:89)
          at org.jenkinsci.plugins.rundeck.client.RundeckClientManager.getOutput(RundeckClientManager.java:83)
          at org.jenkinsci.plugins.rundeck.RunDeckLogTail$RunDeckLogTailIterator.hasNext(RunDeckLogTail.java:99)
          at org.jenkinsci.plugins.rundeck.RundeckNotifier.waitTailingRundeckLogsAndReturnExecution(RundeckNotifier.java:314)
          at org.jenkinsci.plugins.rundeck.RundeckNotifier.notifyRundeck(RundeckNotifier.java:279)
          at org.jenkinsci.plugins.rundeck.RundeckNotifier.perform(RundeckNotifier.java:164)
          at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
          at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:100)
          at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:70)
          at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
          {code}

          This seems to be emanating from:
          https://github.com/jenkinsci/rundeck-plugin/blob/master/src/main/java/org/jenkinsci/plugins/rundeck/RunDeckLogTail.java#L120
          The thing is our jobs run perfectly fine, despite these exceptions and my opinion is that when tailing the log fails occassionally, next iterations resolve it so its intermittent issue.
          It would be better to have these stacktraces logged via logger than direct them directly to console, so we can suppress/isloate these errors from jenkins.log file.           		New: Our jenkins has only 1 active rundeck job which is triggered many times during each day.
          The issue is our jenkins.log file is full with below exceptions:

          {code:java}
          java.io.InterruptedIOException: interrupted
          at okio.Timeout.throwIfReached(Timeout.java:146)
          at okio.Okio$1.write(Okio.java:76)
          at okio.AsyncTimeout$1.write(AsyncTimeout.java:180)
          at okio.RealBufferedSink.flush(RealBufferedSink.java:224)
          at okhttp3.internal.http1.Http1ExchangeCodec.finishRequest(Http1ExchangeCodec.java:190)
          at okhttp3.internal.connection.Exchange.finishRequest(Exchange.java:101)
          at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.java:86)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:43)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:94)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:88)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at org.rundeck.client.util.StaticHeaderInterceptor.intercept(StaticHeaderInterceptor.java:38)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at org.rundeck.client.util.StaticHeaderInterceptor.intercept(StaticHeaderInterceptor.java:38)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
          at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
          at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
          at okhttp3.RealCall.execute(RealCall.java:81)
          at retrofit2.OkHttpCall.execute(OkHttpCall.java:204)
          at org.jenkinsci.plugins.rundeck.client.RundeckClientManager.getOutput(RundeckClientManager.java:89)
          at org.jenkinsci.plugins.rundeck.client.RundeckClientManager.getOutput(RundeckClientManager.java:83)
          at org.jenkinsci.plugins.rundeck.RunDeckLogTail$RunDeckLogTailIterator.hasNext(RunDeckLogTail.java:99)
          at org.jenkinsci.plugins.rundeck.RundeckNotifier.waitTailingRundeckLogsAndReturnExecution(RundeckNotifier.java:314)
          at org.jenkinsci.plugins.rundeck.RundeckNotifier.notifyRundeck(RundeckNotifier.java:279)
          at org.jenkinsci.plugins.rundeck.RundeckNotifier.perform(RundeckNotifier.java:164)
          at jenkins.tasks.SimpleBuildStep.perform(SimpleBuildStep.java:123)
          at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:100)
          at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:70)
          at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
          {code}

          This seems to be emanating from:
          https://github.com/jenkinsci/rundeck-plugin/blob/master/src/main/java/org/jenkinsci/plugins/rundeck/RunDeckLogTail.java#L120
          The thing is our jobs run perfectly fine, despite these exceptions and my opinion is that when tailing the log fails occassionally, next iterations resolve it which means its an intermittent issue.
          It would be better to have these stacktraces logged via logger than send them directly to console, so we can isloate these errors from jenkins.log file.

            vbehar Vincent Behar
            abmaster AB S
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: