Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-24171

BFA is blindly adding text to gzipped log

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      Jenkins has ability to diplay gunzip gzipped logs on the fly (https://issues.jenkins-ci.org/browse/JENKINS-2551) though doesn't natively compress the logs. The BFA doesn't expect that the logs can be compressed and appends a plaintext to the end log:

      1. diff <(hexdump -C /var/lib/jenkins/jobs/sync-images-rhel-7v0/builds/25/log.gz.previous) <(hexdump -C /var/lib/jenkins/jobs/sync-images-rhel-7v0/builds/25/log.gz)
        2800,2801c2800,2805
        < 0000aef0 07 81 6b fd bf f2 2a 01 00 |..k...*..|
        < 0000aef9

        > 0000aef0 07 81 6b fd bf f2 2a 01 00 5b 42 46 41 5d 20 53 |..k...*..[BFA] S|
        > 0000af00 63 61 6e 6e 69 6e 67 20 62 75 69 6c 64 20 66 6f |canning build fo|
        > 0000af10 72 20 6b 6e 6f 77 6e 20 63 61 75 73 65 73 2e 2e |r known causes..|
        > 0000af20 2e 0a 2e 2e 0a 5b 42 46 41 5d 20 44 6f 6e 65 2e |.....[BFA] Done.|
        > 0000af30 20 32 73 0a | 2s.|
        > 0000af34

      This then breaks the displaying of the logs in the sense that every GET results in one core continuously loaded in endless loop.

      "Handling GET /job/sync-images-rhel-7v0/24/console : RequestHandlerThread10" daemon prio=10 tid=0x00007f55c400b000 nid=0x3b4e runnable [0x00007f56077f8000]
      java.lang.Thread.State: RUNNABLE
      at org.kohsuke.stapler.framework.io.LargeText$GzipAwareSession.skip(LargeText.java:437)
      at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:211)
      at hudson.console.AnnotatedLargeText.writeHtmlTo(AnnotatedLargeText.java:156)
      at hudson.model.Run.writeLogTo(Run.java:1351)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.commons.jexl.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:258)
      at org.apache.commons.jexl.parser.ASTMethod.execute(ASTMethod.java:104)
      at org.apache.commons.jexl.parser.ASTReference.execute(ASTReference.java:83)
      at org.apache.commons.jexl.parser.ASTReference.value(ASTReference.java:57)
      at org.apache.commons.jexl.parser.ASTReferenceExpression.value(ASTReferenceExpression.java:51)
      at org.apache.commons.jexl.ExpressionImpl.evaluate(ExpressionImpl.java:80)
      at hudson.ExpressionFactory2$JexlExpression.evaluate(ExpressionFactory2.java:74)
      at org.apache.commons.jelly.parser.EscapingExpression.evaluate(EscapingExpression.java:24)
      at org.apache.commons.jelly.impl.ExpressionScript.run(ExpressionScript.java:66)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.WhitespaceTag.doTag(WhitespaceTag.java:48)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.OtherwiseTag.doTag(OtherwiseTag.java:41)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.TagSupport.invokeBody(TagSupport.java:161)
      at org.apache.commons.jelly.tags.core.ChooseTag.doTag(ChooseTag.java:38)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$1.run(CoreTagLibrary.java:98)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.CallTagLibScript$1.run(CallTagLibScript.java:99)
      at org.apache.commons.jelly.tags.define.InvokeBodyTag.doTag(InvokeBodyTag.java:91)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.kohsuke.stapler.jelly.ReallyStaticTagLibrary$1.run(ReallyStaticTagLibrary.java:99)
      at org.apache.commons.jelly.impl.ScriptBlock.run(ScriptBlock.java:95)
      at org.apache.commons.jelly.tags.core.CoreTagLibrary$2.run(CoreTagLibrary.java:105)
      at org.kohsuke.stapler.jelly.CallTagLibScript.run(CallTagLibScript.java:120)
      at org.kohsuke.stapler.jelly.CompressTag.doTag(CompressTag.java:44)
      at org.apache.commons.jelly.impl.TagScript.run(TagScript.java:269)
      at org.kohsuke.stapler.jelly.JellyViewScript.run(JellyViewScript.java:81)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:63)
      at org.kohsuke.stapler.jelly.DefaultScriptInvoker.invokeScript(DefaultScriptInvoker.java:53)
      at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:95)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
      at org.kohsuke.stapler.MetaClass$12.dispatch(MetaClass.java:390)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
      at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:248)
      at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
      at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:728)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:858)
      at org.kohsuke.stapler.Stapler.invoke(Stapler.java:631)
      at org.kohsuke.stapler.Stapler.service(Stapler.java:225)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
      at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
      at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:96)
      at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:88)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:48)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
      at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.ui.basicauth.BasicProcessingFilter.doFilter(BasicProcessingFilter.java:174)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at jenkins.security.ApiTokenFilter.doFilter(ApiTokenFilter.java:79)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
      at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
      at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
      at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
      at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:164)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:46)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
      at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
      at org.eclipse.jetty.server.Server.handle(Server.java:370)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
      at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:949)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1011)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:644)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
      at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
      at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
      at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
      at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:745)

        Attachments

          Issue Links

            Activity

            Hide
            danielbeck Daniel Beck added a comment -

            Could you explain how you ended up with a gzipped build log? Or in general, how this issue can be reproduced?


            From a superficial BFA source code reading, it seems to do everything right.

            While the log file could be gzipped, it's safe to assume TaskListener.getLogger() (a PrintStream) is handling this transparently.

            Show
            danielbeck Daniel Beck added a comment - Could you explain how you ended up with a gzipped build log? Or in general, how this issue can be reproduced? From a superficial BFA source code reading, it seems to do everything right. While the log file could be gzipped, it's safe to assume TaskListener.getLogger() (a PrintStream) is handling this transparently.
            Hide
            jhenner Jaroslav Henner added a comment - - edited

            We are just using gzip on the log files. As written in the https://issues.jenkins-ci.org/browse/JENKINS-2551. Something like
            # gzip /var/lib/jenkins/jobs/sync-images-rhel-7v0/builds/25/log

            Show
            jhenner Jaroslav Henner added a comment - - edited We are just using gzip on the log files. As written in the https://issues.jenkins-ci.org/browse/JENKINS-2551 . Something like # gzip /var/lib/jenkins/jobs/sync-images-rhel-7v0/builds/25/log
            Hide
            danielbeck Daniel Beck added a comment -

            Assuming to be an issue with BFA opening log file non-gzipped until proven otherwise.

            Show
            danielbeck Daniel Beck added a comment - Assuming to be an issue with BFA opening log file non-gzipped until proven otherwise.
            Hide
            jhenner Jaroslav Henner added a comment -

            Daniel, I don't really know how this "feature" it is implemented, but I smell it is problem on both sides

            1. BFA should not append plaintext to compressed stream
            2. Jenkins should not hang when displaying gzipped log with trailing garbage at the end.
            Show
            jhenner Jaroslav Henner added a comment - Daniel, I don't really know how this "feature" it is implemented, but I smell it is problem on both sides BFA should not append plaintext to compressed stream Jenkins should not hang when displaying gzipped log with trailing garbage at the end.
            Hide
            danielbeck Daniel Beck added a comment - - edited

            Makes sense. Probably needs to throw on 0 returned from gz.skip(int) unless the parameter is also 0.

            However this seems to be a separate issue (robustness in the face of invalid input) and should be filed separately, as the solutions are independent (i.e. fixing BFA does not seem to depend on core fixes or something like that).

            Show
            danielbeck Daniel Beck added a comment - - edited Makes sense. Probably needs to throw on 0 returned from gz.skip(int) unless the parameter is also 0. However this seems to be a separate issue (robustness in the face of invalid input) and should be filed separately, as the solutions are independent (i.e. fixing BFA does not seem to depend on core fixes or something like that).
            Hide
            feindt Jan Feindt added a comment -

            Possibly I ran into the same problem: using "Compress Build Log Plugin" everything is fine until I choose to rescan the Builds with BFA.
            So I think one solution is to "not change the old build log" in this use case using a NullOutputStream instead of a FileOutputStream with the old build log.

            In com.sonyericsson.jenkins.plugins.bfa.sod.ScanOnDemandTask.scanBuild(AbstractBuild)
            change line
            fos = new FileOutputStream(abstractBuild.getLogFile(), true);
            to
            fos = new NullOutputStream();

            Another solution is to check if file is gzipped and then use NullOutputStream or decompress first, change logfile, compress after BFA is done.

            Show
            feindt Jan Feindt added a comment - Possibly I ran into the same problem: using "Compress Build Log Plugin" everything is fine until I choose to rescan the Builds with BFA. So I think one solution is to "not change the old build log" in this use case using a NullOutputStream instead of a FileOutputStream with the old build log. In com.sonyericsson.jenkins.plugins.bfa.sod.ScanOnDemandTask.scanBuild(AbstractBuild) change line fos = new FileOutputStream(abstractBuild.getLogFile(), true); to fos = new NullOutputStream(); Another solution is to check if file is gzipped and then use NullOutputStream or decompress first, change logfile, compress after BFA is done.
            Hide
            lkraider Paul Eipper added a comment -

            Can we have an option to disable build log output from BFA?

            Show
            lkraider Paul Eipper added a comment - Can we have an option to disable build log output from BFA?
            Hide
            andyjohns1 ajohns added a comment -

            Any chance this bug could be addressed in some way? Would love to use BFA plug in, but this ancient bug is a dealbreaker.

            Show
            andyjohns1 ajohns added a comment - Any chance this bug could be addressed in some way? Would love to use BFA plug in, but this ancient bug is a dealbreaker.

              People

              Assignee:
              t_westling Tomas Westling
              Reporter:
              jhenner Jaroslav Henner
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

                Dates

                Created:
                Updated: