Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-54833

Warnings Parser fails parsing log

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • warnings-ng-plugin
    • None
    • Jenkins ver. 2.138.3 (Docker Hub)
      Wanings NG 1.0.0-beta5

      The log is attached, this is a freestyle job and 2 warnings parsers are configured (Make + GCC and a custom Groovy parser).

      After the build, the plugin fails with the following output, while also marking the build as failure.

      Archiving artifacts
Skipping issues blame since Git is the only supported SCM up to now.
[GNU Make + GNU C Compiler (gcc)] Searching for all files in '/home/build/jenkins/jenkins/workspace/poe-baseline.binaries.tfs' that match the pattern 'build.log'
[GNU Make + GNU C Compiler (gcc)] -> found 1 file
[GNU Make + GNU C Compiler (gcc)] Successfully parsed file /home/build/jenkins/jenkins/workspace/poe-baseline.binaries.tfs/build.log
[GNU Make + GNU C Compiler (gcc)] -> found 3 issues (skipped 2022 duplicates)
ERROR: Build step failed with exception
java.nio.charset.MalformedInputException: Input length = 1
	at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:161)
	at java.io.BufferedReader.readLine(BufferedReader.java:324)
	at java.io.BufferedReader.readLine(BufferedReader.java:389)
	at java.io.BufferedReader$1.hasNext(BufferedReader.java:571)
Also:   hudson.remoting.Channel$CallSiteStackTrace: Remote call to des_build
		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741)
		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:357)
		at hudson.remoting.Channel.call(Channel.java:955)
		at hudson.FilePath.act(FilePath.java:1071)
		at hudson.FilePath.act(FilePath.java:1060)
		at io.jenkins.plugins.analysis.core.steps.IssuesScanner.postProcess(IssuesScanner.java:154)
		at io.jenkins.plugins.analysis.core.steps.IssuesScanner.scanInWorkspace(IssuesScanner.java:99)
		at io.jenkins.plugins.analysis.core.steps.IssuesScanner.scan(IssuesScanner.java:78)
		at io.jenkins.plugins.analysis.core.steps.IssuesRecorder.scanWithTool(IssuesRecorder.java:590)
		at io.jenkins.plugins.analysis.core.steps.IssuesRecorder.record(IssuesRecorder.java:555)
		at io.jenkins.plugins.analysis.core.steps.IssuesRecorder.perform(IssuesRecorder.java:529)
		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
		at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690)
		at hudson.model.Build$BuildExecution.post2(Build.java:186)
		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635)
		at hudson.model.Run.execute(Run.java:1844)
		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
		at hudson.model.ResourceController.execute(ResourceController.java:97)
		at hudson.model.Executor.run(Executor.java:429)
Caused: java.io.UncheckedIOException
	at java.io.BufferedReader$1.hasNext(BufferedReader.java:574)
	at java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1811)
	at java.util.Spliterators$1Adapter.hasNext(Spliterators.java:681)
	at edu.hm.hafner.analysis.FullTextFingerprint.extractContext(FullTextFingerprint.java:111)
	at edu.hm.hafner.analysis.FullTextFingerprint.createFingerprint(FullTextFingerprint.java:84)
	at edu.hm.hafner.analysis.FullTextFingerprint.compute(FullTextFingerprint.java:73)
	at edu.hm.hafner.analysis.FingerprintGenerator.run(FingerprintGenerator.java:35)
	at io.jenkins.plugins.analysis.core.steps.IssuesScanner$ReportPostProcessor.createFingerprints(IssuesScanner.java:269)
	at io.jenkins.plugins.analysis.core.steps.IssuesScanner$ReportPostProcessor.invoke(IssuesScanner.java:208)
	at io.jenkins.plugins.analysis.core.steps.IssuesScanner$ReportPostProcessor.invoke(IssuesScanner.java:177)
	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3085)
	at hudson.remoting.UserRequest.perform(UserRequest.java:212)
	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
	at hudson.remoting.Request$2.run(Request.java:369)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Build step 'Record compiler warnings and static analysis results' marked build as failure

        1. build.tar.xz
          42 kB

          [JENKINS-54833] Warnings Parser fails parsing log

          Ulli Hafner added a comment -

          What encoding did you specify for your source files?

          Ulli Hafner added a comment - What encoding did you specify for your source files?

          Norbert Lange added a comment -

          I specified nothing, and the log consists entirely of ASCII Characters. Specifying UTF-8 did not make a difference either.

          I run the Jenkins server on musl (jenkins/jenkins:2.138.3-alpine from docker hub), while the builder is based on glibc (debian 8) and has locale/encoding "en_US.UTF-8". musl has some differences to glibc when it comes to locale handling (and no "locale" tool), maybe that's relevant.

           

           

          Norbert Lange added a comment - I specified nothing, and the log consists entirely of ASCII Characters. Specifying UTF-8 did not make a difference either. I run the Jenkins server on musl (jenkins/jenkins:2.138.3-alpine from docker hub), while the builder is based on glibc (debian 8) and has locale/encoding "en_US.UTF-8". musl has some differences to glibc when it comes to locale handling (and no "locale" tool), maybe that's relevant.    

          Norbert Lange added a comment -

          Hmm, just to add: do you try to parse the source-code files? What happens if filename relates not to a text-file but an executable/DSO (like with linker-warnings/errors).

          I tried setting up a "fake-build" which just outputs the log I attached, no source-code involved at all. The issue stays the same, so atleast my problem is not related to the source-code encodings.

          Norbert Lange added a comment - Hmm, just to add: do you try to parse the source-code files? What happens if filename relates not to a text-file but an executable/DSO (like with linker-warnings/errors). I tried setting up a "fake-build" which just outputs the log I attached, no source-code involved at all. The issue stays the same, so atleast my problem is not related to the source-code encodings.

          Ulli Hafner added a comment -

          Yes, the exception occurs while reading your source files. That is the reason for two different encoding settings. One for the report and one for your sources.

          Ulli Hafner added a comment - Yes, the exception occurs while reading your source files. That is the reason for two different encoding settings. One for the report and one for your sources.

          Norbert Lange added a comment -

          I changes the source encoding and I now understand the issue (the new fake job still tried to read the files from the real jobs workspace). Some thoughts/inputs:

          1) The log contained filenames that did not reside in the workspace and the plugin apparently does try to read and decode them, even if it then decides to do nothing with them. Means it bugged out for no reason

          [GNU Make + GNU C Compiler (gcc)] Copying affected files to Jenkins' build folder /var/jenkins_home/jobs/ztest/builds/19
[GNU Make + GNU C Compiler (gcc)] -> 0 copied, 2 not in workspace, 0 not-found, 0 with I/O error

          2) If the filename points to a exe, DSO or object file, then no encoding setting will help - parsing will always fail (like for linker warnings). I have some custom messages about the object format and a fitting parser - not setting the linenumber or setting it to a negative value could indicate to skip source-file parsing,

          3) failing to parse a file should not be an error (just as failing to copying the files is not).

           

          Norbert Lange added a comment - I changes the source encoding and I now understand the issue (the new fake job still tried to read the files from the real jobs workspace). Some thoughts/inputs: 1) The log contained filenames that did not reside in the workspace and the plugin apparently does try to read and decode them, even if it then decides to do nothing with them. Means it bugged out for no reason [GNU Make + GNU C Compiler (gcc)] Copying affected files to Jenkins' build folder /var/jenkins_home/jobs/ztest/builds/19 [GNU Make + GNU C Compiler (gcc)] -> 0 copied, 2 not in workspace, 0 not-found, 0 with I/O error 2) If the filename points to a exe, DSO or object file, then no encoding setting will help - parsing will always fail (like for linker warnings). I have some custom messages about the object format and a fitting parser - not setting the linenumber or setting it to a negative value could indicate to skip source-file parsing, 3) failing to parse a file should not be an error (just as failing to copying the files is not).  

          Ulli Hafner added a comment -

          1) files outside of the workspace are not shown as this would be a security risk
          2) 3) actually I am catching some exceptions already, seems that I missed some (see JENKINS-54389). On an exception a fallback fingerprint will be used. That should work for exe files as well. Maybe Ishould use a hash of the whole file in such cases as fallback.

          Ulli Hafner added a comment - 1) files outside of the workspace are not shown as this would be a security risk 2) 3) actually I am catching some exceptions already, seems that I missed some (see JENKINS-54389 ). On an exception a fallback fingerprint will be used. That should work for exe files as well. Maybe Ishould use a hash of the whole file in such cases as fallback.

          Norbert Lange added a comment -

          ad. 1 What I meant is that you don't show the file, but still try to parse it. ie. decide first if you have any use for the file, do the work only in that case =)

          Norbert Lange added a comment - ad. 1 What I meant is that you don't show the file, but still try to parse it. ie. decide first if you have any use for the file, do the work only in that case =)

          Ulli Hafner added a comment -

          Hmm, I don’t think that generating a hash code to track new warnings is a security risk so I handle those files differently in these two separate post processing steps. Do you think it is necessary to prevent access to those files in any cases?

          Ulli Hafner added a comment - Hmm, I don’t think that generating a hash code to track new warnings is a security risk so I handle those files differently in these two separate post processing steps. Do you think it is necessary to prevent access to those files in any cases?

          Norbert Lange added a comment -

          Its not necessary because of security, but rather efficiency and generally avoiding issues (like triggering this bug in files that could simply be ignored). For example I would guess that for builds on docker agents you might try to parse/hash unrelated files from the host and not those inside the container?
          The less you do, the less trouble there can be. 

          Norbert Lange added a comment - Its not necessary because of security, but rather efficiency and generally avoiding issues (like triggering this bug in files that could simply be ignored). For example I would guess that for builds on docker agents you might try to parse/hash unrelated files from the host and not those inside the container? The less you do, the less trouble there can be. 

            drulli Ulli Hafner
            nolange79 Norbert Lange
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: