FYI, I can't reproduce it at the moment...

I'm getting this – or something very similar to it – myself, running a pre-release build of Timestamper 1.18.11 (from JENKINS-54081):

Avoid calling getLogFile on **** #31
java.lang.UnsupportedOperationException
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getLogFile(WorkflowRun.java:1064)
	at hudson.plugins.warnings.WarningsPublisher.parseConsoleLog(WarningsPublisher.java:371)
	at hudson.plugins.warnings.WarningsPublisher.perform(WarningsPublisher.java:312)
	at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:69)
	at hudson.plugins.analysis.core.HealthAwareRecorder.perform(HealthAwareRecorder.java:298)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:80)
	at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:67)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1$1.call(SynchronousNonBlockingStepExecution.java:50)
	at hudson.security.ACL.impersonate(ACL.java:290)
	at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution$1.run(SynchronousNonBlockingStepExecution.java:47)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I don't think it's actually Timestamper, but rather the changes in Pipeline Job 2.26.

I'm seeing this as well in the Jenkins log. It seems to be an intentional message from workflow-job-plugin.

Now I'll need to reconsider how I test specific bugs with Pipeline jobs. I've used the groovy post build plugin to read the logfile from the manager object, but that is apparently no longer a preferred action.

Any hints of the preferred replacement for getLogFile()? That message was added to workflow-job-plugin 2 years ago. I'm a little ashamed I didn't detect the message sooner, though looking at the tags which include that commit, it seems to have first appeared in workflow-job-plugin 2.26, not in any of the preceding plugin releases. Maybe I'm not as inattentive as I thought...

That message was added to workflow-job-plugin 2 years ago .... it seems to have first appeared in workflow-job-plugin 2.26, not in any of the preceding plugin releases.

markewaite The message was added in this PR sometime in its 2-year lifespan, but only recently merged and released in workflow-job 2.26. The log-handling rewrite for JEP-210 has changed the nature of various log-related methods. Previously, `getLogFile` always returned a reference to a file that already existed on disk, but now with external logging, that call has to read the entire log (which may involve network calls) and write it to a temporary file which is then returned. The best alternative that I know of would be to call WorkflowRun#getLogText and then handle it directly, but depending on your use case, maybe WorkflowRun#getLogReader or WorkflowRun#getLog(int) would be useful as well.

For the timestamper case, this warning is displayed only if you use the View as plain text link—which AFAIK never worked for Pipeline builds anyway. I checked whether it was straightforward to suppress this link in the patch for JENKINS-48344, but unfortunately it is not, due to the JavaScript tricks involved.

The call from WarningsPublisher.parseConsoleLog is unrelated.

As are calls from GroovyPostbuildRecorder.

jglick I don't quite understand the comment where you said:

The call from WarningsPublisher.parseConsoleLog is unrelated.

As are calls from GroovyPostbuildRecorder.

I believe I'm using the GroovyPostbuildRecorder when I use the manage object provided by the groovy post build plugin to call logContains(). Should I be using something else, or just accept the extra message in the Jenkins log and hope for a fix from the groovy post build plugin?

just accept the extra message in the Jenkins log and hope for a fix from the groovy post build plugin

This.

I've added the linenumbers-plugin here, as it too has this issue now that workflow-job 2.26 is released. In my case, it seems to generate hundreds to thousands of copies of the original log file, all named with "deprecated" (e.g., deprecated5973622952721928030.log). I haven't figured out what causes the particular quantity of these files (I've seen over 6000 created by a single build), but it seems perhaps related to how the build was executed – triggering a build directly generates 1, but triggering a build from another build (in my case, a Pipeline build that spawns a number of builds to do a regression test) generates hundreds to thousands of copies. These all coincide with log messages in the Jenkins log like:

java.lang.UnsupportedOperationException
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getLogFile(WorkflowRun.java:1082)
	at org.jenkinsci.plugins.linenumbers.LineNumbersAnnotator.annotate(LineNumbersAnnotator.java:54)
	at hudson.console.ConsoleAnnotator$1Aggregator.annotate(ConsoleAnnotator.java:114)
	at hudson.console.ConsoleAnnotationOutputStream.eol(ConsoleAnnotationOutputStream.java:145)
	at hudson.console.LineTransformationOutputStream.eol(LineTransformationOutputStream.java:60)
	at hudson.console.LineTransformationOutputStream.write(LineTransformationOutputStream.java:56)
	at java.io.FilterOutputStream.write(FilterOutputStream.java:77)

Given the impact to the master node to store this number of files (6000 files = 14GB for a single build, in my case), and unless I know it's something other than this bug that causes it, I'd think it makes sense to raise to Major (if not Blocker), if the affected plugins don't have their own issues tracking this problem.

Sure enough, uninstalling the linenumbers-plugin and rerunning my problematic build generated just 1 "deprecated*.log" file, and I have only one "Avoid calling getLogFile" error in the logs, which comes from hudson.plugins.warnings.WarningsPublisher.parseConsoleLog.

That one could be solved with a simple s/getLogFile/getLogText/ I think.

 I can look at having WorkflowRun avoid creating multiple temp files like this.

Pipeline Job Plugin version 2.29 was just released with a fix that prevents a new temporary file from being created each time WorkflowRun#getLogFile is called. The warning in the logs will need to be fixed by the plugins that are calling Run#getLogFile themselves, likely by switching to using Run#getLogText instead..

Best to leave this open at least until distinct issues are filed for such plugins.

Should Run#getLogText be used for pipelines and freestyle jobs? Or just for pipelines?

Are there some recommendation on how to process the AnnotatedLargeText instance? In the warnings plugin I need to scan the log line by line with a parser. One way I see is to use writeLogTo and write the output stream to an intermediate file. Or is this not recommended?

You may use Run.getLogText, or really anything besides getLogFile, on any build, Pipeline or freestyle. In your case the simplest fix looks to be to just call Run.getLogReader rather than using ParserRegistry.createReader(File).

I see, that looks quite easy. Is there a direct way to access the log on the agent? The I could run all warnings parsers on the agent.

Is there a direct way to access the log on the agent?

No. If you want to run warning parsers on the agent side, what you would rather want is a block-scoped step (perhaps via SimpleBuildWrapper) that uses a ConsoleLogFilter to track warnings as they are emitted, rather than parsing them later. (This would also be far more efficient in conjunction with cloud log storage.) But it is a significant rewrite to do that, and of course requires a different kind of project configuration as well.

I removed the component 'warnings' since I fixed it in the beta of the warnings-ng plugin now.

FTR we're having this issue on XWiki. Every day we get 1TB (yes, terabyte not gigabyte!) of jenkins.log files because of the following filling the logs:

May 21, 2019 8:09:53 AM org.jenkinsci.plugins.workflow.job.WorkflowRun getLogFile
WARNING: Avoid calling getLogFile on XWiki/xwiki-platform/master #1822
java.lang.UnsupportedOperationException
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getLogFile(WorkflowRun.java:1087)
	at org.jvnet.hudson.plugins.groovypostbuild.GroovyPostbuildRecorder$BadgeManager.logContains(GroovyPostbuildRecorder.java:257)

I'll try replace our call to logContains() and reimplement it using Run.getLogText or Run.getLogReader in the meantime.

Any idea why the following pipeline:

node() {
    echo "test"
    def reader = currentBuild.rawBuild.getLogReader()
    reader.eachLine() { line ->
        echo "Line: ${line}"
        if (line.matches(".*test.*")) {
            echo "got it!"
        }
    }
}

Only prints the first log line:

Started by user Vincent Massol
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/Test
[Pipeline] {
[Pipeline] echo
test
[Pipeline] echo
Line: Started by user Vincent Massol
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

?

Thanks

EDIT: see https://gitter.im/jenkinsci/jenkins?at=5ce3c760ad024978c6026d6f Seems it could be related to JENKINS-46988

ok works with:

    new BufferedReader(currentBuild.rawBuild.getLogReader()).with { br ->
        def line = null
        line = br.readLine()
        while (line != null) {
            ...
            line = br.readLine()
        }
    }

jglick I'd love your help on this. I reimplemented BadgeManager#logContains() in our pipeline library in Groovy, since BadgeManager#logContains() calls Run#getLogFile() which emits the warnings in the jenkins logs. The only change I did was to use Run#getLogReader() as in:

private def logContains(regexp)
{
    return contains(currentBuild.rawBuild.getLogReader(), regexp)
}

private def contains(reader, regexp)
{
    def matcher = getMatcher(reader, regexp)
    return (matcher != null) && matcher.matches()
}

private def getMatcher(reader, regexp)
{
    def matcher = null
    new BufferedReader(reader).with { br ->
        def pattern = Pattern.compile(regexp)
        def line = null
        line = br.readLine()
        while (line != null) {
            def m = pattern.matcher(line);
            if (m.matches()) {
                matcher = m
                break
            }
            line = br.readLine()
        }
    }
    return matcher
}

Source here: https://github.com/xwiki/xwiki-jenkins-pipeline/blob/63df1a4a1a4f75e47b97803da614f7fc9db88c39/vars/checkForFalsePositives.groov

However, when we execute it, it's extra slow.... as the log grows in size, it takes substantially more and more time (we run 24 maven builds in a single pipeline job). On the 24th Maven build, it takes 24mn to execute (so 300mn = 5 hours in total) whereas it was taking 28s with BadgeManager#logContains(). I can't figure out why that speed change. Could it be simply because the code logic was moved from Java to Groovy? Or is there something that makes Run#getLogReader() slow?

Thanks a lot for any help on this, I'm struggling to find a workaround (see my other message above, the warnings fills out logs at a rate of over 1TB of logs every day...).

All of these methods ought to be marked @NonCPS.

All of these methods ought to be marked @NonCPS.

jglick hmm, are you saying that the performance issues I see are actually caused by the fact that I didn't mark them @NonCPS? Indeed, didn't think of this. I'm surprised it would make such a difference but maybe that's the case.

I'm surprised it would make such a difference

Orders of magnitude. CPS-transformed code is intended for “glue”: a handful of if-statements and the like. Not actual programming.

The issue in groovy-postbuild is fixed in groovy-postbuild-2.5.
(Removed groovy-postbuild from component/s)
It will be available in the update center in a day.
Please try that.

Awesome, thanks for both of you jglick and ikedam!

Thanks very much ikedam and jglick. I'm now using the groovy-postbuild plugin 2.5 and am not seeing any log messages for my use of the groovy post build plugin to assert that specific strings are detected in the log file of the current build.

Getting similar exceptions in the log:

Oct 24, 2019 8:32:17 PM WARNING org.jenkinsci.plugins.workflow.job.WorkflowRun getLogFileAvoid calling getLogFile on build-packer #8
java.lang.UnsupportedOperationException
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getLogFile(WorkflowRun.java:1088)
	at org.jenkinsci.plugins.compressbuildlog.BuildLogCompressor$CompressBuildlogRunListener.onFinalized(BuildLogCompressor.java:85)
	at hudson.model.listeners.RunListener.fireFinalized(RunListener.java:255)
	at hudson.model.Run.onEndBuilding(Run.java:2000)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.finish(WorkflowRun.java:612)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.access$800(WorkflowRun.java:133)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun$GraphL.onNewHead(WorkflowRun.java:999)
	at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.notifyListeners(CpsFlowExecution.java:1463)
	at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$3.run(CpsThreadGroup.java:458)
	at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$1.run(CpsVmExecutorService.java:37)
	at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:131)
	at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Oct 24, 2019 8:34:35 PM INFO hudson.model.AsyncPeriodicWork$1 run

Jenkins version: 2.201

I don't have groovy-postbuild plugin installed.

Plugin versions.

docker-workflow  1.21
workflow-aggregator 2.6
workflow-api 2.37
workflow-basic-steps 2.18
workflow-cps 2.74
workflow-cps-global-lib 2.15
workflow-durable-task-step 2.34
workflow-job 2.35
workflow-multibranch 2.21
workflow-remote-loader 1.5
workflow-scm-step 2.9
workflow-step-api 2.20
workflow-support 3.3

Any help would much appreciated.

cosbug that seems to be a call reading the log file from org.jenkinsci.plugins.compressbuildlog, the compress build log plugin, probably at line 85. You're welcome to submit a pull request to that repository proposing to correct that plugin. Alternately, you could disable build log compression in your job definitions.

Thank you Mark.

Is there a fix for timestamper-plugin? Since LogFileReader.nextLine() is called for every line when downloading logs from Jenkins, the log is flooded with bogus errors. I took a look at https://github.com/jenkinsci/timestamper-plugin/blob/master/src/main/java/hudson/plugins/timestamper/io/LogFileReader.java#L157 but am at a loss on how to keep the contract of that method while avoiding the call to getLogFile. (Would be nice if Run had a method hasLog()...)

tgr, I have a rough draft of a fix for Timestamper. I plan to post a PR soon and release the fix in the forthcoming Timestamper 1.11.1 release. To help me test this new functionality, would you mind posting the stack trace that you are seeing with the call to getLogFile?

I posted jenkinsci/timestamper-plugin#52, which eliminates the last call to the last call to Run#getLogFile in Timestamper and replaces it with a call to Run#getLogReader.

I removed the timestamper-plugin component from this issue, since a fix has been released in Timestamper 1.11.1.

FYI, the usage-in-plugins job shows that there are still 16 plugins with calls to the deprecated Run#getLogFile method. Most of these plugins haven't been updated in over a year, but I noticed that google-storage, sauce-ondemand, stoplightio-report, and token-macro have all had recent updates, so I added them to this issue's component list.

I removed token macro since the PR was merged and released.

Fix merged in linenumbers plugin 1.3 released 11 months ago.