Details
-
Bug
-
Status: Fixed but Unreleased (View Workflow)
-
Major
-
Resolution: Fixed
-
workflow-plugin 1.12
warnings plugin 4.51
jenkins 1.641
Description
I came up with the following workflow script brooken.groovy, which
compiles a file with clang, and then invokes the warnings plugin.
In the console log, there are warnings, but the warnings plugin detects zero warnings.
I modified the script slightly (fixed.groovy) by inserting a sleep statement before calling the warnings plugin. I re-ran the script and the warnings plugin detected the warnings.
It looks like the workflow-plugin is not providing the console to the warnings plugin when it needs it? Maybe a flush is missing?
Attachments
Issue Links
- is duplicated by
-
-
- Resolved
-
- relates to
-
JENKINS-38381 [JEP-210] Optimize log handling in Pipeline and Durable Task
-
- Resolved
-
-
-
- Open
-
- links to
Activity
Code changed in jenkins
User: Ulli Hafner
Path:
src/test/java/plugins/WarningsPluginTest.java
http://jenkins-ci.org/commit/acceptance-test-harness/d01a8c1546e1a4b2aac6dea692529da83e117bf3
Log:
JENKINS-32191 Added testcase in order to try to expose the bug.
Craig, I just committed a test case with your build step. The test currently passes on my machine, lets see how it runs in https://jenkins.ci.cloudbees.com/job/core/job/acceptance-test-harness-stable/.
Can you still reproduce it on your machine?
Daniel Daugherty
added a comment - I am still seeing this issue with Jenkins 2 and warnings plugin 4.56. The test link above also shows a regression on the test.
Daniel Daugherty
added a comment - I am still seeing this issue with Jenkins 2 and warnings plugin 4.56. The test link above also shows a regression on the test. I also get this error now from time to time. The ATH test case fails sometimes: seems that the output of the pipeline is not correctly flushed in some circumstances.
(I changed the test locally now, seems that I can reproduce it on my machine. I'll push it if I finished a refactoring)
Not sure offhand what the problem is (not aware of any such issue affecting any other plugin), but if WorkflowRun.copyLogs, JENKINS-38381 deletes this.
I wrote another test case that reproduces the problem. Makes parsing of the console log with the warnings plug-in in pipelines quite a lottery. One workaround would be to add a 5 sec sleep directly in a warnings plug-in step.
Probably addressed by JEP-210. If not, please open a fresh issue with steps to reproduce.
Seems to be still problem:
Jenkins LTS with latest plugins.
See
- CI: https://travis-ci.org/jenkinsci/warnings-ng-plugin/builds/502005684?utm_medium=notification&utm_source=email
- Test Case: https://github.com/jenkinsci/warnings-ng-plugin/blob/master/src/test/java/io/jenkins/plugins/analysis/warnings/StepsITest.java#L116
- Enable: https://github.com/jenkinsci/warnings-ng-plugin/commit/a86b5bac02267b30609c548ae838839945a84370
- Disable: https://github.com/jenkinsci/warnings-ng-plugin/commit/d7de3c70228ce89f81dc055bc7126081c5f285ea
Yeah, the full build log is flushed only lazily while the build is running, and getLogReader does not attempt to wait for it. While I continue to recommend using a block-scoped step with ConsoleLogFilter or TaskListenerDecorator to capture console warnings as they are printed, rather than trying to load the log after the fact, you would presumably need to insert a flush call there somewhere. I do not follow the complex code and wrapper APIs well enough to say where.
master branch @ a86b5bac02267b30609c548ae838839945a84370 builds only with -Denforcer.skip=true BTW.
Yeah, the full build log is flushed only lazily while the build is running, and getLogReader does not attempt to wait for it. While I continue to recommend using a block-scoped step with ConsoleLogFilter or TaskListenerDecorator to capture console warnings as they are printed, rather than trying to load the log after the fact, you would presumably need to insert a flush call there somewhere. I do not follow the complex code and wrapper APIs well enough to say where.
Yes, that is still my plan, it is tracked as JENKINS-44450.
master branch @ a86b5bac02267b30609c548ae838839945a84370 builds only with -Denforcer.skip=true BTW.
Hmm, what is the error on your machine? It builds locally fine on my machine with mvn verify, as well as in CI, see https://ci.jenkins.io/job/Plugins/job/warnings-ng-plugin/job/master/228/.
what is the error on your machine?
$ git show
commit a86b5bac02267b30609c548ae838839945a84370 (HEAD -> master, origin/master, origin/HEAD)
Author: Ulli Hafner <ullrich.hafner@gmail.com>
Date: Tue Mar 5 15:23:30 2019 +0100
Revert "Do not wait for console log flush anymore."
This reverts commit d7de3c70228ce89f81dc055bc7126081c5f285ea.
diff --git a/src/main/java/io/jenkins/plugins/analysis/core/model/ReportScanningTool.java b/src/main/java/io/jenkins/plugins/analysis/core/model/ReportScanningTool.java
index e19ab7aa..8362b241 100644
--- a/src/main/java/io/jenkins/plugins/analysis/core/model/ReportScanningTool.java
+++ b/src/main/java/io/jenkins/plugins/analysis/core/model/ReportScanningTool.java
@@ -163,6 +163,8 @@ public abstract class ReportScanningTool extends Tool {
"Static analysis tool %s cannot scan console log output, please define a file pattern",
getActualName());
+ waitForConsoleToFlush(logger);
+
Report consoleReport = new Report();
consoleReport.logInfo("Parsing console log (workspace: '%s')", workspace);
logger.log(consoleReport);
$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
$ mvn validate
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------< io.jenkins.plugins:warnings-ng >-------------------
[INFO] Building Warnings Next Generation Plugin 4.0.0-SNAPSHOT
[INFO] --------------------------------[ hpi ]---------------------------------
[INFO]
[INFO] --- maven-hpi-plugin:2.7:validate (default-validate) @ warnings-ng ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:display-info (display-info) @ warnings-ng ---
[INFO] Maven Version: 3.6.0
[INFO] JDK Version: 1.8.0_191 normalized as: 1.8.0-191
[INFO] OS Info: Arch: amd64 Family: unix Name: linux Version: 4.18.0-15-generic
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (display-info) @ warnings-ng ---
[INFO] Adding ignore: module-info
[INFO] Adding ignore: **/ModuleUtils*
[WARNING] Rule 4: org.apache.maven.plugins.enforcer.RequireUpperBoundDeps failed with message:
Failed while enforcing RequireUpperBoundDeps. The error(s) are [
Require upper bound dependencies error for org.jenkins-ci.plugins.workflow:workflow-step-api:2.13 paths to dependency are:
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.13
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-job:2.16
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.13
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:token-macro:2.5
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.7
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.16
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-cps:2.42
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.13
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:pipeline-stage-step:2.3
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.11
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-durable-task-step:2.16
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.11
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-basic-steps:2.7
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.13
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-job:2.16
+-org.jenkins-ci.plugins.workflow:workflow-api:2.22
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.10
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-job:2.16
+-org.jenkins-ci.plugins.workflow:workflow-support:2.16
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.13
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins.workflow:workflow-scm-step:2.7
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.9
,
Require upper bound dependencies error for org.jenkins-ci.plugins:structs:1.10 paths to dependency are:
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:structs:1.10
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-step-api:2.13
+-org.jenkins-ci.plugins:structs:1.5
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins:structs:1.17
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-cps:2.42
+-org.jenkins-ci.plugins:structs:1.7
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:pipeline-stage-step:2.3
+-org.jenkins-ci.plugins:structs:1.6
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-basic-steps:2.7
+-org.jenkins-ci.plugins:structs:1.6
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-job:2.16
+-org.jenkins-ci.plugins.workflow:workflow-api:2.22
+-org.jenkins-ci.plugins:structs:1.7
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:matrix-project:1.13
+-org.jenkins-ci.plugins:junit:1.20
+-org.jenkins-ci.plugins:structs:1.2
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins:git-client:2.7.3
+-org.jenkins-ci.plugins:structs:1.9
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins:credentials:2.1.18
+-org.jenkins-ci.plugins:structs:1.7
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins:scm-api:2.2.0
+-org.jenkins-ci.plugins:structs:1.9
,
Require upper bound dependencies error for org.jenkins-ci.plugins:scm-api:2.2.0 paths to dependency are:
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins:scm-api:2.2.0
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-cps:2.42
+-org.jenkins-ci.plugins:scm-api:2.2.0 (managed) <-- org.jenkins-ci.plugins:scm-api:2.0.8
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:pipeline-stage-step:2.3
+-org.jenkins-ci.plugins:scm-api:2.2.0 (managed) <-- org.jenkins-ci.plugins:scm-api:2.0.7
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-job:2.16
+-org.jenkins-ci.plugins.workflow:workflow-api:2.22
+-org.jenkins-ci.plugins:scm-api:2.2.0 (managed) <-- org.jenkins-ci.plugins:scm-api:2.2.8
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-job:2.16
+-org.jenkins-ci.plugins.workflow:workflow-support:2.16
+-org.jenkins-ci.plugins:scm-api:2.2.0 (managed) <-- org.jenkins-ci.plugins:scm-api:2.0.8
,
Require upper bound dependencies error for org.jenkins-ci.plugins:mailer:1.18 paths to dependency are:
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins:git:3.9.1
+-org.jenkins-ci.plugins:mailer:1.18 (managed) <-- org.jenkins-ci.plugins:mailer:1.21
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.main:maven-plugin:3.1.2
+-org.jenkins-ci.plugins:mailer:1.18 (managed) <-- org.jenkins-ci.plugins:mailer:1.7
and
+-io.jenkins.plugins:warnings-ng:4.0.0-SNAPSHOT
+-org.jenkins-ci.plugins.workflow:workflow-basic-steps:2.7
+-org.jenkins-ci.plugins:mailer:1.18 (managed) <-- org.jenkins-ci.plugins:mailer:1.7
]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
…
Not sure how it could be working in CI. That there are bad deps is backed up by:
$ mvn -Denforcer.skip -Dtest=InjectedTest clean test … Mar 05, 2019 1:51:33 PM jenkins.InitReactorRunner$1 onAttained INFO: Listed all plugins Mar 05, 2019 1:51:33 PM jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin JDK Tool Plugin v1.1 (jdk-tool) java.io.IOException: JDK Tool Plugin v1.1 failed to load. - You must update Jenkins from v2.89.1 to v2.112 or later to run this plugin. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:626) at hudson.PluginManager$2$1$1.run(PluginManager.java:513) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1065) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Mar 05, 2019 1:51:33 PM jenkins.InitReactorRunner$1 onTaskFailed SEVERE: Failed Loading plugin Jenkins Git plugin v3.9.1 (git) java.io.IOException: Jenkins Git plugin v3.9.1 failed to load. - You must update Jenkins from v2.89.1 to v2.138.3-cb-2 or later to run this plugin. - Pipeline: Step API v2.13 is older than required. To fix, install v2.16 or later. - JDK Tool Plugin v1.1 failed to load. Fix this plugin first. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:626) at hudson.PluginManager$2$1$1.run(PluginManager.java:513) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1065) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Mar 05, 2019 1:51:36 PM jenkins.InitReactorRunner$1 onAttained INFO: Prepared all plugins … [ERROR] Tests run: 107, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 18.044 s <<< FAILURE! - in TestSuite with 107 tests [example: org.jvnet.hudson.test.JellyTestSuiteBuilder$JellyTestSuite@4f25b795] [ERROR] testPluginActive Time elapsed: 0.013 s <<< ERROR! java.lang.Error: Plugin jdk-tool failed to start at org.jvnet.hudson.test.PluginAutomaticTestBuilder$OtherTests.testPluginActive(PluginAutomaticTestBuilder.java:99) at … Caused by: java.io.IOException: JDK Tool Plugin v1.1 failed to load. - You must update Jenkins from v2.89.1 to v2.112 or later to run this plugin. at hudson.PluginWrapper.resolvePluginDependencies(PluginWrapper.java:626) at hudson.PluginManager$2$1$1.run(PluginManager.java:513) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1065) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) ... 3 more [INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] PluginAutomaticTestBuilder$OtherTests.testPluginActive:99 Plugin jdk-tool fai... [INFO] [ERROR] Tests run: 107, Failures: 0, Errors: 1, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE …
Possibly the issue is something to do with the timestamped snapshot dependencies (analysis-model-api.version + analysis-model.version) malfunctioning, and the Maven build pulling in more or less random versions of stuff. I would advise you to switch to JEP-305. It is generally not hard to do and you can bypass a lot of weird issues.
I added a 5sec sleep before I am scanning the console log quite some time ago. Since nobody mentioned that bug again, I think it is save to close this issue even though only a workaround has been applied.
FTR the workaround for the workaround is to not rely on console scanning, but rather to use the report file syntax and be explicit about what it is you wish to scan:
tee('log.txt') { sh 'mvn -B -ntp -Dmaven.test.failure.ignore verify' } recordIssues(tools: [javaDoc(pattern: 'log.txt')])
or
sh 'bash -c "set -o pipefail; mvn -B -ntp -Dmaven.test.failure.ignore verify | tee log.txt"' recordIssues(tools: [javaDoc(pattern: 'log.txt')])
This is more robust and modular anyway. A complex parallelized build might have a bunch of steps doing things only some of which make sense to scan for warnings in a given context.
I would still suggest a block-scoped step, though:
recordingIssues(tools: [javaDoc()]) {
sh 'mvn -B -ntp -Dmaven.test.failure.ignore verify'
}
Yes, of course. I started an implementation in the last year Hacktoberfest but never managed to finish it. Too many other open issues.
drulli the work around may not be needed anymore after workflow-job 2.41? see https://github.com/jenkinsci/workflow-job-plugin/pull/184
I'm reopening so that I do not forget to remove the workaround in the warnings plugin.
I do have a build that reports this as a notice before instantly aborting
(the last line might lack a bunch of line breaks already from the start... stars were consumed by inserting it here):
[GNU C Compiler (gcc)] Sleeping for 5 seconds due to
JENKINS-32191...
[GNU C Compiler (gcc)] Parsing console log (workspace: '/home/jenkins/agent/workspace/workspace/<project_name>/build')
Unable to change directory using: .Entering directory (?<dir>.) to match Entering directory: D0750 0 deploy
I now have to guess if its related in some way to the ticket right here
(like some subdirs not yet present due to insufficient other process completion)
or if it is something totally different (that i was unable to clearly track down for now).
Setup:
- Jenkins 2.346.3
- workflow-job / Pipeline Job plugin 1207.ve6191ff089f8
side note: if you had released your change already, then it would be less of a question, at least underneath this ticket.
Alexander
added a comment - - edited I do have a build that reports this as a notice before instantly aborting
(the last line might lack a bunch of line breaks already from the start... stars were consumed by inserting it here):
[GNU C Compiler (gcc)] Sleeping for 5 seconds due to JENKINS-32191 ...
[GNU C Compiler (gcc)] Parsing console log (workspace: '/home/jenkins/agent/workspace/workspace/<project_name>/build')
Unable to change directory using: . Entering directory (?<dir>. ) to match Entering directory: D0750 0 deploy
I now have to guess if its related in some way to the ticket right here
(like some subdirs not yet present due to insufficient other process completion)
or if it is something totally different (that i was unable to clearly track down for now).
Setup:
Jenkins 2.346.3
workflow-job / Pipeline Job plugin 1207.ve6191ff089f8
side note: if you had released your change already, then it would be less of a question, at least underneath this ticket. Alexander
added a comment - Found the reported line and breakage to be unrelated to this ticket.
When doing some "scp" command with a "-v" flag for verbosity, then you might see messages like:
Entering directory: D0750 0 deploy
This will later on result in an error when doing some parsing with maybe e.g. warnings-ng (or similar).
The final line where the codes then will exit is of that format:
throw new ParsingException("Unable to change directory using: %s to match %s", makePath.toString(), line);
For now i will drop the "-v" switch - but i probably also will do some report to the respective tooling that fails to hard for such a line.
References:
https://github.com/jenkinsci/analysis-model/blob/master/src/main/java/edu/hm/hafner/analysis/LookaheadParser.java
https://unix.stackexchange.com/questions/655319/how-to-scp-from-remote-root-directory
Alexander
added a comment - Found the reported line and breakage to be unrelated to this ticket.
When doing some "scp" command with a "-v" flag for verbosity, then you might see messages like:
Entering directory: D0750 0 deploy
This will later on result in an error when doing some parsing with maybe e.g. warnings-ng (or similar).
The final line where the codes then will exit is of that format:
throw new ParsingException("Unable to change directory using: %s to match %s", makePath.toString(), line);
For now i will drop the "-v" switch - but i probably also will do some report to the respective tooling that fails to hard for such a line.
References:
https://github.com/jenkinsci/analysis-model/blob/master/src/main/java/edu/hm/hafner/analysis/LookaheadParser.java
https://unix.stackexchange.com/questions/655319/how-to-scp-from-remote-root-directory Please create an issue in the warnings plugin (or directly provide a PR that will improve the regular expression that scans for directory changes).
jglick: It would be helpful if you can write at least a comment why you see this issue in the warnings plug-in and not in core/workflow.