-
Bug
-
Resolution: Fixed
-
Major
-
None
-
Jenkins 2.138.1 and NG Warnings plugin version 2.1.2
(related to JENKINS-56001)
I'm running multiple parallel builds. Some of these builds are run within docker containers. The code would look something like this:
docker.image("$docker_server/$docker_image-dev-$arch:$build_tag").inside("--privileged -v $docker_image-$arch-ccache:/root/.ccache") { sh '''#!/bin/bash -e pushd /workspace <build here> ''' recordIssues(tools: [gcc4(id: "gcc4-$os$arch", name: "GNU C Compiler $target")]) }
It does appear to capture all/only [?] the issues for that build within the docker container, but on the build page we are unable to see any of the issues in source context. The build happens at /workspace within the docker container instead of $WORKSPACE, so the parallel build containers may share the one $WORKSPACE. I don't think this is our issue though.
It appears the console output being parsed has the actual absolute path to the source files and this location is not visible to recordIssues. Since I can't tell recordIssues to ignore /workspace and instead look for the source in $WORKSPACE, it seems I'm stuck.
Example output from info screen:
Error Messages Can't resolve absolute paths for some files: - /workspace/live555/groupsock/../../live555/transport/include/TransportMediaSession.h - /workspace/TinyXml/pixelproDevice.cpp - /workspace/live555/BasicUsageEnvironment/../shared/ipplugin.h - /workspace/TinyXml/../../live555/transport/include/StreamsockTransport.h - /opt/gtest/1.8.0/x64/include/gtest/gtest.h - /workspace/TinyXml/Services/soapC_003.cpp - /workspace/TinyXml/./source/TransportMultiFramedRTPSource.cpp - /workspace/TinyXml/tests.cpp - /workspace/TinyXml/AudioOut.h ... skipped logging of 453 additional errors ... Can't create fingerprints for some files: - '/workspace/TinyXml/../AES/aeskey.c', IO exception has been thrown: java.nio.file.NoSuchFileException: /workspace/TinyXml/../AES/aeskey.c - '/workspace/TinyXml/../AES/aeskey.c', IO exception has been thrown: java.nio.file.NoSuchFileException: /workspace/TinyXml/../AES/aeskey.c .......
Information Messages Parsing console log (workspace: '/var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ') Post processing issues on 'DockerAgent0' with encoding 'UTF-8' Resolving absolute file names for all issues -> 0 resolved, 473 unresolved, 0 already resolved Copying affected files to Jenkins' build folder /var/jenkins_home/jobs/Server Branch/branches/build_updates/builds/169 -> 0 copied, 0 not in workspace, 473 not-found, 0 with I/O error Resolving module names from module definitions (build.xml, pom.xml, or Manifest.mf files) -> resolved module names for 3061 issues Resolving package names (or namespaces) by parsing the affected files -> resolved package names of 473 affected files No filter has been set, publishing all 3061 issues Creating fingerprints for all affected code blocks to track issues over different builds -> created fingerprints for 0 issues Skipping blaming as requested in the job configuration Using reference build 'Server Branch/build_updates #166' to compute new, fixed, and outstanding issues Issues delta (vs. reference build): outstanding: 2683, new: 378, fixed: 189 No quality gates have been set - skipping Health report is disabled - skipping
We do not have this issue with the Legacy Warnings plugin. Of course I have moved the previous warnings() call from a post-stage success{} block call to a recordIssues() call inside the docker.image.inside{} block, but I also tried the recordIssues() call in the same post-satge success{} block and had the same (no source) result. So, it appears this just no longer works the way it did with the Legacy Warnings plugin.
- depends on
-
JENKINS-48802 Possibility to add/change the path that used to display source files
-
- Resolved
-
[JENKINS-55998] recordIssues cannot see /workspace within docker.image.inside
Well, I wonder if we can tell
- either Jenkins to return /workspace as $WORKSPACE and not
/var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ - or make to print /var/lib/docker/builder/repo/workspace/_Branch_build_updates-SDSYGOEWO53Z6ASKV6W4GSRWQU4DXCNNDGKGTWMQJ4O7LTMGYQVQ rather than /workspace
I don't think I can get option 2 to work. If I had a way to capture the log and run sed over it to do the replacement, I guess that could get me what I need, but that seems hacky.
If I wrap the recordIssues call in the docker.image.inside block, should recordIssues be able to see the filesystem within the docker container and see /workspace?
I don't think that the wrapping is the problem. Currently the parser combines a path by concatenating the output of make and gcc. Copying of the files is then done by a call that is initiated by the master. This call does know nothing about a /workspace.
Ah, understood. Please let me know if there is anything I can do. I may just have to capture stdout/err from the build into a log file, use sed to change /workspace to $WORKSPACE, and then use recordIssues on that log file. I really want to deploy the new Warnings plugin, but I can't lose this functionality. Thanks!
I think providing a source code prefix as suggested in JENKINS-48802 (and disabling the automatic path resolution) should help here as well.
The requirements of JENKINS-48802 are currently checked in a security review. Will need some time... It is not clear how this impacts security, since currently only workspace files are allowed to be shown.
OK, I understand. So, we may lose the ability to see our warnings in context if we move to the Next Gen Warnings plugin because we use the overlay for our builds within docker containers? Using the overlay allows us to checkout one time and share that checkout with 5 or sometimes more target builds. I can't see us moving to checking out 5 times and I'm not even sure how we'd do that since all of the docker containers are used in parallel from the same host.
I'm using the updated plugins from last night and, as you would expect, I have the same issue with my Linux builds. I'm not sure what we're going to do here except maybe stay with the legacy plugin for our gcc warnings processing. If you have any ideas for me, I'm open to them. If we had a way of telling the plugin to replace "/workspace" with "$WORKSPACE", that might work.