-
Improvement
-
Resolution: Fixed
-
Major
-
None
-
Master: RHEL 6.2, Jenkins LTS, Warnings 4.5; Slave: Fedora Core 16
-
-
beta8 (warnings-ng), beta17 (analysis-model-api)
Hi,
I set up a VPATH build of an autotools-based software in a custom workspace (set per node) in "build" directory:
/mnt/ram/workspace/[job-id]/build
Therefore generally files are referenced in the log as ../../folder/file.cpp, because the build directory only contains Makefiles and other temporarily generated data. E.g. when files from nestkernel are compiled through the Makefiles in
/mnt/ram/workspace/[job-id]/build/nestkernel
the source files are referenced as ../../nestkernel/file.cpp so that it resolves as
/mnt/ram/workspace/[job-id]/nestkernel/file.cpp
I'm afraid this can't be possibly changed. Now because of that LLVM parser extracts warnings as belonging e.g. to "../../nestkernel", so apparently the file references are later resolved as
/mnt/nestkernel/file.cpp
relative to
/mnt/ram/workspace/[job-id]/
and not to
/mnt/ram/workspace/[job-id]/build/nestkernel
which of course doesn't exist so I'm getting the following kind of errors when I try to see the file:
Content of file connection_manager.cpp 01 Copying the source file '../../nestkernel/connection_manager.cpp' from the workspace to the build folder '/var/lib/jenkins/jobs/nest-clang-test/builds/2012-06-08_17-46-33/workspace-files/7976711b.tmp' on the Hudson master failed. 02 Seems that the path is relative, however an absolute path is required when copying the sources.%nIs the file 'connection_manager.cpp' contained more than once in your workspace? 03 Is the file '../../nestkernel/connection_manager.cpp' a valid filename? 04 If you are building on a slave: please check if the file is accessible under '$HUDSON_HOME/[job-name]/../../nestkernel/connection_manager.cpp' 05 If you are building on the master: please check if the file is accessible under '$HUDSON_HOME/[job-name]/workspace/../../nestkernel/connection_manager.cpp' 06 hudson.util.IOException2: remote file operation failed: ../../nestkernel/connection_manager.cpp at hudson.remoting.Channel@68b7cdc6:fc-16-i386-2 07 at hudson.FilePath.act(FilePath.java:780) 08 at hudson.FilePath.act(FilePath.java:766) 09 at hudson.FilePath.copyTo(FilePath.java:1460) 10 at hudson.plugins.analysis.core.HealthAwarePublisher.copyFilesWithAnnotationsToBuildFolder(HealthAwarePublisher.java:398) 11 at hudson.plugins.analysis.core.HealthAwarePublisher.perform(HealthAwarePublisher.java:355) 12 at hudson.tasks.BuildStepMonitor$2.perform(BuildStepMonitor.java:27) 13 at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:697) 14 at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:672) 15 at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:650) 16 at hudson.model.Build$RunnerImpl.post2(Build.java:162) 17 at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:619) 18 at hudson.model.Run.run(Run.java:1429) 19 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46) 20 at hudson.model.ResourceController.execute(ResourceController.java:88) 21 at hudson.model.Executor.run(Executor.java:238) 22 Caused by: java.io.FileNotFoundException: ../../nestkernel/connection_manager.cpp (No such file or directory) 23 at java.io.FileInputStream.open(Native Method) 24 at java.io.FileInputStream.<init>(FileInputStream.java:137) 25 at hudson.FilePath$31.invoke(FilePath.java:1465) 26 at hudson.FilePath$31.invoke(FilePath.java:1460) 27 at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2045) 28 at hudson.remoting.UserRequest.perform(UserRequest.java:118) 29 at hudson.remoting.UserRequest.perform(UserRequest.java:48) 30 at hudson.remoting.Request$2.run(Request.java:287) 31 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 32 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 33 at java.util.concurrent.FutureTask.run(FutureTask.java:166) 34 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 35 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 36 at java.lang.Thread.run(Thread.java:679)
A typical log snippet looks like this:
mv -f .deps/libdevelopermodule_la-maturing_connector_fr.Tpo .deps/libdevelopermodule_la-maturing_connector_fr.Plo /bin/sh ../libtool --tag=CXX --mode=compile clang++ -DHAVE_CONFIG_H -I. -I../../developer -I../libnestutil -I../../libnestutil -I../../librandom -I../../sli -I../../nestkernel -I../../models -I../../libltdl -I/usr/include -W -Wall -pedantic -Wno-long-long -O2 -MT libdevelopermodule_la-stdp_triplet_connection.lo -MD -MP -MF .deps/libdevelopermodule_la-stdp_triplet_connection.Tpo -c -o libdevelopermodule_la-stdp_triplet_connection.lo `test -f 'stdp_triplet_connection.cpp' || echo '../../developer/'`stdp_triplet_connection.cpp libtool: compile: clang++ -DHAVE_CONFIG_H -I. -I../../developer -I../libnestutil -I../../libnestutil -I../../librandom -I../../sli -I../../nestkernel -I../../models -I../../libltdl -I/usr/include -W -Wall -pedantic -Wno-long-long -O2 -MT libdevelopermodule_la-stdp_triplet_connection.lo -MD -MP -MF .deps/libdevelopermodule_la-stdp_triplet_connection.Tpo -c ../../developer/stdp_triplet_connection.cpp -fPIC -DPIC -o .libs/libdevelopermodule_la-stdp_triplet_connection.o In file included from ../../developer/stdp_triplet_connection.cpp:19: In file included from ../../nestkernel/network.h:24: In file included from ../../nestkernel/model.h:23: In file included from ../../nestkernel/node.h:28: In file included from ../../sli/dictdatum.h:22: In file included from ../../sli/interpret.h:28: In file included from ../../sli/tokenstack.h:25: ../../sli/tokenarray.h:422:19: warning: operands of ? are integers of different signs: 'unsigned long' and 'long' [-Wsign-compare] rot = (rot < 0) ? rot+size() : rot; ^ ~~~~~~~~~~ ~~~
Apparently this problem is not specific to Clang, but probably is also valid for GCC (but I don't seem to remember anything like that, can it be that workspace scan is not working for LLVM?!), so as long as VPATH (out of source tree) builds are involved this problem manifests itself.
Is there any way in which this can be disambiguated?
Thanks!
- is blocking
-
JENKINS-32150 Cannot resolve relative filenames referencing parent directory
-
- Resolved
-
- is duplicated by
-
-
- Resolved
-
- is related to
-
-
- Resolved
-
-
-
- Resolved
-
[JENKINS-14064] Use make output to map relative paths to absolute paths
Yury Zaytsev
added a comment - I have to add, that in fact it works in a very strange way. Some files it recognizes fine, e.g.
sli/tokenarray.h:422
On the other hand, the very same warning appears again as
../../sli/tokenarray.h
and is not correctly dereferenced. Same for ../../nestkernel, ../../models, ../../developer, but not for pynest. Is there any way I can help to diagnose the issue?
Yury Zaytsev
added a comment - I have to add, that in fact it works in a very strange way. Some files it recognizes fine, e.g.
sli/tokenarray.h:422
On the other hand, the very same warning appears again as
../../sli/tokenarray.h
and is not correctly dereferenced. Same for ../../nestkernel, ../../models, ../../developer, but not for pynest. Is there any way I can help to diagnose the issue? A simple workaround would be to pipe the output through a script that replaces the "../../sli" references with "sli".
Do you have an idea on how to resolve that directly in the plug-in?
Yury Zaytsev
added a comment - Hi,
A simple workaround would be to pipe the output through a script that replaces the "../../sli" references with "sli".
Yes, this could work, for example, in the plugin you can make a field to enter regexps to apply to the file names, i.e. s/\.\.\/(.*)$/\1/, but this solution kind of sucks.
Do you have an idea on how to resolve that directly in the plug-in?
I think your idea of trying to find whether files are in the workspace by looking for them was a very good and generic one. Only maybe make the logic a bit smarter (I didn't have a look at the source code yet, to figure how you are doing it at the moment)... but what I can't understand is why it works in some cases, and in some cases not??? The file names are unique (see example above).
Also, now I can confirm that it's the same problem for gcc: https://qa.nest-initiative.org/job/nest-matrix-10kproject/3/warnings12Result/? .
Yury Zaytsev
added a comment - Hi,
A simple workaround would be to pipe the output through a script that replaces the "../../sli" references with "sli".
Yes, this could work, for example, in the plugin you can make a field to enter regexps to apply to the file names, i.e. s/\.\.\/(.*)$/\1/, but this solution kind of sucks.
Do you have an idea on how to resolve that directly in the plug-in?
I think your idea of trying to find whether files are in the workspace by looking for them was a very good and generic one. Only maybe make the logic a bit smarter (I didn't have a look at the source code yet, to figure how you are doing it at the moment)... but what I can't understand is why it works in some cases, and in some cases not??? The file names are unique (see example above).
Also, now I can confirm that it's the same problem for gcc: https://qa.nest-initiative.org/job/nest-matrix-10kproject/3/warnings12Result/? . I see. Someone else is already working on a new file detector that is smarter and faster. I'm not sure about the progress though...
Yury Zaytsev
added a comment - Hi Ulli,
Now that I've got gcc + make working for me thanks to you merging my pull request, I wonder if same can be done to LLVM? Could you please suggest a way to go? I mean, I could diff the gcc4 plugin and the gcc+make plugin to try to produce llvm + make plugin out of this changes and submit it to you, but this does feel like duplicating lots of code that will result into maintenance problems, doesn't it?
Do you think it is possible to disentangle the make support in some way, so that one can use it together with other parsers stacking them on top of each other without producing custom variants of each parser + make support?
Sorry for naïve questions, but I really have no idea how your plugin works and I'm no Java programmer. So I'm willing to help, but all I could get done so far are very simple hacks...
Thanks!
Yury Zaytsev
added a comment - Hi Ulli,
Now that I've got gcc + make working for me thanks to you merging my pull request, I wonder if same can be done to LLVM? Could you please suggest a way to go? I mean, I could diff the gcc4 plugin and the gcc+make plugin to try to produce llvm + make plugin out of this changes and submit it to you, but this does feel like duplicating lots of code that will result into maintenance problems, doesn't it?
Do you think it is possible to disentangle the make support in some way, so that one can use it together with other parsers stacking them on top of each other without producing custom variants of each parser + make support?
Sorry for naïve questions, but I really have no idea how your plugin works and I'm no Java programmer. So I'm willing to help, but all I could get done so far are very simple hacks...
Thanks! I'll investigate how the make part can be extracted so that it is available as an option for all parsers. That actually is a good idea!
satya b
added a comment - Hi Ulli,
I have upgraded to warnings plugin of version 4.35 recently. i have identified issue with relative paths in this plugin. i am unable to see file contents from warning plugin.
i too got same error message like yury message. any assistance here to resolve the same.
regards
Satya
satya b
added a comment - Hi Ulli,
I have upgraded to warnings plugin of version 4.35 recently. i have identified issue with relative paths in this plugin. i am unable to see file contents from warning plugin.
i too got same error message like yury message. any assistance here to resolve the same.
regards
Satya Satya, can zou please add more details? What are your parsing, are the paths relative or absolute?
satya b
added a comment - Hafner,I used custom workspace used in my jenkins job. softwares used in my application are mainly eclipse,ant,gnu and version control is mks. application working fine till upgration of warnings plugin. i am able to access my source code files from trend graphs in jenkins. Once i have upgraded, few files,showing with absolute paths,able to access and remaining files,showing with relative path,not able to access from plugin.
let me know if you require more details.
satya b
added a comment - Hafner,I used custom workspace used in my jenkins job. softwares used in my application are mainly eclipse,ant,gnu and version control is mks. application working fine till upgration of warnings plugin. i am able to access my source code files from trend graphs in jenkins. Once i have upgraded, few files,showing with absolute paths,able to access and remaining files,showing with relative path,not able to access from plugin.
let me know if you require more details. 
Exemplar console build log