Rob, I'm willing to try to fix this issue because I need JENKINS-7619 fixed. I've been looking through the code and wondering if I were to add the workspace root in the depot (e.g. //Depot/foo/bar) to the Changelist bean, I could then adjust the file path in the changelog.xml by removing this workspace root from its path. Effectively replacing the current file path of "//Depot/foo/bar/project/module/src/blah/File.java" with "project/module/src/blah/File.java".

I could also output the workspace root in the depot to the changelog.xml just in case someone needed to reconstruct the file path within the depot.

What do you think?

I think that will only work in the simplest case. Consider the following view:

//depot/product/a/... //workspace/a/...
//depot2/vendor/... //workspace/a/vendor/...
//depot/product/%1/depends/... //workspace/a/depends/%1/...
+//depot/product/branch/feature/... //workspace/a/...

It becomes significantly more complicated, since it's not a 1-1 mapping between the depot and the workspace. Perforce has a built in function to deal with this sort of thing called 'p4 where' that shows how a particular file gets mapped through the view:

rpetti@petti:~/workspace/Install/trunk/Installers$ p4 where build.properties    //Install/trunk/Installers/build.properties //rpetti/Install/trunk/Installers/build.properties /home/rpetti/workspace/Install/trunk/Installers/build.properties

Here it shows the depot path, the workspace path, and the path of the file on disk. The only problems here are that the values are space delimited, making it difficult to parse reliably, and the command needs to be run for every file in the changelist.

There are two ways to get around the parsing issue. The first is that you know what two of the delimiters are from the get go: In this case '//rpetti' which is just the name of the workspace and '/home/rpetti/workspace' which is the workspace root. You could hypothetically use those values to reliably parse the output.

The other is to ask perforce to output the results as a marshalled python dictionary (using 'p4 -G where'). The problem is that I haven't been able to find a java library for parsing these, so you might have to write one.

So ideally, when the plugin finishes grabbing a changelist description, it will proceed with finding the mapping of every file in the changeset by running 'p4 where'. It will then dump this information into a new field in the FileEntry class. The getAffectedPaths() function would just need to be changed to reference that new field instead of the depot path.

If this is too much, then I can take another look at it maybe this week or next week? I admit that I've been putting this one off since it didn't seem all that critical at the time.

It's true I hadn't considered more complex mappings since all our projects typically don't require anything more than a single mapping.

Have you used the Perforce Java API (http://www.perforce.com/perforce/doc.091/manuals/p4java-javadoc/index.html), specifically P4JFileSpec? I wonder if it can be used to get the correct file path within the workspace?

I'm sure it could, but preliminary tests using the p4java api were not good, and it would take a considerable amount of work to switch over to it.

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/com/tek42/perforce/model/Changelist.java
src/main/java/com/tek42/perforce/parse/Changes.java
src/main/java/hudson/plugins/perforce/PerforceChangeLogEntry.java
src/main/java/hudson/plugins/perforce/PerforceSCMHelper.java
src/test/java/hudson/plugins/perforce/PerforceSCMHelperTest.java
http://jenkins-ci.org/commit/perforce-plugin/4f82c3a48009061c99bf905a39af59178b924094
Log:
JENKINS-7618 potential fix for workspace path problem.

Using p4 -G where <depotFilePath> we can determine what the workspace path should be.

Tim, did you want to try testing the latest snapshot? http://files.robpetti.com/perforce-plugin/target/perforce.hpi The build log might be a little noisy right now, but it should be resolving the workspace paths for the affected files correctly now.

It failed for me:

ERROR: Processing failed due to a bug in the code. Please report this to jenkinsci-users@googlegroups.com
java.lang.NullPointerException
at hudson.maven.MavenModuleSetBuild$1.isDescendantOf(MavenModuleSetBuild.java:244)
at hudson.maven.MavenModuleSetBuild$1.<init>(MavenModuleSetBuild.java:213)
at hudson.maven.MavenModuleSetBuild.getChangeSetFor(MavenModuleSetBuild.java:207)
at hudson.maven.MavenModuleSetBuild$RunnerImpl.doRun(MavenModuleSetBuild.java:633)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:430)
at hudson.model.Run.run(Run.java:1376)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:466)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:175)

What version of Jenkins are you running? I suspect I might know what the problem is, but I want to make sure before I send you another build.

I'm running 1.419-SNAPSHOT (I just built it yesterday from git to debug this issue).

I was stepping through your code and in the 'change' object the workspacePath looks good (modules/foo/impl/....) but the filename (//Common/maventest/main/maventest/modules/foo/impl/....) is still being written to changelog.xml. Is that the correct behavior?

Yes, the depot path is still being used by SCM Browser integration, so we need to store both. getAffectedPaths will return the workspacePaths, however. I suspect that the maven plugin may be reading old changesets in order to determine what needs to be rebuilt, in which case the perforce plugin will return null for the affected paths. I've changed the code to provide the workspace path only when it's available. If you've got the time, you can test using the latest snapshot (same link as before).

Hopefully that will fix it.

That change eliminated the NPE, but it still triggered a full build of project despite correctly identifying the one changed file. "com.saptest:maventest" is the top POM of the project. I thought it would build just the changed foo.impl module, then the EAR that depends on foo.impl.

Using master perforce client: i821885-temp
[workspace] $ "c:\Program Files\Perforce\p4.exe" workspace -o i821885-temp
Last build changeset: 134076
[workspace] $ "c:\Program Files\Perforce\p4.exe" counter change
[workspace] $ "c:\Program Files\Perforce\p4.exe" -s changes //i821885-temp/...@134077,@134077
[workspace] $ "c:\Program Files\Perforce\p4.exe" describe -s 134077
[workspace] $ "c:\Program Files\Perforce\p4.exe" -G where //Common/maventest/main/maventest/modules/foo/impl/src/main/java/com/saptest/foo/impl/FooService.java
Sync'ing workspace to changelist 134077.
[workspace] $ "c:\Program Files\Perforce\p4.exe" -s sync //i821885-temp/...@134077
Sync complete, took 686 ms
Parsing POMs
[workspace] $ D:\p4views\BuildSystem\java\jdk1.6.0_24/bin/java -Xmx1024m -XX:MaxPermSize=256m -cp d:\java\.m2\org\jenkins-ci\main\maven\maven-agent\1.2\maven-agent-1.2.jar;D:\p4views\BuildSystem\tools\apache-maven-2.2.1\boot\classworlds-1.1.jar hudson.maven.agent.Main D:\p4views\BuildSystem\tools\apache-maven-2.2.1 D:\java\.m2\org\jenkins-ci\main\remoting\1.419-SNAPSHOT\remoting-1.419-SNAPSHOT.jar d:\java\.m2\org\jenkins-ci\main\maven\maven-interceptor\1.2\maven-interceptor-1.2.jar 63212 d:\java\.m2\org\jvnet\hudson\maven2.1-interceptor\1.2\maven2.1-interceptor-1.2.jar
<===[JENKINS REMOTING CAPACITY]===>channel started
+ Error stacktraces are turned on.
Executing Maven: -B -f D:\java\jenkins-src\war\work\jobs\maventest-main\workspace\pom.xml -amd -pl com.saptest:maventest -X -P maventest,netweaver-71 clean install

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/hudson/plugins/perforce/PerforceChangeLogSet.java
http://jenkins-ci.org/commit/perforce-plugin/0216bfc4eceb48478890dc85c16255ace485196a
Log:
JENKINS-7618

get workspace path for changed files saved and loaded properly

I hate to do this to you, but could you have another go at it? I forgot that the changelist objects have manual serialization and parsing routines...

Yes! That worked:

Executing Maven: -B -f D:\java\jenkins-src\war\work\jobs\maventest-main\workspace\pom.xml -amd -pl com.saptest:maventest.foo.impl -X -P maventest,netweaver-71 clean install

and the build page shows only the foo.impl, EAR (depends on foo.impl), and SCA (depends on EAR) being built. I will try it tomorrow on our production Jenkins system which has much, much larger projects on it (which is why this feature is so critical).

Many thanks for your work, Rob.

Thanks for testing it.

Let me know if you run into any more issues. I'll close this ticket and it's dependent once I clean up the build log output a bit. It'll currently show a line for every file in every changeset that's being pulled down, which can get long depending on project activity.

I've just installed it on our main build server (jenkins 1.416) and it's working fine.

After working all morning, I just got this NPE:

java.lang.NullPointerException
at hudson.plugins.perforce.PerforceSCMHelper.parseWhereMapping(PerforceSCMHelper.java:133)
at com.tek42.perforce.parse.Changes.calculateWorkspacePaths(Changes.java:78)
at com.tek42.perforce.parse.Changes.getChangelist(Changes.java:67)
at com.tek42.perforce.parse.Changes.getChangelistsFromNumbers(Changes.java:403)
at hudson.plugins.perforce.PerforceSCM.checkout(PerforceSCM.java:628)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1182)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:537)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:425)
at hudson.model.Run.run(Run.java:1376)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:465)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:146)

[deleted] after I figured out the output of p4 -G is binary and may have control characters...

Rob, let me know if you want the output of p4 -G on my system and I can send it to you.

Is it failing consistently on the same call?

Yeah, feel free to pipe the output into a file and send it to me, I'll take a look ASAP.

It appears that readInt() is not handling the bytes as unsigned bytes. At one point, the 4 size bytes are -106,0,0,0 and this is being interpreted as a size of -106.

in PerforceSCMHelper.readInt() I changed it to:
result += (int) (bytes[i]&0xff) << (8*(i-offset));

and this seems to work.

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/hudson/plugins/perforce/PerforceSCMHelper.java
src/test/java/hudson/plugins/perforce/PerforceSCMHelperTest.java
http://jenkins-ci.org/commit/perforce-plugin/3e159833ed2fde3682f68a48eb8620a7ecaaf0a2
Log:
JENKINS-7618 fixing bad integer parsing

Sweet, thanks! I've committed the fix and the snapshot has been updated.

Rob, I still get an NPE on our production jenkins server so it's not fixed completely. I'll work on it today and hopefully get you another patch.

The conversion of the P4 response StringBuffer to bytes via .toBytes() is producing bad results. For one file, the path length is 143 which shows up in the StringBuffer as an unknown character (at least to Eclipse - it prints a solid diamond with a white '?' inside). When converted to a byte[], the value is 63d which is clearly wrong and it throws the parser off. I tried .toBytes(Charset.forName("UTF8")) as a wild guess and really fouled things up. I haven't dealt with String-to-byte parsing in Java; this was always much easier in C. I'll get it eventually, but was hoping you might have some insight to speed up the process...

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/com/tek42/perforce/parse/AbstractPerforceTemplate.java
src/main/java/com/tek42/perforce/parse/Changes.java
http://jenkins-ci.org/commit/perforce-plugin/6f25b91e0461cde44a7aaec7f4824b16c4be041b
Log:
JENKINS-7618 implemented function for getting the perforce response directly from the p4 process as a raw byte array

I think that change should fix it. It now pulls the data from the pipe as a byte array, so it never converts it into a string in the first place. The snapshot has been updated if you want to test again. I really appreciate all of your patience and help with this!

Still not fixed. Same issue. The output of "where -G" produces a set of length bytes of „,0,0,0 (not sure if that first one will paste) which, according to this table (http://doc.infosnel.nl/extreme_utf-8.html), is 132,0,0,0. But when I look at "bytes" in getRawPerforceResponseBytes, there's 30d instead of 132d. Is this an encoding issue? Our Perforce charset is set to utf8.

I'm thinking that CmdLineExecutor.exec() needs to instantiate the reader with a charset encoding. Ideally it would be from the P4CHARSET variable returned by "p4 set".

UPDATE: I forced the creation of the InputStreamReader to use UTF8 and it made no difference in the output. I can't understand how this issue can be so difficult.

CmdLineExecutor isn't used, and it doesn't make sense to have any character encoding at all. It's pure binary data, which is why I'm attempting to read it as raw bytes. It looks like the readers used by the jenkins remoting api are what's at fault here, since they do the encoding.

Well that makes a difference then. After setting a few breakpoints, I found that it's using HudsonP4DefaultExecutor and by hardcoding the reader to instantiate the input stream via UTF8 encoding:

reader = new BufferedReader(new InputStreamReader(p4in, "UTF8"));

I got the byte array to read the size in as -3 which still screws up the parser and, well, I have no idea how that's supposed to be 132d....

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/com/tek42/perforce/parse/AbstractPerforceTemplate.java
src/main/java/com/tek42/perforce/process/CmdLineExecutor.java
src/main/java/com/tek42/perforce/process/Executor.java
src/main/java/hudson/plugins/perforce/HudsonP4DefaultExecutor.java
src/main/java/hudson/plugins/perforce/HudsonP4Executor.java
src/main/java/hudson/plugins/perforce/HudsonP4RemoteExecutor.java
http://jenkins-ci.org/commit/perforce-plugin/bc63a279f47f357d9bf12f9ebca26bbc8bcb203f
Log:
JENKINS-7618 refactor executors to expose raw data streams in order to correctly read 'p4 -G' formatted data

Could you try the latest again? If it still doesn't work, I'll need the output that's causing it to fail so I can debug it faster.

It wasn't sure if my changes were mixing in with your stuff and I don't really know how to use git yet, so I blew it all away and recloned. This is what I'm getting now - did I get the right branch of code?

[workspace] $ "c:\Program Files\Perforce\p4.exe" -G where //VM/Base/main/modules/legacy/api/src/main/java/com/sap/me/production/CheckConfigInterface.java
FATAL: Not supported yet.
java.lang.UnsupportedOperationException: Not supported yet.
at hudson.plugins.perforce.HudsonP4DefaultExecutor.getInputStream(HudsonP4DefaultExecutor.java:138)
at com.tek42.perforce.parse.AbstractPerforceTemplate.getRawPerforceResponseBytes(AbstractPerforceTemplate.java:496)
at com.tek42.perforce.parse.Changes.calculateWorkspacePaths(Changes.java:77)
at com.tek42.perforce.parse.Changes.getChangelist(Changes.java:67)
at com.tek42.perforce.parse.Changes.getChangelistsFromNumbers(Changes.java:403)
at hudson.plugins.perforce.PerforceSCM.checkout(PerforceSCM.java:628)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1184)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:537)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:425)
at hudson.model.Run.run(Run.java:1376)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:466)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:175)

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/hudson/plugins/perforce/HudsonP4DefaultExecutor.java
http://jenkins-ci.org/commit/perforce-plugin/a4c0750b6fb65c31287e2e59f2925fda6984189a
Log:
JENKINS-7618 fixing silly mistake in last commit.

That was my bad... I only tested it on a slave machine, and missed a problem in the master executor.

This worked on my laptop. I've installed it on our build server and I'll let it run for the day. I'll let you know if there are any problems.

Rob, our build system has run the build about 5-6 times today (fairly low number of check-ins), but the latest patch has worked flawlessly.

Hi Rob, just tried the latest snapshot. I've discovered a parsing issue:

Last build changeset: 58015
[workspace] $ /usr/local/bin/p4 counter change
[workspace] $ /usr/local/bin/p4 -s changes //jenkins-indexing/...@58016,@58016
[workspace] $ /usr/local/bin/p4 describe -s 58016
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpData/brill/LEXICON.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpData/brill/nounCat.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/Adverbial.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/CD.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/Clause.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/NPEquiv.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/PP.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/VED.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/VP.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/sure_VP.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/ie/CauseEffect.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetChunker_Sentence2.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetLinker_Sentence1.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetLinker_Sentence2.xml
Sync'ing workspace to changelist 58016.
[workspace] $ /usr/local/bin/p4 -s sync //jenkins-indexing/...@58016
Sync complete, took 743 ms
Found mavenVersion 3.0.3 from file jar:file:/acc1/jenkins/p4/acc/oem/apache-maven-3.0.3/lib/maven-core-3.0.3.jar!/META-INF/maven/org.apache.maven/maven-core/pom.properties
No emails were triggered.
Parsing POMs
[src] $ /usr/java/jdk/bin/java -Xmx1024m -XX:MaxPermSize=256m -client -cp /acc1/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.1.jar:/acc1/jenkins/p4/acc/oem/apache-maven-3.0.3/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /acc1/jenkins/p4/acc/oem/apache-maven-3.0.3 /opt/tomcat/webapps/jenkins/WEB-INF/lib/remoting-1.409.1.jar /acc1/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.1.jar 39198
<===[HUDSON REMOTING CAPACITY]===>���channel started
Executing Maven:  -B -f /acc1/jenkins/jobs/indexing-nonstop/workspace/branch/indexing/src/pom.xml -Dmaven.repo.local=/acc1/jenkins/jobs/indexing-nonstop/workspace/.repository -amd -pl com.netbase:nlp,com.netbase:nlp-data,com.netbase:test,com.netbase:test-data -s /acc1/jenkins/jobs/indexing-nonstop/workspace/acc/oem/apache-maven-3.0.3/conf/hudson_settings.xml -e -T1C clean source:jar install

Notice in the above snippet that there were only code changes to test-data and nlp-data. However, in the -pl switch, my nlp and test projects were also included.

Sorry, I'm not sure what I'm looking at here. I don't see any parsing errors... Can you check the changelog.xml file for that build and make sure workspacePath is correct? If it is, then I would suspect something is wrong with either the maven plugin, or your maven project itself.

Hmm, ok, it just looks very suspicious that the code changes affected my nlp-data and test-data projects, yet my nlp and test projects were also included:
modules
nlp-data (changed)
nlp (NOT changed)
test-data (changed)
test (NOT changed)

-amd -pl com.netbase:nlp,com.netbase:nlp-data,com.netbase:test,com.netbase:test-data

Are those the only 4 projects? Maybe it's just building everything like it was before. If not, could test and nlp depend on test-data and nlp-data respectively?

Nope, there are 35 projects total, so it is very close to doing the right thing. I thought how this code worked was to parse the submitted changelists for the build and determine the affected modules. Then add -pl switch with the affected modules and -amd will take care of their dependents.

I did inspect the changelog.xml and workspacePath looks correct.

I ran dependency:tree on these two modules:

[INFO] com.netbase:nlp-data:jar:5.3-SNAPSHOT
[INFO] +- junit:junit:jar:4.8.2:test
[INFO] \- findbugs:findbugs:jar:1.3.9:compile

[INFO] com.netbase:test-data:jar:5.3-SNAPSHOT
[INFO] +- junit:junit:jar:4.8.2:test
[INFO] \- findbugs:findbugs:jar:1.3.9:compile

On the very next build, changes were made to the exact same two projects, yet the -pl switch is completely different from the previous run (-pl com.netbase:consumerbase,com.netbase:elsevier-prospero,com.netbase:engine,com.netbase:megatron,com.netbase:nldemo,com.netbase:nlp,com.netbase:nlp-data,com.netbase:ovid,com.netbase:prospero,com.netbase:test,com.netbase:test-data):

Started by an SCM change
Building on master
Using master perforce client: jenkins-indexing
[workspace] $ /usr/local/bin/p4 workspace -o jenkins-indexing
Saving modified client jenkins-indexing
[workspace] $ /usr/local/bin/p4 -s client -i
Last build changeset: 58016
[workspace] $ /usr/local/bin/p4 counter change
[workspace] $ /usr/local/bin/p4 -s changes //jenkins-indexing/...@58017,@58018
[workspace] $ /usr/local/bin/p4 describe -s 58018
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/nlp-data/src/main/resources/com/accelovation/nlpRules/chunking/NPPPEquiv.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetChunker_Sentence1.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetChunker_Sentence1.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetChunker_Sentence2.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetChunker_Sentence2.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetEvent_Sentence1.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetEvent_Sentence1.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetEvent_Sentence2.txt
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetEvent_Sentence2.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetFeature_Sentence1.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetFeature_Sentence2.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetLinker_Sentence1.xml
[workspace] $ /usr/local/bin/p4 -G where //depot/branch/indexing/src/test-data/src/main/resources/com/accelovation/testData/nlprules/SnippetLinker_Sentence2.xml
Sync'ing workspace to changelist 58018.
[workspace] $ /usr/local/bin/p4 -s sync //jenkins-indexing/...@58018
Sync complete, took 1082 ms
Found mavenVersion 3.0.3 from file jar:file:/acc1/jenkins/p4/acc/oem/apache-maven-3.0.3/lib/maven-core-3.0.3.jar!/META-INF/maven/org.apache.maven/maven-core/pom.properties
No emails were triggered.
Parsing POMs
[src] $ /usr/java/jdk/bin/java -Xmx1024m -XX:MaxPermSize=256m -client -cp /acc1/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-agent-1.1.jar:/acc1/jenkins/p4/acc/oem/apache-maven-3.0.3/boot/plexus-classworlds-2.4.jar org.jvnet.hudson.maven3.agent.Maven3Main /acc1/jenkins/p4/acc/oem/apache-maven-3.0.3 /opt/tomcat/webapps/jenkins/WEB-INF/lib/remoting-1.409.1.jar /acc1/jenkins/plugins/maven-plugin/WEB-INF/lib/maven3-interceptor-1.1.jar 34807
<===[HUDSON REMOTING CAPACITY]===>���channel started
Executing Maven:  -B -f /acc1/jenkins/jobs/indexing-nonstop/workspace/branch/indexing/src/pom.xml -Dmaven.repo.local=/acc1/jenkins/jobs/indexing-nonstop/workspace/.repository -amd -pl com.netbase:consumerbase,com.netbase:elsevier-prospero,com.netbase:engine,com.netbase:megatron,com.netbase:nldemo,com.netbase:nlp,com.netbase:nlp-data,com.netbase:ovid,com.netbase:prospero,com.netbase:test,com.netbase:test-data -s /acc1/jenkins/jobs/indexing-nonstop/workspace/acc/oem/apache-maven-3.0.3/conf/hudson_settings.xml -e -T1C clean source:jar install

Ok, this looks to be user error, sorry! I noticed that this is happening on an unstable build, and previous to the first incremental build, the nlp had failed test cases. Then on the next run, those other projects had failed test cases. So it looks like the incremental build does include previous failing projects. I'm still unclear as to why the test project gets added since it didn't have code changes and it didn't fail...

In Jenkins, projects with status of anything less than "SUCCESS" are added into the list for the -pl argument. We have the same issue because unit tests are failing on several modules so they continue to be built even though nothing has changed within them. I don't agree with this strategy - after all how is a test supposed to succeed if nothing has changed? However, that issue doesn't belong here.

Rob, this patch continues to work without error on our build system. Do I "resolve" this issue or do you do that?

You can do the honors if you'd like.

verified on jenkins 1.416 and 1.419-SNAPSHOT

I do not believe this is fixed. Below is what I am seeing on Jenkins 1.419 with perforce plugin 1.2.8.

The fallout from getting into this state is that the job get's queued every SCM polling period, fails, and sends out failure notification, every minute, for eternity. Downgrading to perforce plugin 1.2.7 made this go away for me.

Here's the log with NPE stacktrace....

Using master perforce client: jenkins_trunk_simpleProject [workspace] $ /usr/bin/p4 workspace -o jenkins_trunk_simpleProject Changing P4 Client View from:
//depot/sandboxes/smacdonald/projects/simpleProject/... //jenkins_trunk_simpleProject/...

Changing P4 Client View to:
//depot/sandboxes/smacdonald/projects/simpleProjectDependency/... //jenkins_trunk_simpleProject/...
Saving modified client jenkins_trunk_simpleProject [workspace] $ /usr/bin/p4 -s client -i Last build changeset: 0 [workspace] $ /usr/bin/p4 counter change [workspace] $ /usr/bin/p4 -s changes //jenkins_trunk_simpleProject/...@40492,@40541
[workspace] $ /usr/bin/p4 describe -s 40541 [workspace] $ /usr/bin/p4 -G where //depot/sandboxes/smacdonald/projects/simpleProject/pom.xml
FATAL: null
java.lang.NullPointerException
at hudson.plugins.perforce.PerforceSCMHelper.parseWhereMapping(PerforceSCMHelper.java:133)
at com.tek42.perforce.parse.Changes.calculateWorkspacePaths(Changes.java:78)
at com.tek42.perforce.parse.Changes.getChangelist(Changes.java:67)
at com.tek42.perforce.parse.Changes.getChangelistsFromNumbers(Changes.java:403)
at hudson.plugins.perforce.PerforceSCM.checkout(PerforceSCM.java:628)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1184)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:537)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:425)
at hudson.model.Run.run(Run.java:1376)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:466)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:175)

Reopening...

Scott, please run the following command on the build machine. Be sure to replace the P4USER, P4PASS, and P4PORT with the values defined in the plugin settings.

/usr/bin/p4 -u P4USER -P P4PASS -p P4PORT -c jenkins_trunk_simpleProject -G where //depot/sandboxes/smacdonald/projects/simpleProject/pom.xml > whereData.dat

And attach the resulting file 'whereData.dat'.

Rob, I got another NPE the afternoon before going on a 1 week vacation, so I didn't capture the -G output and we reverted back to the release plugin. I'll update the plugin source, rebuild and deploy and if any new NPEs occur, I'll send you the output via gmail. BTW, I have Google+ invites if you're interested...

I really need that output, or else I can't figure out what the problem is. Since you know you got an NPE, I'm assuming you still have the log? Can't you just run the command it failed on and attach the output? BTW, this functionality was released with 1.2.8, so you don't need to build from source.

Thanks for the G+ offer, but I've already got one.

Attaching -G output.

The test successfully parses that output...

Does this happen every time on the same file, or is it random?

Rob, I got this same issue yesterday. Turned out the issue was caused by a changelist that had 1 file that was not in my workspace view. Not sure if that is the same issue here.

Code changed in jenkins
User: Rob Petti
Path:
src/main/java/com/tek42/perforce/parse/Changes.java
src/main/java/hudson/plugins/perforce/PerforceChangeLogEntry.java
src/main/java/hudson/plugins/perforce/PerforceSCMHelper.java
src/test/java/hudson/plugins/perforce/PerforceSCMHelperTest.java
http://jenkins-ci.org/commit/perforce-plugin/ba734198ade6038ff0dcbc0671009ecff6763809
Log:
JENKINS-7618 improvements to where parsing

• correctly handle cases where the file isn't in the client workspace
• getAffectedPaths will only provide files that are in the client workspace
• added retry in the event of data corruption

I've added a retry in the event that the data gets corrupted for some reason, which I think should solve Scott's issue. It should also now correctly handle files that are not in the client-workspace. You can grab the snapshot with these changes here: http://files.robpetti.com/perforce-plugin/target/perforce.hpi

I'll gladly grab the snapshot and give it a try as we are still see this issue y intermittently on perforce plugin 1.2.8 ( never on 1.2.7) That said, I have to plead jenkins newbie. What do I do with the perforce.hpi file?

Install it using the Jenkins plugin manager interface. http://<yourJenkinsServer>/pluginManager/advanced

I poked around install dir bit. I assume I just drop it in plugin dir . I'll give it a try Get back to you.

Ahhh Ok I'll do that.

After I go to http://<yourJenkinsServer>/pluginManager/advanced. browse to the perforce.hpi, and select upload, jenkins then shows me the "update center" showing me that I have 1.2.7 installed and that there is a 1.2.8 update available. How do I know if I'm running the snapshot version with the fix? Is the snapshot version 1.2.7 and is that act of uploading it, sufficient to expect that I'm now running that snapshot version? (we never saw this issue on 1.2.7 so I just want to be sure I'm actually running the snapshot with the fix)

You need to restart your Jenkins instance in order to upgrade/install plugins. The version number should show up as 1.2.9-SNAPSHOT.

Ran a couple builds on 1.2.9-SNAPSHOT and so far all is well. I'll give a few days soak time and get back to you.

I've been running this snapshot build since it was built, and I haven't seen any other issues.

Integrated in plugins_perforce #117
JENKINS-7618 improvements to where parsing

Rob Petti :
Files :

• src/main/java/hudson/plugins/perforce/PerforceChangeLogEntry.java
• src/main/java/com/tek42/perforce/parse/Changes.java
• src/main/java/hudson/plugins/perforce/PerforceSCMHelper.java
• src/test/java/hudson/plugins/perforce/PerforceSCMHelperTest.java

Resolving as fixed.