-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
Powered by SuggestiMate
Exception:
ERROR: [withMaven] WARNING Exception parsing the logs generated by the Jenkins Maven Event Spy /var/lib/jenkins/workspace/_appia=gui/F__appia_-_gui_-_master@tmp/withMaven1afdee4c/maven-spy-20170831-215714-662.log, ignore file. Please report a bug associated for the component 'pipeline-maven-plugin' at https://issues.jenkins-ci.org ERROR: org.xml.sax.SAXParseException; lineNumber: 3824; columnNumber: 53; XML document structures must start and end within the same entity.
- is related to
-
JENKINS-43449 Ensure that the maven-events-spy is thread safe
-
- Closed
-
-
JENKINS-46609 withMaven should detect when a maven build was interrupted instead of reporting an XML parsing exception
-
- Closed
-
[JENKINS-46579] Maven Event Spy seems to still have a thread safe problem
I am aware that an issue regarding this was resolved. However, running 3.0.0 with a threaded build results in this error from time to time.
Hi,
Can you attach one of the broken log files? Also any information you could give us about the pipeline you are using and the pom, etc would be helpful to try to reproduce it.
Yes I will try to give you more info.
I am debugging right now as this is important to us to keep the trust in the ci.
However, quickly glancing at the code I am wondering if the source of the problem is not the onEvent method. Nothing prevents the different threads to output content in the middle of another unfinished task. I know the underlying classes use thread safe operations but handlers will use more than 1 output operation and therefore another thread can output stuff right in the middle of it.
But that's just speculation after looking at the code on my cellphone fast fast.
I'm not sure if that is the problem given that all the handlers print in a single operation passing the Xpp3Dom object and the method is thread safe.
That's funny.
I was about to write you about the exact same thing. Because I just isolated the problem and it looks to be it.
I don't think it's ever a good idea to use a timestamp as a "unique" id for anything. I will run some tests but I am pretty sure that's the problem as I have added output and see 2 outFile with the same name.
Ok, so yeah I've got more than 1 case. There one where if the build is reentrant and 2 or more concurrent builds run we might get the same file name.
However, this is not the only case. I am trying to package details on the problem.
Yes I found that one and already have a fix. Will push that fix while waiting for the details on the other
I don't think it is two builds at the same time but a fork on the maven process. Anyway I had already proved that issue with a unit test and have a fix, I'm sending the PR in a while.
So, after many many builds the pattern seems to be that maven spy log is incomplete and of course it fails.
Looks like this:
<ExecutionEvent type="MojoStarted" class="org.apache.maven.lifecycle.internal.DefaultExecutionEvent" _time="2017-09-01 21:05:52.118"> <project baseDir="/var/lib/jenkins/workspace/_appia=platform/F__appia_-_platform_-_master/ejb/user/ejb-client" file="/var/lib/jenkins/workspace/_appia=platform/F__appia_-_platform_-_master/ejb/user/ejb-client/pom.xml" groupId="com.expretio.appia.ejbs" name="Appia Platform : EJB : User Client" artifactId="appia-user-ejb-client" version="4.54.0-SNAPSHOT"> <build sourceDirectory="/var/lib/jenkins/workspace/_appia=platform/F__appia_-_platform_-_master/ejb/user/ejb-client/src/main/java" directory="/var/lib/jenkins/workspace/_appia=platform/F__appia_-_platform_-_master/ejb/user/ejb-client/target"/> </project> <plugin executionId="default-cli" goal="jar" groupId="org.apache.maven.plugins" artifactId="maven-javadoc-plugin" version="2.10.4"> <additionalJOption>${additionalJOption}</additionalJOption> <additionalparam>${additionalparam}</additionalparam> <aggregate>${aggregate}</aggregate> <applyJavadocSecurityFix>${maven.javadoc.applyJavadocSecurityFix}</applyJavadocSecurityFix> <attach>${attach}</attach> <author>${author}</author> <bootclasspath>${bootclasspath}</bootclasspath> <bootclasspathArtifacts>${bootclasspathArtifacts}</bootclasspathArtifacts> <bottom>${bottom}</bottom> <breakiterator>${breakiterator}</breakiterator> <charset>${charset}</charset> <classifier>${maven.javadoc.classifier}</classifier> <debug>${debug}</debug> <defaultManifestFile>${project.build.outputDirectory}/META-INF/MANIFEST.MF</defaultManifestFile> <destDir>${destDir}</destDir> <detectJavaApiLink>${detectJavaApiLink}</detectJavaApiLink> <detectLinks>${detectLinks}</detectLinks> <detectOfflineLinks>${detectOfflineLinks}</detectOfflineLinks> <docencoding>${docencoding}</docencoding> <docfilessubdirs>${docfilessubdirs}</docfilessubdirs> <doclet>${doclet}</doclet> <docletArtifact>${docletArtifact}</docletArtifact> <docletArtifacts>${docletArtifacts}</docletArtifacts> <docletPath>${docletPath}</docletPath> <doctitle>${doctitle}</doctitle> <encoding>${encoding}</encoding> <excludePackageNames>${excludePackageNames}</excludePackageNames> <excludedocfilessubdir>${excludedocfilessubdir}</excludedocfilessubdir> <extdirs>${extdirs}</extdirs> <failOnError>${maven.javadoc.failOnError}</failOnError> <finalName>${project.build.finalName}</finalName> <footer>${footer}</footer> <groups>${groups}</groups> <header>${header}</header> <helpfile>${helpfile}</helpfile> <includeDependencySources>false</includeDependencySources> <includeTransitiveDependencySources>false</includeTransitiveDependencySources> <isOffline>${settings.offline}</isOffline> <jarOutputDirectory>${project.build.directory}</jarOutputDirectory> <javaApiLinks>${javaApiLinks}</javaApiLinks> <javadocDirectory>${basedir}/src/main/javadoc</javadocDirectory> <javadocExecutable>${javadocExecutable}</javadocExecutable> <javadocOptionsDir>${project.build.directory}/javadoc-bundle-options</javadocOptionsDir> <javadocVersion>${javadocVersion}</javadocVersion> <keywords>${keywords}</keywords> <links>${links}</links> <linksource>${linksource}</linksource> <localRepository>${localRepository}</localRepository> <locale>${locale}</locale> <maxmemory>${maxmemory}</maxmemory> <minmemory>${minmemory}</minmemory> <nocomment>${nocomment}</nocomment> <nodeprecated>${nodeprecated}</nodeprecated> <nodeprecatedlist>${nodeprecatedlist}</nodeprecatedlist> <nohelp>${nohelp}</nohelp> <noindex>${noindex}</noindex> <nonavbar>${nonavbar}</nonavbar> <nooverview>${nooverview}</nooverview> <noqualifier>${noqualifier}</noqualifier> <nosince>${nosince}</nosince> <notimestamp>${notimestamp}</notimestamp> <notree>${no
Is it possible the StepContext is notified that the executioon ended before it actually ended? Or maybe the something is not properly "released"...
I have attached a build log where it fails, there's some debugging output. It looks like maven isn't done when maven spy fails.
mvn -Dmaven.test.failure.ignore=true -Dmaven.javadoc.failOnError=false -e -U -T 2 clean deploy -DaltDeploymentRepository=xo-snapshots::default::http://repo.expretio.com/artifactory/m2-xo-snapshots
Code changed in jenkins
User: Alvaro Lobato
Path:
maven-spy/src/main/java/org/jenkinsci/plugins/pipeline/maven/eventspy/reporter/FileMavenEventReporter.java
maven-spy/src/test/java/org/jenkinsci/plugins/pipeline/maven/eventspy/JenkinsMavenEventSpyMTTest.java
http://jenkins-ci.org/commit/pipeline-maven-plugin/810fe9a4c7a61c4c7ccb7a35636811e22d437480
Log:
JENKINS-46579 - Concurrency problems with spy log (#91)
JENKINS-46579 maven event spy: Fix concurrency issue in the handling of the output file
FYI I have created JENKINS-46609 . pino could the Maven process could have been killed by the operating system? I saw this problem in docker/cgroup environments.
Well well, that was it. The maven process was getting killed for no apparent reason. I played with the java Xmx and it looks much better now. I don't understand why it would fail with Xmx3G when in dev we use Xmx2G but I have increased this to 4G and it seems to work.
Can you clarify which Xmx you are referring to?
Do you run this process in a Docker container or in cgroups?
Do the memory of this agents fits with the sumof the memory of all the known processes ? Slave.jar + mvn process + maybe mvn sub processes
Can you monitor the memory, CPU and load of this agent to detect problems?
Fixed by https://repo.jenkins-ci.org/releases/org/jenkins-ci/plugins/pipeline-maven/3.0.1-beta-1/pipeline-maven-3.0.1-beta-1.hpi
We detect when the Maven build has been killed and don't display this confusing XML parsing exception. See Pipeline Maven Plugin> Why do I see messages...
pino could you test the 3.0.1-beta?
https://github.com/jenkinsci/pipeline-maven-plugin/releases/tag/pipeline-maven-3.0.1-beta-2
Thanks pino, as I have already tested, we may release before you are back from holidays
we run version 3.6.11 and still see this from time to time:
ERROR: [withMaven] WARNING Exception parsing the logs generated by the Jenkins Maven Event Spy /home/ec2-user/workspace/org_yooture_develop@tmp/withMaven5705f0d0/maven-spy-20190520-135648-9715492809875305253764.log, ignore file. Please report a bug associated for the component 'pipeline-maven-plugin' at https://issues.jenkins-ci.org ERROR: org.xml.sax.SAXParseException; lineNumber: 18407; columnNumber: 263; XML document structures must start and end within the same entity.
Hello pino, we have fixed such issue in version 2.0.1 on April 2017. What version do you use?
As you say "still have", you seem to be in a more recent version.