[JENKINS-14814] "Ping-pong" builds store excessively large CauseAction - Jenkins Jira

Details

Type: Bug
Status: Resolved (View Workflow)
Priority: Major
Resolution: Fixed
Component/s: core
Labels:

Similar Issues:

Show

Description

Create two jobs, a and b, each of which does nothing but trigger the other. (Useless by itself, but a similar idiom can be used to alternate indefinitely between two actions, such as checking for changes in some part of the environment and reacting to those changes.)

Let this run for a few dozen iterations. The build.xml file for each build can soon run into multiple megabytes. For example, on just the fourth cycle, a/builds/4/build.xml has this lengthy "cause" section:

<hudson.model.CauseAction>
  <causes>
    <hudson.model.Cause_-UpstreamCause>
      <upstreamProject>b</upstreamProject>
      <upstreamUrl>job/b/</upstreamUrl>
      <upstreamBuild>3</upstreamBuild>
      <upstreamCauses>
        <hudson.model.Cause_-UpstreamCause>
          <upstreamProject>a</upstreamProject>
          <upstreamUrl>job/a/</upstreamUrl>
          <upstreamBuild>3</upstreamBuild>
          <upstreamCauses>
            <hudson.model.Cause_-UpstreamCause>
              <upstreamProject>b</upstreamProject>
              <upstreamUrl>job/b/</upstreamUrl>
              <upstreamBuild>2</upstreamBuild>
              <upstreamCauses>
                <hudson.model.Cause_-UpstreamCause>
                  <upstreamProject>a</upstreamProject>
                  <upstreamUrl>job/a/</upstreamUrl>
                  <upstreamBuild>2</upstreamBuild>
                  <upstreamCauses>
                    <hudson.model.Cause_-UpstreamCause>
                      <upstreamProject>b</upstreamProject>
                      <upstreamUrl>job/b/</upstreamUrl>
                      <upstreamBuild>1</upstreamBuild>
                      <upstreamCauses>
                        <hudson.model.Cause_-UpstreamCause>
                          <upstreamProject>a</upstreamProject>
                          <upstreamUrl>job/a/</upstreamUrl>
                          <upstreamBuild>1</upstreamBuild>
                          <upstreamCauses>
                            <hudson.model.Cause_-UserIdCause/>
                          </upstreamCauses>
                        </hudson.model.Cause_-UpstreamCause>
                      </upstreamCauses>
                    </hudson.model.Cause_-UpstreamCause>
                  </upstreamCauses>
                </hudson.model.Cause_-UpstreamCause>
              </upstreamCauses>
            </hudson.model.Cause_-UpstreamCause>
          </upstreamCauses>
        </hudson.model.Cause_-UpstreamCause>
      </upstreamCauses>
    </hudson.model.Cause_-UpstreamCause>
  </causes>
</hudson.model.CauseAction>

Excessive disk usage consumption is not the only problem; Jenkins may even refuse to start, since loading one of these files can produce a stack overflow:

hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:180) 
hudson.util.XStream2$PassthruConverter.unmarshal(XStream2.java:323) 
hudson.util.XStream2$AssociatedConverterImpl.unmarshal(XStream2.java:293) 
com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82) 
com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63) 
com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76) 
com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60) 
com.thoughtworks.xstream.converters.collections.AbstractCollectionConverter.readItem(AbstractCollectionConverter.java:71) 
hudson.util.RobustCollectionConverter.populateCollection(RobustCollectionConverter.java:85) 
com.thoughtworks.xstream.converters.collections.CollectionConverter.unmarshal(CollectionConverter.java:61) 
hudson.util.RobustCollectionConverter.unmarshal(RobustCollectionConverter.java:76) 
com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:82) 
com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:63) 
com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:76) 
com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:60) 
hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:290) 
hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:233) 
hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:180)

While recording transitive causes may be useful in general, there must be some sanity cutoff point.

Attachments

Issue Links

depends on

JENKINS-24161 StackOverflowError from UpstreamCause.onLoad

Resolved

is related to

JENKINS-6338 Job's cyclic dependency not detected

Open

JENKINS-14818 Stack overflow when jobs are mutually downstream

Open

JENKINS-15747 Logging all UpstreamCause's floods Jenkins in large setups

Resolved

JENKINS-14816 404 from Cause/UpstreamCause/description.jelly

Resolved

Activity

Ascending order - Click to sort in descending order

View 4 older comments

SCM/JIRA link daemon added a comment - 2012-08-15 22:17

Code changed in jenkins
User: Jesse Glick
Path:
changelog.html
http://jenkins-ci.org/commit/jenkins/2aa272a0fae0390eb2da679ab7c9e0d402b1f53b
Log:
~~JENKINS-14814~~ Noting.

SCM/JIRA link daemon added a comment - 2012-08-15 22:17 Code changed in jenkins User: Jesse Glick Path: changelog.html http://jenkins-ci.org/commit/jenkins/2aa272a0fae0390eb2da679ab7c9e0d402b1f53b Log: JENKINS-14814 Noting.

Jesse Glick added a comment - 2012-08-16 12:15

Workaround:

perl -i.bak -n -e 'print unless m{<hudson\.model\.CauseAction>}..m{</hudson\.model\.CauseAction>}' jobs/*/builds/*-*/build.xml

Jesse Glick added a comment - 2012-08-16 12:15 Workaround: perl -i.bak -n -e 'print unless m{<hudson\.model\.CauseAction>}..m{</hudson\.model\.CauseAction>}' jobs/*/builds/*-*/build.xml

dogfood added a comment - 2012-08-23 19:19

Integrated in jenkins_main_trunk #1856
~~JENKINS-14814~~ Reproduced. (Revision b0278767831d751b93ad77dc2c5c92b6d140e865)
[FIXED JENKINS-14814] Truncate deeply nested upstream causes. (Revision bfa648152f266540536e76f7b3ebcb1598ad5f6f)
~~JENKINS-14814~~ Noting. (Revision 2aa272a0fae0390eb2da679ab7c9e0d402b1f53b)

Result = UNSTABLE
Jesse Glick : b0278767831d751b93ad77dc2c5c92b6d140e865
Files :

test/src/test/java/hudson/model/CauseTest.java

Jesse Glick : bfa648152f266540536e76f7b3ebcb1598ad5f6f
Files :

core/src/main/java/hudson/model/Cause.java

Jesse Glick : 2aa272a0fae0390eb2da679ab7c9e0d402b1f53b
Files :

changelog.html

dogfood added a comment - 2012-08-23 19:19 Integrated in jenkins_main_trunk #1856 JENKINS-14814 Reproduced. (Revision b0278767831d751b93ad77dc2c5c92b6d140e865) [FIXED JENKINS-14814] Truncate deeply nested upstream causes. (Revision bfa648152f266540536e76f7b3ebcb1598ad5f6f) JENKINS-14814 Noting. (Revision 2aa272a0fae0390eb2da679ab7c9e0d402b1f53b) Result = UNSTABLE Jesse Glick : b0278767831d751b93ad77dc2c5c92b6d140e865 Files : test/src/test/java/hudson/model/CauseTest.java Jesse Glick : bfa648152f266540536e76f7b3ebcb1598ad5f6f Files : core/src/main/java/hudson/model/Cause.java Jesse Glick : 2aa272a0fae0390eb2da679ab7c9e0d402b1f53b Files : changelog.html

Meng Xin Zhu added a comment - 2013-02-23 14:37

Do you know which build starts to have this fix? Because I still see similar issue happened in LTS 1.480.2.

Meng Xin Zhu added a comment - 2013-02-23 14:37 Do you know which build starts to have this fix? Because I still see similar issue happened in LTS 1.480.2.

Jesse Glick added a comment - 2013-02-25 15:45

Should be in 1.479, but you might be hitting ~~JENKINS-15747~~.

Jesse Glick added a comment - 2013-02-25 15:45 Should be in 1.479, but you might be hitting JENKINS-15747 .

Jenkins

"Ping-pong" builds store excessively large CauseAction

Details

Description

Attachments

Issue Links

Activity

People

Dates