[JENKINS-62442] Lots of changes in source repo results in malformed XML Plastic->Jenkins which fails build job

Type: Bug
Resolution: Fixed
Priority: Blocker
Component/s: plasticscm-plugin
Labels:
None
Environment:
Jenkins 2.238, Plastic SCM plugin 3.3, Plastic SCM client 9.0.16.4255

Similar Issues:
Powered by SuggestiMate

Show
Released As:
https://github.com/jenkinsci/plasticscm-plugin/blob/master/CHANGELOG.md#version-34-07-oct-2020

We recently performed a significant update of a sub-repository. (Significant = 150k changed files). Now, when Jenkins attempts to build a project related to the parent repository, something goes wrong when Plastic is to provide Jenkins with the list of changes.

It begins normally, like so:

[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe unco --all --silent c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe ss wk:shl-2143265103
Selector for workspace shl-2143265103:
repository "JenkinsBuildScripts@MyOrg@cloud"
  path "/"
    smartbranch "/main"[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe update c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
Searching for changed items in the workspace...
The workspace c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts is up-to-date (cset:119@JenkinsBuildScripts@MyOrg@cloud)
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe status --cset C:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
cs:119@rep:JenkinsBuildScripts@repserver:MyOrg@cloud
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe log cs:119 --xml --encoding=utf-8

That command begins to emit XML...

<?xml version="1.0" encoding="utf-8"?>
<LogList>
  <Changeset>
    <ObjId>3026</ObjId>
    <ChangesetId>119</ChangesetId>
    <Branch>/main</Branch>
    <Comment>Added "sendSlackBuildSucceededNotification_custom" command, which will allow us to customize the info part (arbitrary download links etc)</Comment>
    <Owner>one@email.address</Owner>
    <GUID>34134429-84ea-4667-8084-d80eb4e9eadd</GUID>
    <Changes>
      <Item>
        <Branch>/main</Branch>
        <RevNo>119</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>3002</RevId>
        <ParentRevId>789</ParentRevId>
        <SrcCmPath>/src/com/falldamagestudio/FormatSlackNotification.groovy</SrcCmPath>
        <SrcParentItemId>159</SrcParentItemId>
        <DstCmPath>/src/com/falldamagestudio/FormatSlackNotification.groovy</DstCmPath>
        <DstParentItemId>159</DstParentItemId>
        <Date>2020-03-23T08:15:49.0000000+01:00</Date>
        <Type>Changed</Type>
      </Item>
      <Item>
        <Branch>/main</Branch>
        <RevNo>119</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>3004</RevId>
        <ParentRevId>791</ParentRevId>
        <SrcCmPath>/test/groovy/FormatSlackNotification/FormatSlackNotificationTest.groovy</SrcCmPath>
        <SrcParentItemId>631</SrcParentItemId>
        <DstCmPath>/test/groovy/FormatSlackNotification/FormatSlackNotificationTest.groovy</DstCmPath>
        <DstParentItemId>631</DstParentItemId>
        <Date>2020-03-23T08:15:49.0000000+01:00</Date>
        <Type>Changed</Type>
      </Item>
...

... snip 55MB ...

... and toward the end, it looks like this:

...
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>636454</RevId>
        <ParentRevId>-1</ParentRevId>
        <SrcCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_CN.txt</SrcCmPath>
        <SrcParentItemId>758792</SrcParentItemId>
        <DstCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_CN.txt</DstCmPath>
        <DstParentItemId>758792</DstParentItemId>
        <Date>2020-05-20T22:54:20.0000000+02:00</Date>
        <Type>Added</Type>
      </Item>
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>636455</RevId>
        <ParentRevId>-1</ParentRevId>
        <SrcCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_Hans.txt</SrcCmPath>
        <SrcParentItemId>758792</SrcParentItemId>
        <DstCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_Hans.txt</DstCmPath>
        <DstParentItemId>758792</DstParentItemId>FATAL: Parse error: XML document structures must start and end within the same entity.

Notice how the XML tags aren't closed properly.

We have tried kicking the same build job, 3-4 times, with cutoff happning after 44-65MB of output. On one occasion the cutoff was in the middle of an XML tag:

...
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevFATAL: Parse error: XML document structures must start and end within the same entity.

Given the above, it seems that either cm.exe is truncating its output while writing, the plasticscm plugin is truncating the stream while reading, or Jenkins is truncating the stream during transmission/ingestion. I'm not sure where exactly the problem is located.

After this error, the pipeline fails. This stops our build jobs dead. We do not need the change info, but also, we do not see a way to navigate around this blocker.

Mikael Kalms created issue - 2020-05-25 13:49

Mikael Kalms made changes - 2020-05-25 14:59

Environment

Original: Jenkins 2.226, Plastic SCM plugin 3.3

New: Jenkins 2.238, Plastic SCM plugin 3.3, Plastic SCM client 9.0.16.4255

Mikael Kalms made changes - 2020-05-25 14:59

Description

Original: We recently performed a significant update of a sub-repository. (Significant = 150k changed files). Now, when Jenkins attempts to build a project related to the parent repository, something goes wrong when Plastic is to provide Jenkins with the list of changes.

It begins normally, like so:

{noformat}
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe unco --all --silent c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe ss wk:shl-2143265103
Selector for workspace shl-2143265103:
repository "JenkinsBuildScripts@MyOrg@cloud"
  path "/"
    smartbranch "/main"[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe update c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
Searching for changed items in the workspace...
The workspace c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts is up-to-date (cset:119@JenkinsBuildScripts@MyOrg@cloud)
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe status --cset C:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
cs:119@rep:JenkinsBuildScripts@repserver:MyOrg@cloud
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe log cs:119 --xml --encoding=utf-8
{noformat}
That command begins to emit XML...

{noformat}
<?xml version="1.0" encoding="utf-8"?>
<LogList>
  <Changeset>
    <ObjId>3026</ObjId>
    <ChangesetId>119</ChangesetId>
    <Branch>/main</Branch>
    <Comment>Added "sendSlackBuildSucceededNotification_custom" command, which will allow us to customize the info part (arbitrary download links etc)</Comment>
    <Owner>one@email.address</Owner>
    <GUID>34134429-84ea-4667-8084-d80eb4e9eadd</GUID>
    <Changes>
      <Item>
        <Branch>/main</Branch>
        <RevNo>119</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>3002</RevId>
        <ParentRevId>789</ParentRevId>
        <SrcCmPath>/src/com/falldamagestudio/FormatSlackNotification.groovy</SrcCmPath>
        <SrcParentItemId>159</SrcParentItemId>
        <DstCmPath>/src/com/falldamagestudio/FormatSlackNotification.groovy</DstCmPath>
        <DstParentItemId>159</DstParentItemId>
        <Date>2020-03-23T08:15:49.0000000+01:00</Date>
        <Type>Changed</Type>
      </Item>
      <Item>
        <Branch>/main</Branch>
        <RevNo>119</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>3004</RevId>
        <ParentRevId>791</ParentRevId>
        <SrcCmPath>/test/groovy/FormatSlackNotification/FormatSlackNotificationTest.groovy</SrcCmPath>
        <SrcParentItemId>631</SrcParentItemId>
        <DstCmPath>/test/groovy/FormatSlackNotification/FormatSlackNotificationTest.groovy</DstCmPath>
        <DstParentItemId>631</DstParentItemId>
        <Date>2020-03-23T08:15:49.0000000+01:00</Date>
        <Type>Changed</Type>
      </Item>
...{noformat}
... snip 55MB ...

... and toward the end, it looks like this:

{noformat}
...
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>636454</RevId>
        <ParentRevId>-1</ParentRevId>
        <SrcCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_CN.txt</SrcCmPath>
        <SrcParentItemId>758792</SrcParentItemId>
        <DstCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_CN.txt</DstCmPath>
        <DstParentItemId>758792</DstParentItemId>
        <Date>2020-05-20T22:54:20.0000000+02:00</Date>
        <Type>Added</Type>
      </Item>
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>636455</RevId>
        <ParentRevId>-1</ParentRevId>
        <SrcCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_Hans.txt</SrcCmPath>
        <SrcParentItemId>758792</SrcParentItemId>
        <DstCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_Hans.txt</DstCmPath>
        <DstParentItemId>758792</DstParentItemId>FATAL: Parse error: XML document structures must start and end within the same entity.{noformat}
Notice how the XML tags aren't closed properly.

We have tried kicking the same build job, 3-4 times, with cutoff happning after 44-55MB of output. On one occasion the cutoff was in the middle of an XML tag:
{noformat}
...
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevFATAL: Parse error: XML document structures must start and end within the same entity.{noformat}
Given the above, it seems that either cm.exe is truncating its output while writing, the plasticscm plugin is truncating the stream while reading, or Jenkins is truncating the stream during transmission/ingestion. I'm not sure where exactly the problem is located.

After this error, the pipeline fails. This stops our build jobs dead. We do not need the change info, but also, we do not see a way to navigate around this blocker.

New: We recently performed a significant update of a sub-repository. (Significant = 150k changed files). Now, when Jenkins attempts to build a project related to the parent repository, something goes wrong when Plastic is to provide Jenkins with the list of changes.

It begins normally, like so:

{noformat}
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe unco --all --silent c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe ss wk:shl-2143265103
Selector for workspace shl-2143265103:
repository "JenkinsBuildScripts@MyOrg@cloud"
  path "/"
    smartbranch "/main"[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe update c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
Searching for changed items in the workspace...
The workspace c:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts is up-to-date (cset:119@JenkinsBuildScripts@MyOrg@cloud)
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe status --cset C:\Jenkins\workspace\MyJob@libs\JenkinsBuildScripts
cs:119@rep:JenkinsBuildScripts@repserver:MyOrg@cloud
[JenkinsBuildScripts] $ C:\PlasticSCM\client\cm.exe log cs:119 --xml --encoding=utf-8
{noformat}
That command begins to emit XML...

{noformat}
<?xml version="1.0" encoding="utf-8"?>
<LogList>
  <Changeset>
    <ObjId>3026</ObjId>
    <ChangesetId>119</ChangesetId>
    <Branch>/main</Branch>
    <Comment>Added "sendSlackBuildSucceededNotification_custom" command, which will allow us to customize the info part (arbitrary download links etc)</Comment>
    <Owner>one@email.address</Owner>
    <GUID>34134429-84ea-4667-8084-d80eb4e9eadd</GUID>
    <Changes>
      <Item>
        <Branch>/main</Branch>
        <RevNo>119</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>3002</RevId>
        <ParentRevId>789</ParentRevId>
        <SrcCmPath>/src/com/falldamagestudio/FormatSlackNotification.groovy</SrcCmPath>
        <SrcParentItemId>159</SrcParentItemId>
        <DstCmPath>/src/com/falldamagestudio/FormatSlackNotification.groovy</DstCmPath>
        <DstParentItemId>159</DstParentItemId>
        <Date>2020-03-23T08:15:49.0000000+01:00</Date>
        <Type>Changed</Type>
      </Item>
      <Item>
        <Branch>/main</Branch>
        <RevNo>119</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>3004</RevId>
        <ParentRevId>791</ParentRevId>
        <SrcCmPath>/test/groovy/FormatSlackNotification/FormatSlackNotificationTest.groovy</SrcCmPath>
        <SrcParentItemId>631</SrcParentItemId>
        <DstCmPath>/test/groovy/FormatSlackNotification/FormatSlackNotificationTest.groovy</DstCmPath>
        <DstParentItemId>631</DstParentItemId>
        <Date>2020-03-23T08:15:49.0000000+01:00</Date>
        <Type>Changed</Type>
      </Item>
...{noformat}
... snip 55MB ...

... and toward the end, it looks like this:

{noformat}
...
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>636454</RevId>
        <ParentRevId>-1</ParentRevId>
        <SrcCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_CN.txt</SrcCmPath>
        <SrcParentItemId>758792</SrcParentItemId>
        <DstCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_CN.txt</DstCmPath>
        <DstParentItemId>758792</DstParentItemId>
        <Date>2020-05-20T22:54:20.0000000+02:00</Date>
        <Type>Added</Type>
      </Item>
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevId>636455</RevId>
        <ParentRevId>-1</ParentRevId>
        <SrcCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_Hans.txt</SrcCmPath>
        <SrcParentItemId>758792</SrcParentItemId>
        <DstCmPath>/UE4/Engine/Source/ThirdParty/ICU/icu4c-53_1/source/data/coll/zh_Hans.txt</DstCmPath>
        <DstParentItemId>758792</DstParentItemId>FATAL: Parse error: XML document structures must start and end within the same entity.{noformat}
Notice how the XML tags aren't closed properly.

We have tried kicking the same build job, 3-4 times, with cutoff happning after 44-65MB of output. On one occasion the cutoff was in the middle of an XML tag:
{noformat}
...
      <Item>
        <Branch />
        <RevNo>12</RevNo>
        <Owner>kalms@falldamagestudio.com</Owner>
        <RevFATAL: Parse error: XML document structures must start and end within the same entity.{noformat}
Given the above, it seems that either cm.exe is truncating its output while writing, the plasticscm plugin is truncating the stream while reading, or Jenkins is truncating the stream during transmission/ingestion. I'm not sure where exactly the problem is located.

After this error, the pipeline fails. This stops our build jobs dead. We do not need the change info, but also, we do not see a way to navigate around this blocker.

Miguel González made changes - 2020-11-13 06:40

Assignee

New: Miguel González [ mig42 ]

Miguel González made changes - 2020-11-13 06:45

Released As		New: https://github.com/jenkinsci/plasticscm-plugin/blob/master/CHANGELOG.md#version-34-07-oct-2020
Resolution		New: Fixed [ 1 ]
Status	Original: Open [ 1 ]	New: Fixed but Unreleased [ 10203 ]

Miguel González made changes - 2020-11-13 06:45

Status

Original: Fixed but Unreleased [ 10203 ]

New: Resolved [ 5 ]

Miguel González made changes - 2020-11-13 06:45

Status

Original: Resolved [ 5 ]

New: Closed [ 6 ]

Assignee:: Miguel González

Reporter:: Mikael Kalms

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Created:: 2020-05-25 13:49

Updated:: 2020-11-13 06:45

Resolved:: 2020-11-13 06:45

Jenkins

Details

Description

Attachments

Activity

People

Dates