-
Bug
-
Resolution: Unresolved
-
Minor
-
None
-
Jenkins 2.14 on Windows Server 2012, Job Configuration History Plugin 2.15, Groovy 2.4, java 8
Jenkins 2.46.1 on Ubuntu 16.04
When using the CLI or Rest API, Jenkins is saving the config.xml a different xml than I am providing. Specifically it is removing the first EOL, adding carriage returns, and replacing empty xml tags with self closing ones.
I manage Jenkins instances for my organization. Each has a "test" instance that is a mirror of the prod one. Developers can experiment with things like changes to jobs and interactions with new plugins on the test version and we can port them to the prod one where configure permissions are disabled. For a first time copy the Job Import Plugin works but for edits to existing ones I have to either manually duplicate the changes or script something.
I have tried this with both the CLI jar and groovy rest client. I get the config.xml from the test and prod servers. I perform a diff which lets me quickly see what the changes are and if approved will update the prod instance (job-update for CLI, POST for rest).
If a job is edited in the Jenkins UI the config.xml is saved with newlines. When updated through one of the remote tools it is saved with carriage returns and newlines. So, if I get the xml I just updated and diff it to the one I just sent they will register as completely different.
The config xmls retrieved will have carriage returns if the last modification was was from a remote update or just newlines if the last edit was in the UI.
I have looked at the requests I am sending and they do not have carriage returns. I looked at the code for CLI saves and it is creating a file from the parsed xml model rather than the supplied xml string. It removes a line break between the xml preamble and root <project> tag, saves each line with a carriage return and writes out self-closing tags. [edit: last sentence is describing observed behaviour not something seen in the code]
This messes up the Job Configuration History plugin as a one line change displays as if every line is changed. This also interferes with my local diff before sending the update.
The two images attached (missing-eol and exact-same-config) are from back to back runs of the update script, meaning the configurations are exactly the same. The config history plugin thinks the entire file has been changed. Even with a manual correction for the carriage returns, my local diff shows differences because of the tag differences.
The third (ui-save-no-change) is from a manual save in the UI (click "configure", click "save" with no changes). With no content change, we see that the <project> tag is again on its own line and
If I'm not mistaken about the cause, is it feasible to have Jenkins save the configuration consistently regardless of how the configuration has changed?
- is related to
-
JENKINS-41178 Ignore line endings in diff
-
- Open
-
[JENKINS-41177] Identical job configuration can have different config.xml based on edit method.
Paul G
created issue -
Paul G
made changes -
| Description |
Original:
When using the CLI or Rest API, Jenkins is saving the config.xml a different xml than I am providing. Specifically it is removing the first EOL, adding carriage returns, and replacing empty xml tags with self closing ones. I manage Jenkins instances for my organization. Each has a "test" instance that is a mirror of the prod one. Developers can experiment with things like changes to jobs and interactions with new plugins on the test version and we can port them to the prod one where configure permissions are disabled. For a first time copy the Job Import Plugin works but for edits to existing ones I have to either manually duplicate the changes or script something. I have tried this with both the CLI jar and groovy rest client. I get the config.xml from the test and prod servers. I perform a diff which lets me quickly see what the changes are and if approved will update the prod instance (job-update for CLI, POST for rest). If a job is edited in the Jenkins UI the config.xml is saved with newlines. When updated through one of the remote tools it is saved with carriage returns and newlines. So, if I get the xml I just updated and diff it to the one I just sent they will register as completely different. The config xmls retrieved will have carriage returns if the last modification was was from a remote update or just newlines if the last edit was in the UI. I have looked at the requests I am sending and they do not have carriage returns. I looked at the code for CLI saves and it is creating a file from the parsed xml model rather than the supplied xml string. It removes a line break between the xml preamble and root <project> tag, saves each line with a carriage return and writes out self-closing tags. This messes up the Job Configuration History plugin as a one line change displays as if every line is changed. This also interferes with my local diff before sending the update. The two images attached (missing-eol and exact-same-config) are from back to back runs of the update script, meaning the configurations are exactly the same. The config history plugin thinks the entire file has been changed. Even with a manual correction for the carriage returns, my local diff shows differences because of the tag differences. The third (ui-save-no-change) is from a manual save in the UI (click "configure", click "save" with no changes). With no content change, we see that the <project> tag is again on its own line and If I'm not mistaken about the cause, is it feasible to have Jenkins save the configuration consistently regardless of how the configuration has changed? |
New:
When using the CLI or Rest API, Jenkins is saving the config.xml a different xml than I am providing. Specifically it is removing the first EOL, adding carriage returns, and replacing empty xml tags with self closing ones. I manage Jenkins instances for my organization. Each has a "test" instance that is a mirror of the prod one. Developers can experiment with things like changes to jobs and interactions with new plugins on the test version and we can port them to the prod one where configure permissions are disabled. For a first time copy the Job Import Plugin works but for edits to existing ones I have to either manually duplicate the changes or script something. I have tried this with both the CLI jar and groovy rest client. I get the config.xml from the test and prod servers. I perform a diff which lets me quickly see what the changes are and if approved will update the prod instance (job-update for CLI, POST for rest). If a job is edited in the Jenkins UI the config.xml is saved with newlines. When updated through one of the remote tools it is saved with carriage returns and newlines. So, if I get the xml I just updated and diff it to the one I just sent they will register as completely different. The config xmls retrieved will have carriage returns if the last modification was was from a remote update or just newlines if the last edit was in the UI. I have looked at the requests I am sending and they do not have carriage returns. I looked at the code for CLI saves and it is creating a file from the parsed xml model rather than the supplied xml string. It removes a line break between the xml preamble and root <project> tag, saves each line with a carriage return and writes out self-closing tags. [edit: last sentence is describing observed behaviour not something seen in the code] This messes up the Job Configuration History plugin as a one line change displays as if every line is changed. This also interferes with my local diff before sending the update. The two images attached (missing-eol and exact-same-config) are from back to back runs of the update script, meaning the configurations are exactly the same. The config history plugin thinks the entire file has been changed. Even with a manual correction for the carriage returns, my local diff shows differences because of the tag differences. The third (ui-save-no-change) is from a manual save in the UI (click "configure", click "save" with no changes). With no content change, we see that the <project> tag is again on its own line and If I'm not mistaken about the cause, is it feasible to have Jenkins save the configuration consistently regardless of how the configuration has changed? |
Paul G
made changes -
| Link | New: This issue is related to JENKINS-41178 [ JENKINS-41178 ] |
Paul G
added a comment - Thanks. I may be missing something then. It looks like Jenkins loses the formatting from CLI/REST data because the XML is parsed and written do disk based on the in-memory model rather than what was received through either API. It seems like Jenkins chooses different ways to write the xml files (or different in-memory representations persist differently) based on what's requesting the save (UI button click vs API update received) rather than the content of the save, rather than it being a case of the data received being different each time.
Paul G
added a comment - Thanks. I may be missing something then. It looks like Jenkins loses the formatting from CLI/REST data because the XML is parsed and written do disk based on the in-memory model rather than what was received through either API. It seems like Jenkins chooses different ways to write the xml files (or different in-memory representations persist differently) based on what's requesting the save (UI button click vs API update received) rather than the content of the save, rather than it being a case of the data received being different each time. We are having the same problem with our Jenkins instance. Locally we work around it by normalizing the XML we receive from the master and showing the diff of the normalized data (https://github.com/ros-infrastructure/ros_buildfarm/blob/16bc85c4e312fb1975d061c6e749fb4af9e3c4ff/ros_buildfarm/jenkins.py#L229-L249). But the Job Configuration History plugin shows many unnecessary changes which makes it more difficult to see the actual differences. Please see the attached picture which shows a huge diff with actually no changes at all.
If this is considered "not important enough" to be addressed by the devs it would be great if someone could shed some light into why the output is different and provide some pointers to the involved code blocks. Maybe we can come up with a patch for it.
| Attachment | New: jenkins-config-diff.png [ 36855 ] |
| Environment | Original: Jenkins 2.14 on Windows Server 2012, Job Configuration History Plugin 2.15, Groovy 2.4, java 8 |
New:
Jenkins 2.14 on Windows Server 2012, Job Configuration History Plugin 2.15, Groovy 2.4, java 8 Jenkins 2.46.1 on Ubuntu 16.04 |
It is intentional that the provided XML is parsed and the model resaved, so do not expect your formatting to survive intact. I do not know offhand why there would be any difference in formatting relative to doConfigSubmit-saved configurations, though. Possibly the Document object improperly retains some memory of the origin’s formatting when using CLI/REST uploads.
I don't mind if the server decides to "modify" the style of the xml before it is written to the config file. But I do mind that it deterministically uses two different approaches when editing the config using the web ui vs. using the remote api. That makes the job config history much less useful and clutters the diff.
IMHO it is a kind of RFE in the scope of the PluggableStorage-related patches.
Jenkins itself does not store info about XML file formatting. Each time these files are being serialized to the disk from the memory model, hence the formatting may change compared to versions coming from CLI/REST API