Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53399

Lost all saved managed files after upgrade to 3.0

      Upgraded Jenkins from 2.140 to 2.141 and in the same time config-file-provider-plugin from 2.18 to 3.0 

      After Jenkins restart, all configured files were gone.

      Jenkins shows warning about unreadable data:

      The first one is generic:

      org.jenkinsci.plugins.configfiles.GlobalConfigFiles Managed files ConversionException: org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 ---- Debugging information ---- message : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException cause-message : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 class : java.util.TreeSet required-type : java.util.TreeSet converter-type : com.thoughtworks.xstream.converters.collections.TreeSetConverter path : /org.jenkinsci.plugins.configfiles.GlobalConfigFiles/configs/comparator line number : 4 -------------------------------, MissingFieldException: No field 'org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles', MissingFieldException: No field 'org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles', MissingFieldException: No field 'org.jenkinsci.plugins.configfiles.maven.MavenToolchainsConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles', MissingFieldException: No field 'org.jenkinsci.plugins.configfiles.maven.MavenToolchainsConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles'

       

      The second one is specific to one Jenkins Job folder, where there should be no such configuration at all.

      com.cloudbees.hudson.plugins.folder.Folder releng+target ConversionException: org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1 : org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1 ---- Debugging information ---- message : org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1 cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException cause-message : org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1 class : java.util.TreeSet required-type : java.util.TreeSet converter-type : com.thoughtworks.xstream.converters.collections.TreeSetConverter path : /com.cloudbees.hudson.plugins.folder.Folder/properties/org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty/configs/comparator line number : 13 -------------------------------, ConversionException: Could not call org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty.readResolve() : null : Could not call org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty.readResolve() : null ---- Debugging information ---- message : Could not call org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty.readResolve() : null cause-exception : com.thoughtworks.xstream.converters.reflection.ObjectAccessException cause-message : Could not call org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty.readResolve() : null class : org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty required-type : java.util.TreeSet converter-type : hudson.util.RobustReflectionConverter path : /com.cloudbees.hudson.plugins.folder.Folder/properties/org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty/configs line number : 14 -------------------------------

       

      Now I don't know if this version is expected to break all stored provided files (the version number change could suggest it). If it is, then there should be a strong warning at the Jenkins plugin page stating that it is incompatible with the old version.

      Otherwise, please fix the problem  Thanks.

          [JENKINS-53399] Lost all saved managed files after upgrade to 3.0

          Oleg Nenashev added a comment -

          Oleg Nenashev added a comment - Yes, pom.xml refers 2.15 in compatibleSince: https://github.com/jenkinsci/config-file-provider-plugin/blob/master/pom.xml#L275 . CC domi srempfer

          Jens Beyer added a comment -

          Just for completeness' sake: After downgrading the plugin back to 2.18, the configured managed files are back.

          Jens Beyer added a comment - Just for completeness' sake: After downgrading the plugin back to 2.18, the configured managed files are back.

          I confirm Beyer's note

          Maurizio Nagni added a comment - I confirm Beyer's note

          Hi, I downgraded the plugin to 2.18, but the error remains. 

          Additionally Jenkins complains about not readable data now. 

          Patrick Wolfart added a comment - Hi, I downgraded the plugin to 2.18, but the error remains.  Additionally Jenkins complains about not readable data now. 

          If you used the "Manage Old Data" functionality and removed the unreadable configuration downgrading is useless. So I kept the latest version and I recreated the configuration using the interface and the data I found in a backup in the org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml file. Than I had to reselect by hand the new configuration name for each job!

          Moreover I found that NPM configuration's id field is read only. I don't know if it's related to this release, but I surely set it by hand the first time I created the NPM configuration.

          Giacomo Boccardo added a comment - If you used the "Manage Old Data" functionality and removed the unreadable configuration downgrading is useless. So I kept the latest version and I recreated the configuration using the interface and the data I found in a backup in the  org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml file. Than I had to reselect by hand the new configuration name for each job! Moreover I found that NPM configuration's id field is read only. I don't know if it's related to this release, but I surely set it by hand the first time I created the NPM configuration.

          Tomi Pakarinen added a comment - - edited

          I got old configuration to load by changing comparator class

          <comparator class="org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1"/>

          to

          <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>

           and per project config

          <comparator class="org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1"/>

           to

          <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>

           

          Tomi Pakarinen added a comment - - edited I got old configuration to load by changing comparator class <comparator class="org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1"/> to <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>  and per project config <comparator class="org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1"/>  to <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>  

          oeuftete added a comment -

          I can open a new issue, but assuming the same root cause is responsible for the Organization Folder plugin not being able to load its config either.  A downgrade of the Config File Provider plugin back to 2.18 was a successful workaround.

           

          2018-09-04T12:56:53.932168699Z SEVERE: Failed Loading item MyOrg
          2018-09-04T12:56:53.932184399Z java.lang.NullPointerException
          2018-09-04T12:56:53.932195699Z  at jenkins.branch.OrganizationFolder.onLoad(OrganizationFolder.java:199)
          2018-09-04T12:56:53.932207199Z  at hudson.model.Items.load(Items.java:372)
          2018-09-04T12:56:53.932218199Z  at jenkins.model.Jenkins$15.run(Jenkins.java:3125)
          2018-09-04T12:56:53.932229399Z  at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
          2018-09-04T12:56:53.932240499Z  at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
          2018-09-04T12:56:53.932251499Z  at jenkins.model.Jenkins$5.runTask(Jenkins.java:1068)
          2018-09-04T12:56:53.932262499Z  at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
          2018-09-04T12:56:53.932273499Z  at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
          2018-09-04T12:56:53.932284499Z  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          2018-09-04T12:56:53.932295599Z  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          2018-09-04T12:56:53.932306799Z  at java.lang.Thread.run(Thread.java:748)
          

          oeuftete added a comment - I can open a new issue, but assuming the same root cause is responsible for the Organization Folder plugin not being able to load its config either.  A downgrade of the Config File Provider plugin back to 2.18 was a successful workaround.   2018-09-04T12:56:53.932168699Z SEVERE: Failed Loading item MyOrg 2018-09-04T12:56:53.932184399Z java.lang.NullPointerException 2018-09-04T12:56:53.932195699Z at jenkins.branch.OrganizationFolder.onLoad(OrganizationFolder.java:199) 2018-09-04T12:56:53.932207199Z at hudson.model.Items.load(Items.java:372) 2018-09-04T12:56:53.932218199Z at jenkins.model.Jenkins$15.run(Jenkins.java:3125) 2018-09-04T12:56:53.932229399Z at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) 2018-09-04T12:56:53.932240499Z at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) 2018-09-04T12:56:53.932251499Z at jenkins.model.Jenkins$5.runTask(Jenkins.java:1068) 2018-09-04T12:56:53.932262499Z at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) 2018-09-04T12:56:53.932273499Z at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) 2018-09-04T12:56:53.932284499Z at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2018-09-04T12:56:53.932295599Z at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 2018-09-04T12:56:53.932306799Z at java.lang.Thread.run(Thread.java:748)

          Same for me. On Jenkins 2.121.3 after upgrading the config-file-provider-plugin from 2.18 to 3.0 folders' configurations are lost. Downgrade to 2.18 solves the problem.

          Dmitry Mikhirev added a comment - Same for me. On Jenkins 2.121.3 after upgrading the config-file-provider-plugin from 2.18 to 3.0 folders' configurations are lost. Downgrade to 2.18 solves the problem.

          Andrew Bayer added a comment -

          I've got at least one report of the upgrade breaking shared libraries and per-project permissions on folders as well - danielbeck, I think we should be seriously considering removing 3.0 from the update center.

          Andrew Bayer added a comment - I've got at least one report of the upgrade breaking shared libraries and per-project permissions on folders as well - danielbeck , I think we should be seriously considering removing 3.0 from the update center.

          I second beyerj's comment for a compatibility warning prior to upgrade.  How to upgrade correctly (patope's comment)?

          Scott Schreckengaust added a comment - I second beyerj 's comment for a compatibility warning prior to upgrade.  How to upgrade correctly ( patope 's comment)?

          Andrew Bayer added a comment -

          Opened https://github.com/jenkins-infra/update-center2/pull/236 to remove 3.0 from the update center.

          Andrew Bayer added a comment - Opened https://github.com/jenkins-infra/update-center2/pull/236 to remove 3.0 from the update center.

          Why comparator is serialized into config file? It is static constant anyway.. 

           

          https://github.com/jenkinsci/config-file-provider-plugin/commit/2366eaee6980564e59771897274f38d462bb0abc

          Can it be made transient?

          Tomi Pakarinen added a comment - Why comparator is serialized into config file? It is static constant anyway..    https://github.com/jenkinsci/config-file-provider-plugin/commit/2366eaee6980564e59771897274f38d462bb0abc Can it be made transient?

          org.jenkinsci.plugins.configfiles.GlobalConfigFiles
          
          Managed files
          
          ConversionException: org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 ---- Debugging information ---- message : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException cause-message : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 class : java.util.TreeSet required-type : java.util.TreeSet converter-type : com.thoughtworks.xstream.converters.collections.TreeSetConverter path : /org.jenkinsci.plugins.configfiles.GlobalConfigFiles/configs/comparator line number : 4 -------------------------------, MissingFieldException: No field 'org.jenkinsci.plugins.configfiles.xml.XmlConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles', MissingFieldException: No field 'org.jenkinsci.plugins.managedscripts.PowerShellConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles', MissingFieldException: No field 'org.jenkinsci.plugins.managedscripts.PowerShellConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles', MissingFieldException: No field 'org.jenkinsci.plugins.managedscripts.PowerShellConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles'
          

          We also had the same problem when updated from 2.18 to 3.0

          Hector Santana added a comment - org.jenkinsci.plugins.configfiles.GlobalConfigFiles Managed files ConversionException: org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 ---- Debugging information ---- message : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 cause-exception : com.thoughtworks.xstream.mapper.CannotResolveClassException cause-message : org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1 class : java.util.TreeSet required-type : java.util.TreeSet converter-type : com.thoughtworks.xstream.converters.collections.TreeSetConverter path : /org.jenkinsci.plugins.configfiles.GlobalConfigFiles/configs/comparator line number : 4 -------------------------------, MissingFieldException: No field 'org.jenkinsci.plugins.configfiles.xml.XmlConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles' , MissingFieldException: No field 'org.jenkinsci.plugins.managedscripts.PowerShellConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles' , MissingFieldException: No field 'org.jenkinsci.plugins.managedscripts.PowerShellConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles' , MissingFieldException: No field 'org.jenkinsci.plugins.managedscripts.PowerShellConfig' found in class 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles' We also had the same problem when updated from 2.18 to 3.0

          Dominik Bartholdi added a comment - - edited

          hmm, oh damn seems the easiest looking change of all broke the whole thing... and yes sure the idea never was to have the comparator beeing serialized into the config... 

          anyway, abayer and oleg_nenashev many thanks for the quick reaction, I will need to check how I can fix this without lifting the 'compatibleSince'

          sorry to everyone I caused issues with this!!!!

          Dominik Bartholdi added a comment - - edited hmm, oh damn seems the easiest looking change of all broke the whole thing... and yes sure the idea never was to have the comparator beeing serialized into the config...  anyway, abayer and oleg_nenashev many thanks for the quick reaction, I will need to check how I can fix this without lifting the 'compatibleSince' sorry to everyone I caused issues with this!!!!

          Andrew Bayer added a comment -

          FYI, 3.0 has been removed from the update center. I've had multiple reports that downgrading config-files-provider to 2.18 fixes all symptoms those reports encountered (including folder permissions and shared libraries being broken), so that's a clear path forward for now.

          Andrew Bayer added a comment - FYI, 3.0 has been removed from the update center. I've had multiple reports that downgrading config-files-provider to 2.18 fixes all symptoms those reports encountered (including folder permissions and shared libraries being broken), so that's a clear path forward for now.

          I've installed 3.0 and reconfigured everything.

          Please, consider that the next update should not break all the configurations which are still using 3.0.

          Giacomo Boccardo added a comment - I've installed 3.0 and reconfigured everything. Please, consider that the next update should not break all the configurations which are still using 3.0.

          domi: May this could help you https://wiki.jenkins.io/display/JENKINS/Serialization+of+anonymous+classes (see chapter Usages in XStream)

          Stefan Rempfer added a comment - domi : May this could help you https://wiki.jenkins.io/display/JENKINS/Serialization+of+anonymous+classes (see chapter Usages in XStream)

          Pavel Novak added a comment -

          Hi, all, just for being in touch, 
          so its the problem of config provider? 

          I guess the issue is not possible to revert, right? 

          We reverted from backups about 1TB of data (jenk. home, jobs and plugins folders) and then it was for sure back in the origin state, we did not found any other option :/

           

          Pavel Novak added a comment - Hi, all, just for being in touch,  so its the problem of config provider?  I guess the issue is not possible to revert, right?  We reverted from backups about 1TB of data (jenk. home, jobs and plugins folders) and then it was for sure back in the origin state, we did not found any other option :/  

          jhack srempfer not sure this will be possible - the problem is that the serialized field is not part of my code, but a member of TreeMap -> java.util.TreeMap#comparator

          oleg_nenashev do you have any idea how this could be solved? I currently only see the possibility to lift 'compatibleSince' or roll back and therefore screw the users who upgraded to 3.0 again...

          Dominik Bartholdi added a comment - jhack srempfer not sure this will be possible - the problem is that the serialized field is not part of my code, but a member of TreeMap -> java.util.TreeMap#comparator oleg_nenashev do you have any idea how this could be solved? I currently only see the possibility to lift 'compatibleSince' or roll back and therefore screw the users who upgraded to 3.0 again...

          Pavel Novak added a comment -

          domi 

          Hi, maybe override the default comparator, and implement custom one will be kind of workaround, how to fix the issue? 

          look eg. at following link:

          http://www.java2s.com/Tutorials/Java/Collection_How_to/Map/Create_custom_Comparator_for_TreeMap.htm 

           

          And that I understood it well, there is a bug in java implementation ? 
          Then, the issue should be reported into Java provider  

          Btw I can remember I was facing similar issue in the past, there was missing implementation of some method in OpenJDK, which was available in Oracle JDK, what I can remember it was also some kind of comparator, but on hashmap or something like these classes., or another abstract map.

           

          jhack, oleg_nenashev FYI

          Pavel Novak added a comment - domi   Hi, maybe override the default comparator, and implement custom one will be kind of workaround, how to fix the issue?  look eg. at following link: http://www.java2s.com/Tutorials/Java/Collection_How_to/Map/Create_custom_Comparator_for_TreeMap.htm     And that I understood it well, there is a bug in java implementation ?  Then, the issue should be reported into Java provider   Btw I can remember I was facing similar issue in the past, there was missing implementation of some method in OpenJDK, which was available in Oracle JDK, what I can remember it was also some kind of comparator, but on hashmap or something like these classes., or another abstract map.   jhack , oleg_nenashev FYI

          thanks pavenova its not a bug, but my code uses a java.util.TreeSet, which internally uses a java.util.TreeMap which in turn contains the comparator as a member field

          Dominik Bartholdi added a comment - thanks pavenova its not a bug, but my code uses a java.util.TreeSet, which internally uses a java.util.TreeMap which in turn contains the comparator as a member field

          Pavel Novak added a comment -

          domi

          ok, thanks for clarification, then simply ignore the comment  

          Godo luck, with that 

          Pavel Novak added a comment - domi ok, thanks for clarification, then simply ignore the comment   Godo luck, with that 

          I created a test to ensure that the descriptor data could be read. See https://github.com/srempfer/config-file-provider-plugin/commit/2ffe8695f734b48b695e4bd44c35de33e0c24919
          If someone else will contribute there anonymized 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' too, we will have a better test base to prevent breaking thinks in further versions.
          Don't forget information about the Jenkins and plugin version.

          Stefan Rempfer added a comment - I created a test to ensure that the descriptor data could be read. See https://github.com/srempfer/config-file-provider-plugin/commit/2ffe8695f734b48b695e4bd44c35de33e0c24919 If someone else will contribute there anonymized 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' too, we will have a better test base to prevent breaking thinks in further versions. Don't forget information about the Jenkins and plugin version.

          chad ruppert added a comment -

          Can you at least tell us WHERE these files are stored, so I can try to recover the scripts from a partial backup and enter them in the new system?

          We have a ton of builds on this machine, some don't use the scripted stuff. rolling back jenkins would be... awful.

          chad ruppert added a comment - Can you at least tell us WHERE these files are stored, so I can try to recover the scripts from a partial backup and enter them in the new system? We have a ton of builds on this machine, some don't use the scripted stuff. rolling back jenkins would be... awful.

          Andrew Bayer added a comment -

          poundkeys - if you didn't wipe the "old data" from "Manage Jenkins", you should be able to just roll back to version 2.18.

          Andrew Bayer added a comment - poundkeys - if you didn't wipe the "old data" from "Manage Jenkins", you should be able to just roll back to version 2.18.

          chad ruppert added a comment - - edited

          Andrew, Rolling back sucks too. I found the old configs in 

          Jenkins/config-history/org.jenkinsci.plugins.configfiles.GlobalConfigFiles/<dates>

          Thankfully the entry system allows us to specify the ID, so that's wonderful. I was able to strip them out of the xml and make it work, i think. I'll be finding out shortly.

           

          Edit: Yup, worked a treat! 

          chad ruppert added a comment - - edited Andrew, Rolling back sucks too. I found the old configs in  Jenkins/config-history/org.jenkinsci.plugins.configfiles.GlobalConfigFiles/<dates> Thankfully the entry system allows us to specify the ID, so that's wonderful. I was able to strip them out of the xml and make it work, i think. I'll be finding out shortly.   Edit: Yup, worked a treat! 

          Jens Beyer added a comment -

          poundkeys Sorry, but why does rolling back suck? It is easy, quick and foolproof (thanks @Jenkins team).

          This is basically the first thing I usually try before even thinking about looking into our backups. Upgrade, something looks fishy, do not overwrite any configuration, especially do not delete "unreadable data" from Jenkins. Try to keep the executors offline, try to isolate the culprit by rolling back upgraded versions one after another or by divide and conquer - whatever fits your style and the situation best.

          Of course, having a backup keeps blood pressure down while being at it.

          And having a test machine to test upgrades before going into production would be the best, but I know from own experience that this is a wish not easily fulfilled, especially if Ops or bosses say "you have a backup if something goes wrong".

          @all who come here now because of this issue (you shouldn't anymore because the 3.0 version has been pulled back) and see this big list of comments and feel lost - here is your way out: do not delete "unreadable data" and simply revert the config-file-provider-plugin back from version 3.0 to 2.18, then restart Jenkins, and everything is back to normal.

          Jens Beyer added a comment - poundkeys Sorry, but why does rolling back suck? It is easy, quick and foolproof (thanks @Jenkins team). This is basically the first thing I usually try before even thinking about looking into our backups. Upgrade, something looks fishy, do not overwrite any configuration, especially do not delete "unreadable data" from Jenkins. Try to keep the executors offline, try to isolate the culprit by rolling back upgraded versions one after another or by divide and conquer - whatever fits your style and the situation best. Of course, having a backup keeps blood pressure down while being at it. And having a test machine to test upgrades before going into production would be the best, but I know from own experience that this is a wish not easily fulfilled, especially if Ops or bosses say "you have a backup if something goes wrong". @all who come here now because of this issue (you shouldn't anymore because the 3.0 version has been pulled back) and see this big list of comments and feel lost - here is your way out: do not delete "unreadable data" and simply revert the config-file-provider-plugin back from version 3.0 to 2.18, then restart Jenkins, and everything is back to normal.

          chad ruppert added a comment -

          That's the thing. I didn't see Jenkins warn me about the unreadable data. Either I had to restart the service manually after a not great upgrade, or I just flat out missed it.

          A lot of it is probably bad practice by upgrading a ton of plugins at the same time, as well as a jenkins upgrade right before.

          This is, I think, the first time I've had a real issue with an upgrade. Backups help a lot, and that's where I was able to pull the config from. After looking through my backups I found the same folder on the build server.  It's been a week- we were caught up in that South Central Azure outage and a few other things, so the BP was already high.

          In the future I may consider a rollback of Jenkins. I always get worried about that failing, simply because its another feature I haven't tried. Not a fulltime Ops engineer by any stretch of the word. Sorry for the initial panic.

          chad ruppert added a comment - That's the thing. I didn't see Jenkins warn me about the unreadable data. Either I had to restart the service manually after a not great upgrade, or I just flat out missed it. A lot of it is probably bad practice by upgrading a ton of plugins at the same time, as well as a jenkins upgrade right before. This is, I think, the first time I've had a real issue with an upgrade. Backups help a lot, and that's where I was able to pull the config from. After looking through my backups I found the same folder on the build server.  It's been a week- we were caught up in that South Central Azure outage and a few other things, so the BP was already high. In the future I may consider a rollback of Jenkins. I always get worried about that failing, simply because its another feature I haven't tried. Not a fulltime Ops engineer by any stretch of the word. Sorry for the initial panic.

          given the mess I created... I rolled back the culprit change which caused this issue and will release a version 3.1 - 3.1 will then be compatible with 2.18 again. Sorry to everyone who has 3.0 installed and needs to change there configs back to what it was in 2.18.

          basically this means in the file 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' (only one file) you have to change

          <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/> back to: <comparator class="org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1"/>

          and in the folder config.xml's (potentially multiple files) you have to change 

          <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/> back to: <comparator class="org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1"/>

          Dominik Bartholdi added a comment - given the mess I created... I rolled back the culprit change which caused this issue and will release a version 3.1 - 3.1 will then be compatible with 2.18 again. Sorry to everyone who has 3.0 installed and needs to change there configs back to what it was in 2.18. basically this means in the file 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' (only one file) you have to change <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/> back to: <comparator class="org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1"/> and in the folder config.xml's (potentially multiple files) you have to change  <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>  back to: <comparator class="org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1"/>

          just released 3.1 - please let me know if you find its not working out for you

           

          ...and sorry again!!!!

          Dominik Bartholdi added a comment - just released 3.1 - please let me know if you find its not working out for you   ...and sorry again!!!!

          Updated from 2.18 to 3.1 without problems...

          Stefan Rempfer added a comment - Updated from 2.18 to 3.1 without problems...

          Is it safe to update from 3.0 to 3.1? What's the exact procedure to avoid another disaster?

          Update and change 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' or viceversa?

          Giacomo Boccardo added a comment - Is it safe to update from 3.0 to 3.1? What's the exact procedure to avoid another disaster? Update and change 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' or viceversa?

          Dominik Bartholdi added a comment - jhack please see my comment above...  https://issues.jenkins-ci.org/browse/JENKINS-53399?focusedCommentId=348653&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-348653

          domi I read it, so I asked those questions.

          Giacomo Boccardo added a comment - domi I read it, so I asked those questions.

          if you have 3.0 installed (and it is working), then do this:

          upgrade to 3.1 and in the file 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' (only one file) you have to change

          <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/> back to: <comparator class="org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1"/>

          and in the folder config.xml's (potentially multiple files) you have to change 

          <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/> back to: <comparator class="org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1"/>

          Dominik Bartholdi added a comment - if you have 3.0 installed (and it is working), then do this: upgrade to 3.1 and in the file 'org.jenkinsci.plugins.configfiles.GlobalConfigFiles.xml' (only one file) you have to change <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>  back to: <comparator  class="org.jenkinsci.plugins.configfiles.GlobalConfigFiles$1"/> and in the folder config.xml's (potentially multiple files) you have to change  <comparator class="org.jenkinsci.plugins.configfiles.ConfigComparator"/>  back to: <comparator  class="org.jenkinsci.plugins.configfiles.folder.FolderConfigFileProperty$1"/>

          Oleg Nenashev added a comment -

          It would be possible to retain the class introduced in 3.0 and then do some migration to it via readResolve() on the Tree level (replace the Tree instance when pre-3.0 configuration is being read). It would keep pre-3.0 instances compatible and allow migrating to the new data structure as it was intended.

           

           

          Oleg Nenashev added a comment - It would be possible to retain the class introduced in 3.0 and then do some migration to it via readResolve() on the Tree level (replace the Tree instance when pre-3.0 configuration is being read). It would keep pre-3.0 instances compatible and allow migrating to the new data structure as it was intended.    

          thanks Oleg, I can take look at this in the future, but for now this is released again with the 2.18 implementation

          Dominik Bartholdi added a comment - thanks Oleg, I can take look at this in the future, but for now this is released again with the 2.18 implementation

          3.1 fix the issue for me, thanks!

          Cyprien Devillez added a comment - 3.1 fix the issue for me, thanks!

          Jens Beyer added a comment -

          Upgraded from 2.18 to 3.1, had no issues so far. No unreadable data reported. Builds running. Thanks for your quick work.

          What's the workflow now, do I have to set the status of the issue to something or are you devs doing it?

          Jens Beyer added a comment - Upgraded from 2.18 to 3.1, had no issues so far. No unreadable data reported. Builds running. Thanks for your quick work. What's the workflow now, do I have to set the status of the issue to something or are you devs doing it?

          Dominik Bartholdi added a comment - - edited

          thanks beyerj - i'll resolve it... fixed with 3.1

          Dominik Bartholdi added a comment - - edited thanks beyerj - i'll resolve it... fixed with 3.1

          René Scheibe added a comment -

          With pull-request #69 the anonymous class serialization warnings are now fixed while maintaining XStream backward compatibility.

          René Scheibe added a comment - With pull-request #69 the anonymous class serialization warnings are now fixed while maintaining XStream backward compatibility.

            domi Dominik Bartholdi
            beyerj Jens Beyer
            Votes:
            8 Vote for this issue
            Watchers:
            23 Start watching this issue

              Created:
              Updated:
              Resolved: