Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-62661

Views with status filters lost upgrading from 2.238 to 2.240

    XMLWordPrintable

    Details

    • Similar Issues:
    • Released As:
      2.258

      Description

      Upgraded using apt (running on ubuntu bionic).

      I was able to recover the config file from a support plugin archive, and ungarble it and try again and these items are consistently being eaten.

        Attachments

          Issue Links

            Activity

            Hide
            jsoref Josh Soref added a comment -

            I have been able to manually re-create these objects from the gui...

            Show
            jsoref Josh Soref added a comment - I have been able to manually re-create these objects from the gui...
            Hide
            danielbeck Daniel Beck added a comment -

            Were any warnings logged, or did the "old data monitor" trigger and complain about unloadable data? If so, what specifically?

            Show
            danielbeck Daniel Beck added a comment - Were any warnings logged, or did the "old data monitor" trigger and complain about unloadable data? If so, what specifically?
            Hide
            jsoref Josh Soref added a comment -

            I didn't see/couldn't find any.

             

            And I'm used to seeing them.

            Show
            jsoref Josh Soref added a comment - I didn't see/couldn't find any.   And I'm used to seeing them.
            Hide
            tgr Tobias Gruetzmacher added a comment -

            I seem to have a similar issue, all my ListViews where statusFilter is in use are "missing" and unreadable data:

            ConversionException: Could not call hudson.model.ListView.readResolve() : null : Could not call hudson.model.ListView.readResolve() : null
            ---- Debugging information ----
            message             : Could not call hudson.model.ListView.readResolve() : null
            cause-exception     : com.thoughtworks.xstream.converters.reflection.ObjectAccessException
            cause-message       : Could not call hudson.model.ListView.readResolve() : null
            class               : hudson.model.ListView
            required-type       : hudson.model.ListView
            converter-type      : hudson.util.RobustReflectionConverter
            path                : /com.cloudbees.hudson.plugins.folder.Folder/folderViews/views/hudson.model.ListView
            line number         : 56
            -------------------------------
            

            It seems I can force the error with the JobDSL plugin using a view which tries to set a statusFilter:

            Caused by: java.io.IOException: Unable to read
            	at hudson.model.View.createViewFromXML(View.java:1400)
            	at javaposse.jobdsl.plugin.JenkinsJobManagement.createOrUpdateView(JenkinsJobManagement.java:183)
            	... 11 more
            Caused by: com.thoughtworks.xstream.converters.ConversionException: Could not call hudson.model.ListView.readResolve() : null : Could not call hudson.model.ListView.readResolve() : null
            ---- Debugging information ----
            message             : Could not call hudson.model.ListView.readResolve() : null
            cause-exception     : com.thoughtworks.xstream.converters.reflection.ObjectAccessException
            cause-message       : Could not call hudson.model.ListView.readResolve() : null
            class               : hudson.model.ListView
            required-type       : hudson.model.ListView
            converter-type      : hudson.util.RobustReflectionConverter
            path                : /hudson.model.ListView
            line number         : 43
            version             : not available
            -------------------------------
            	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79)
            	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65)
            	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
            	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
            	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
            	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
            	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189)
            	at hudson.util.XStream2.unmarshal(XStream2.java:161)
            	at hudson.util.XStream2.unmarshal(XStream2.java:132)
            	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173)
            	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053)
            	at hudson.model.View.createViewFromXML(View.java:1395)
            	... 12 more
            Caused by: com.thoughtworks.xstream.converters.reflection.ObjectAccessException: Could not call hudson.model.ListView.readResolve() : null
            	at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callReadResolve(SerializationMethodInvoker.java:72)
            	at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:270)
            	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
            	... 23 more
            Caused by: java.lang.UnsupportedOperationException
            	at java.util.AbstractList.add(AbstractList.java:148)
            	at hudson.model.ListView.readResolve(ListView.java:153)
            	at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callReadResolve(SerializationMethodInvoker.java:66)
            	... 25 more
            

            (This is with Jenkins 2.249.1)

            Show
            tgr Tobias Gruetzmacher added a comment - I seem to have a similar issue, all my ListViews where statusFilter is in use are "missing" and unreadable data: ConversionException: Could not call hudson.model.ListView.readResolve() : null : Could not call hudson.model.ListView.readResolve() : null ---- Debugging information ---- message : Could not call hudson.model.ListView.readResolve() : null cause-exception : com.thoughtworks.xstream.converters.reflection.ObjectAccessException cause-message : Could not call hudson.model.ListView.readResolve() : null class : hudson.model.ListView required-type : hudson.model.ListView converter-type : hudson.util.RobustReflectionConverter path : /com.cloudbees.hudson.plugins.folder.Folder/folderViews/views/hudson.model.ListView line number : 56 ------------------------------- It seems I can force the error with the JobDSL plugin using a view which tries to set a statusFilter: Caused by: java.io.IOException: Unable to read at hudson.model.View.createViewFromXML(View.java:1400) at javaposse.jobdsl.plugin.JenkinsJobManagement.createOrUpdateView(JenkinsJobManagement.java:183) ... 11 more Caused by: com.thoughtworks.xstream.converters.ConversionException: Could not call hudson.model.ListView.readResolve() : null : Could not call hudson.model.ListView.readResolve() : null ---- Debugging information ---- message : Could not call hudson.model.ListView.readResolve() : null cause-exception : com.thoughtworks.xstream.converters.reflection.ObjectAccessException cause-message : Could not call hudson.model.ListView.readResolve() : null class : hudson.model.ListView required-type : hudson.model.ListView converter-type : hudson.util.RobustReflectionConverter path : /hudson.model.ListView line number : 43 version : not available ------------------------------- at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:79) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:65) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50) at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134) at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1189) at hudson.util.XStream2.unmarshal(XStream2.java:161) at hudson.util.XStream2.unmarshal(XStream2.java:132) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1173) at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1053) at hudson.model.View.createViewFromXML(View.java:1395) ... 12 more Caused by: com.thoughtworks.xstream.converters.reflection.ObjectAccessException: Could not call hudson.model.ListView.readResolve() : null at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callReadResolve(SerializationMethodInvoker.java:72) at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:270) at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72) ... 23 more Caused by: java.lang.UnsupportedOperationException at java.util.AbstractList.add(AbstractList.java:148) at hudson.model.ListView.readResolve(ListView.java:153) at com.thoughtworks.xstream.converters.reflection.SerializationMethodInvoker.callReadResolve(SerializationMethodInvoker.java:66) ... 25 more (This is with Jenkins 2.249.1)
            Hide
            danielbeck Daniel Beck added a comment -

            Tobias Gruetzmacher That looks promising. Do you have the XML that the plugin attempts to submit there?

            Show
            danielbeck Daniel Beck added a comment - Tobias Gruetzmacher That looks promising. Do you have the XML that the plugin attempts to submit there?
            Hide
            tgr Tobias Gruetzmacher added a comment -

            The minimal JobDSL script provoking this error is:

            import javaposse.jobdsl.dsl.views.ListView
            
            listView('test') {
              statusFilter ListView.StatusFilter.ENABLED
            }
            

            which produces this XML:

            <hudson.model.ListView>
                <filterExecutors>false</filterExecutors>
                <filterQueue>false</filterQueue>
                <properties class='hudson.model.View$PropertyList'></properties>
                <jobNames class='tree-set'>
                    <comparator class='hudson.util.CaseInsensitiveComparator'></comparator>
                </jobNames>
                <jobFilters></jobFilters>
                <columns></columns>
                <statusFilter>true</statusFilter>
            </hudson.model.ListView>
            

            Ah: jobFilters is a DescribableList, which doesn't implement public void add(int index, E element) ...

            Show
            tgr Tobias Gruetzmacher added a comment - The minimal JobDSL script provoking this error is: import javaposse.jobdsl.dsl.views.ListView listView( 'test' ) { statusFilter ListView.StatusFilter.ENABLED } which produces this XML: <hudson.model.ListView> <filterExecutors> false </filterExecutors> <filterQueue> false </filterQueue> <properties class= 'hudson.model.View$PropertyList' > </properties> <jobNames class= 'tree-set' > <comparator class= 'hudson.util.CaseInsensitiveComparator' > </comparator> </jobNames> <jobFilters> </jobFilters> <columns> </columns> <statusFilter> true </statusFilter> </hudson.model.ListView> Ah: jobFilters is a DescribableList, which doesn't implement public void add(int index, E element) ...
            Hide
            timja Tim Jacomb added a comment -
            Show
            timja Tim Jacomb added a comment - There's some notes in here too: https://github.com/jenkins-infra/jenkins.io/pull/3697/files
            Hide
            egutierrez Evaristo Gutierrez added a comment -

            Tim Jacomb I'm not sure I get your comment. Do you mean maybe the users facing this issue did not take the upgrade guidelines into account?

            Tobias Gruetzmacher Daniel Beck just to confirm: job-dsl is not the culprit but a way of triggering the issue, right?

            Show
            egutierrez Evaristo Gutierrez added a comment - Tim Jacomb I'm not sure I get your comment. Do you mean maybe the users facing this issue did not take the upgrade guidelines into account? Tobias Gruetzmacher Daniel Beck just to confirm: job-dsl is not the culprit but a way of triggering the issue, right?
            Hide
            timja Tim Jacomb added a comment -

            Evaristo Gutierrez I meant that Mark Waite has been documenting the config change that users need to make at least for jcasc in relation to this change.

            Show
            timja Tim Jacomb added a comment - Evaristo Gutierrez I meant that Mark Waite has been documenting the config change that users need to make at least for jcasc in relation to this change.
            Hide
            markewaite Mark Waite added a comment -

            https://github.com/jenkins-infra/jenkins.io/pull/3697 is the pull request that adds upgrade guide instructions for the move of statusFilter from an attribute of a view to a separately defined component of a view. It doesn't describe the case that Josh Soref reports but it might be considered to be the start of describing the case that Tobias Gruetzmacher has reported. The description addresses configuration as code and how to make the transition. I assume a similar type of transition will be needed in JobDSL. If someone is willing to submit the JobDSL upgrade instructions, they could also be included in the upgrade guide.

            Show
            markewaite Mark Waite added a comment - https://github.com/jenkins-infra/jenkins.io/pull/3697 is the pull request that adds upgrade guide instructions for the move of statusFilter from an attribute of a view to a separately defined component of a view. It doesn't describe the case that Josh Soref reports but it might be considered to be the start of describing the case that Tobias Gruetzmacher has reported. The description addresses configuration as code and how to make the transition. I assume a similar type of transition will be needed in JobDSL. If someone is willing to submit the JobDSL upgrade instructions, they could also be included in the upgrade guide.
            Hide
            danielbeck Daniel Beck added a comment - - edited

            This has nothing to do with Job DSL and nobody claimed it did.

            It's just that nobody bothered to do an update test here, and there's no autotest for it either.

            Steps to reproduce:

            1. Install 2.238
            2. Skip plugins
            3. create a job
            4. Go back to dashboard
            5. click "+" in the tab bar
            6. select List View
            7. Select "Enabled job only"
            8. Save
            9. Stop Jenkins
            10. Start 2.240

            The list view you just created will be gone.

            Show
            danielbeck Daniel Beck added a comment - - edited This has nothing to do with Job DSL and nobody claimed it did. It's just that nobody bothered to do an update test here, and there's no autotest for it either. Steps to reproduce: Install 2.238 Skip plugins create a job Go back to dashboard click "+" in the tab bar select List View Select "Enabled job only" Save Stop Jenkins Start 2.240 The list view you just created will be gone.
            Hide
            jsoref Josh Soref added a comment -

            That fits our problem. And it matches the earlier note about "status filter".

            Show
            jsoref Josh Soref added a comment - That fits our problem. And it matches the earlier note about "status filter".
            Hide
            jsoref Josh Soref added a comment -

            The attachment clearly has:

            <statusFilter>true</statusFilter>

            Show
            jsoref Josh Soref added a comment - The attachment clearly has: <statusFilter>true</statusFilter>
            Hide
            initialzero Marc Benstein added a comment -

            Same, all views using <statusFilter>true</statusFilter> are yanked from the config on Stable upgrade.

            Show
            initialzero Marc Benstein added a comment - Same, all views using <statusFilter>true</statusFilter> are yanked from the config on Stable upgrade.
            Hide
            markewaite Mark Waite added a comment -

            We just recently added an entry to the Jenkins 2.249.1 upgrade guide to describe the statusFilter transition in Jenkins 2.249.1 LTS. I'm certainly open to more descriptions or better descriptions of techniques for the upgrade.

            Show
            markewaite Mark Waite added a comment - We just recently added an entry to the Jenkins 2.249.1 upgrade guide to describe the statusFilter transition in Jenkins 2.249.1 LTS. I'm certainly open to more descriptions or better descriptions of techniques for the upgrade.
            Hide
            jsoref Josh Soref added a comment -

            Mark Waite: at the very least you have to mention the non declarative case. Crashing on startup isn't a big deal, you search for the error and fix things. Dataloss after upgrade is much worse.

            Personally, I'd suggest adding code on the LTS branch to perform migration at launch.

            Show
            jsoref Josh Soref added a comment - Mark Waite : at the very least you have to mention the non declarative case. Crashing on startup isn't a big deal, you search for the error and fix things. Dataloss after upgrade is much worse. Personally, I'd suggest adding code on the LTS branch to perform migration at launch.
            Hide
            markewaite Mark Waite added a comment -

            Thanks Josh Soref. I hadn't performed the test myself to check the problem as described by Daniel. I wrote the upgrade guide based on my experience as a configuration as code user. My examples were unharmed by removing the statusFilter from the view definition (or at least I didn't detect the harm). I'll need to experiment more to create a better description of the upgrade impact. Daniel's description seems like the right technique - define a 2.235.5 view (JCasC or otherwise) that only uses a status filter and no other setting, then confirm that view is deleted on upgrade to 2.249.1.

            Show
            markewaite Mark Waite added a comment - Thanks Josh Soref . I hadn't performed the test myself to check the problem as described by Daniel. I wrote the upgrade guide based on my experience as a configuration as code user. My examples were unharmed by removing the statusFilter from the view definition (or at least I didn't detect the harm). I'll need to experiment more to create a better description of the upgrade impact. Daniel's description seems like the right technique - define a 2.235.5 view (JCasC or otherwise) that only uses a status filter and no other setting, then confirm that view is deleted on upgrade to 2.249.1.
            Hide
            danielbeck Daniel Beck added a comment -

            This is a bug in the (generally legitimate) data migration. It could be noted as a known issue, but primarily we need a fix towards 2.249.2.

            Show
            danielbeck Daniel Beck added a comment - This is a bug in the (generally legitimate) data migration. It could be noted as a known issue, but primarily we need a fix towards 2.249.2.
            Hide
            pwiseman Peter Wiseman added a comment -

            When upgrading, I typically upgrade to the latest weekly after reviewing known issues.  I have multiple jenkins instances that typically only get upgraded once a year.  Upgrade notes aren't sufficient for my case, which I presume mirrors many other use cases.  Agree with Daniel Beck that a change introducing data loss is a bug.

            Show
            pwiseman Peter Wiseman added a comment - When upgrading, I typically upgrade to the latest weekly after reviewing known issues.  I have multiple jenkins instances that typically only get upgraded once a year.  Upgrade notes aren't sufficient for my case, which I presume mirrors many other use cases.  Agree with Daniel Beck that a change introducing data loss is a bug.
            Hide
            markewaite Mark Waite added a comment -

            Thanks Peter Wiseman. I agree that the upgrade guide is not sufficient. We need a fix, preferably in Jenkins 2.249.2.

            Show
            markewaite Mark Waite added a comment - Thanks Peter Wiseman . I agree that the upgrade guide is not sufficient. We need a fix, preferably in Jenkins 2.249.2.
            Hide
            pwiseman Peter Wiseman added a comment -

            The status has changed to Resolution and tagged as fixed in 2.258.  I've re-tested an upgrade from 2.238 to 2.258 and the problem is still present (the views go missing).  What was the RESOLUTION?

            Show
            pwiseman Peter Wiseman added a comment - The status has changed to Resolution and tagged as fixed in 2.258.  I've re-tested an upgrade from 2.238 to 2.258 and the problem is still present (the views go missing).  What was the RESOLUTION?
            Hide
            markewaite Mark Waite added a comment -

            That sounds like the fix provided in PR-4466 did not resolve the issue you are seeing. The fix was a code change in that pull request that was intended to handle the upgrade.

            Show
            markewaite Mark Waite added a comment - That sounds like the fix provided in PR-4466 did not resolve the issue you are seeing. The fix was a code change in that pull request that was intended to handle the upgrade.
            Hide
            markewaite Mark Waite added a comment - - edited

            Peter Wiseman I suspect more details will be required to duplicate the problem you're seeing. I was unable to duplicate the problem with 2.258, while I was able to duplicate the problem with 2.257. Steps I took:

            Jenkins 2.238

            1. Install Jenkins 2.238 with no plugins
            2. Define a job 'enabled-job' that is an enabled Freestyle job and run it several times
            3. Define a job 'disabled-job' this is a disabled Freestyle job
            4. Define a view 'Enabled' that uses the job status filter to show only enabled jobs, recursing in subfolders, matching the regular expression '.*'
            5. Define a view 'Disabled' that uses the job status filter to show only disabled jobs, recursing in subfolders, matching the regular expression '.*'
            6. Confirm that the view 'Enabled' shows only the enabled job and the view 'Disabled' shows only the disabled job
            7. Stop Jenkins 2.238
            8. Create a backup of the Jenkins home directory for reuse

            Jenkins 2.258

            1. Restore the Jenkins home directory backup from Jenkins 2.238
            2. Start Jenkins 2.258
            3. Confirm that the 'Disabled' and 'Enabled' views are visible in Jenkins 2.258

            Jenkins 2.257

            1. Restore the Jenkins home directory backup from Jenkins 2.238
            2. Start Jenkins 2.257
            3. Confirm that the 'Disabled' and 'Enabled' views are lost in Jenkins 2.257

            If someone needs a copy of the Jenkins home directory backup, I placed it temporarily at https://home.markwaite.net/~mwaite/jenkins-2.238.zip. The login user name is butler with the password butler.

            Show
            markewaite Mark Waite added a comment - - edited Peter Wiseman I suspect more details will be required to duplicate the problem you're seeing. I was unable to duplicate the problem with 2.258, while I was able to duplicate the problem with 2.257. Steps I took: Jenkins 2.238 Install Jenkins 2.238 with no plugins Define a job 'enabled-job' that is an enabled Freestyle job and run it several times Define a job 'disabled-job' this is a disabled Freestyle job Define a view 'Enabled' that uses the job status filter to show only enabled jobs, recursing in subfolders, matching the regular expression '.*' Define a view 'Disabled' that uses the job status filter to show only disabled jobs, recursing in subfolders, matching the regular expression '.*' Confirm that the view 'Enabled' shows only the enabled job and the view 'Disabled' shows only the disabled job Stop Jenkins 2.238 Create a backup of the Jenkins home directory for reuse Jenkins 2.258 Restore the Jenkins home directory backup from Jenkins 2.238 Start Jenkins 2.258 Confirm that the 'Disabled' and 'Enabled' views are visible in Jenkins 2.258 Jenkins 2.257 Restore the Jenkins home directory backup from Jenkins 2.238 Start Jenkins 2.257 Confirm that the 'Disabled' and 'Enabled' views are lost in Jenkins 2.257 If someone needs a copy of the Jenkins home directory backup, I placed it temporarily at https://home.markwaite.net/~mwaite/jenkins-2.238.zip . The login user name is butler with the password butler .
            Hide
            pwiseman Peter Wiseman added a comment -

            Mark Waite Thanks for detailing the test (that helped), and sorry to have required you to re-test.  I found my problem.  I tested too quickly and failed to notice that the views do not appear on the main page unless there is at least one job.  I was looking at the Welcome screen, and thinking there were no views.  I failed to spot this pre-upgrade.  So all is good.  Many thanks for everyone's work on this.

            Show
            pwiseman Peter Wiseman added a comment - Mark Waite Thanks for detailing the test (that helped), and sorry to have required you to re-test.  I found my problem.  I tested too quickly and failed to notice that the views do not appear on the main page unless there is at least one job.  I was looking at the Welcome screen, and thinking there were no views.  I failed to spot this pre-upgrade.  So all is good.  Many thanks for everyone's work on this.

              People

              Assignee:
              Unassigned Unassigned
              Reporter:
              jsoref Josh Soref
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: