Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-64295

An attempt to save the global configuration was made before it was loaded

      I just upgraded Jenkins to Jenkins 2.267 and I am getting the following errors:

      java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded
      	at jenkins.model.Jenkins.save(Jenkins.java:3379)
      	at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3398)
      	at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2637)
      	at jenkins.model.Jenkins$16.run(Jenkins.java:3342)
      	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
      	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
      	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1129)
      	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
      	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused: org.jvnet.hudson.reactor.ReactorException
      	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282)
      	at jenkins.InitReactorRunner.run(InitReactorRunner.java:50)
      	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1162)
      	at jenkins.model.Jenkins.<init>(Jenkins.java:962)
      	at hudson.model.Hudson.<init>(Hudson.java:85)
      	at hudson.model.Hudson.<init>(Hudson.java:81)
      	at hudson.WebAppMain$3.run(WebAppMain.java:295)
      Caused: hudson.util.HudsonFailedToLoad
      	at hudson.WebAppMain$3.run(WebAppMain.java:312)
      

      I noticed in the upgrade guide for 2.204, they said that the ConfigurationAsCode plugin may be cousing this but I don't see that in my plugins folder. These are all my plugins:

      ace-editor
      all-changes
      analysis-core
      analysis-model-api
      ansicolor
      ant
      antisamy-markup-formatter
      apache-httpcomponents-client-4-api
      authentication-tokens
      authorize-project
      aws-java-sdk
      backup
      basic-branch-build-strategies
      bitbucket
      blueocean
      blueocean-autofavorite
      blueocean-bitbucket-pipeline
      blueocean-commons
      blueocean-config
      blueocean-core-js
      blueocean-dashboard
      blueocean-display-url
      blueocean-events
      blueocean-github-pipeline
      blueocean-git-pipeline
      blueocean-i18n
      blueocean-jira
      blueocean-jwt
      blueocean-personalization
      blueocean-pipeline-api-impl
      blueocean-pipeline-editor
      blueocean-pipeline-scm-api
      blueocean-rest
      blueocean-rest-impl
      blueocean-web
      bootstrap4-api
      bouncycastle-api
      branch-api
      build-name-setter
      build-pipeline-plugin
      build-timeout
      build-token-root
      built-on-column
      checks-api
      checkstyle
      cloudbees-bitbucket-branch-source
      cloudbees-folder
      cloud-stats
      cloverphp
      cobertura
      code-coverage-api
      command-launcher
      conditional-buildstep
      config-file-provider
      consul
      consul-kv-builder
      convert-to-pipeline
      copyartifact
      crap4j
      credentials
      credentials-binding
      dashboard-view
      data-tables-api
      digitalocean-plugin
      display-url-api
      docker-commons
      docker-workflow
      drupal-developer
      dry
      dtkit-api
      durable-task
      echarts-api
      email-ext
      emailext-template
      embeddable-build-status
      envinject
      envinject-api
      external-monitor-job
      favorite
      font-awesome-api
      forensics-api
      generic-webhook-trigger
      git
      git-client
      github
      github-api
      github-branch-source
      github-organization-folder
      git-parameter
      git-server
      gradle
      handlebars
      handy-uri-templates-2-api
      htmlpublisher
      icon-shim
      jackson2-api
      javadoc
      jdepend
      jdk-tool
      jenkins-design-language
      jenkins-jira-issue-updater
      jenkins-multijob-plugin
      jira
      jira-steps
      JiraTestResultReporter
      jira-trigger
      jquery
      jquery3-api
      jquery-detached
      jquery-ui
      jsch
      junit
      ldap
      lockable-resources
      mailer
      mapdb-api
      matrix-auth
      matrix-project
      maven-plugin
      mercurial
      metrics
      momentjs
      monitoring
      naginator
      nodejs
      nodelabelparameter
      okhttp-api
      pam-auth
      parameterized-trigger
      pipeline-build-step
      pipeline-github-lib
      pipeline-graph-analysis
      pipeline-input-step
      pipeline-milestone-step
      pipeline-model-api
      pipeline-model-declarative-agent
      pipeline-model-definition
      pipeline-model-extensions
      pipeline-rest-api
      pipeline-stage-step
      pipeline-stage-tags-metadata
      pipeline-stage-view
      plain-credentials
      plot
      plugin-util-api
      pmd
      popper-api
      publish-over
      publish-over-ssh
      pubsub-light
      rebuild
      resource-disposer
      role-strategy
      run-condition
      s3
      scm-api
      script-security
      simple-theme-plugin
      sitemonitor
      slack
      snakeyaml-api
      sse-gateway
      ssh
      ssh-agent
      ssh-credentials
      ssh-slaves
      structs
      subversion
      throttle-concurrents
      timestamper
      token-macro
      trilead-api
      variant
      view-job-filters
      violations
      warnings
      warnings-ng
      windows-slaves
      workflow-aggregator
      workflow-api
      workflow-basic-steps
      workflow-cps
      workflow-cps-global-lib
      workflow-durable-task-step
      workflow-job
      workflow-multibranch
      workflow-scm-step
      workflow-step-api
      workflow-support
      ws-cleanup
      xunit
      

      Any recommendations on how to fix this?

          [JENKINS-64295] An attempt to save the global configuration was made before it was loaded

          Thanks teilo for the suggestion of SecurityRealm. Actually, I took over this Jenkins project and I'm not sure what SecurityRealm is. Any suggestion on how to find it. I did a grep of it and couldn't find anything except the logging:

          rep -r "setSecurityRealm" ./
          ./monitoring/_jenkins/last_shutdown.html:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at jenkins.model.Jenkins.setSecurityRealm(<a href='?part=source&amp;class=jenkins.model.Jenkins#2637' class='lightwindow' type='external' title='jenkins.model.Jenkins'>Jenkins.java:2637</a>)<br/>
          

          Albert Jankowski added a comment - Thanks teilo for the suggestion of SecurityRealm. Actually, I took over this Jenkins project and I'm not sure what SecurityRealm is. Any suggestion on how to find it. I did a grep of it and couldn't find anything except the logging: rep -r "setSecurityRealm" ./ ./monitoring/_jenkins/last_shutdown.html:&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;at jenkins.model.Jenkins.setSecurityRealm(<a href= '?part=source&amp;class=jenkins.model.Jenkins#2637' class= 'lightwindow' type= 'external' title= 'jenkins.model.Jenkins' >Jenkins.java:2637</a>)<br/>

          James Nord added a comment -

          https://www.jenkins.io/doc/book/managing/security/ will tell you what to look for. Basically the "Authentication" configuration (but Authorization may also be helpful here)

          James Nord added a comment - https://www.jenkins.io/doc/book/managing/security/ will tell you what to look for. Basically the "Authentication" configuration (but Authorization may also be helpful here)

          Alexander Dobetsberger added a comment - - edited

          I'm facing the same issue after the update.
          I use the Role-based Authorization Strategy plugin (Role-Based Strategy) as Authentication.

          After the reboot i couldn't login anymore, had do downgrade jenkins to fix my issue.

          Alexander Dobetsberger added a comment - - edited I'm facing the same issue after the update. I use the Role-based Authorization Strategy plugin (Role-Based Strategy) as Authentication. After the reboot i couldn't login anymore, had do downgrade jenkins to fix my issue.

          James Nord added a comment -

          just checked the stacks again... the config is loaded way before the security realm is set.
          "Loading global config" after=EXTENSIONS_AUGMENTED attains SYSTEM_CONFIG_LOADED

          saving the SecurityRealm
          "Finalizing set up" after=JOB_CONFIG_ADAPTED attains=COMPLETED

          What JVM are you using?

          James Nord added a comment - just checked the stacks again... the config is loaded way before the security realm is set. "Loading global config" after=EXTENSIONS_AUGMENTED attains SYSTEM_CONFIG_LOADED saving the SecurityRealm "Finalizing set up" after=JOB_CONFIG_ADAPTED attains=COMPLETED What JVM are you using?

          James Nord added a comment -

          there should also be a log entry before the exception with a little more information not that I expect it to help in this case though )

          can you add that to the ticket please

          James Nord added a comment - there should also be a log entry before the exception with a little more information not that I expect it to help in this case though ) can you add that to the ticket please

          James Nord added a comment -

          the call to persist in the stack in the comment is from
          here https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3342

          and that is inside the final reactor stage
          https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3301-L3356

          loading the config is called from the first stage
          https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3256-L3269

          and sets the flag to true https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3148-L3166

          The variable is never set to false afterwards.

          The variable is `volatile` and so the different reactor tasks even when done in different threads should see the end result.

          so the only thing I can think of is that the reactor ordering / blocking is completely broken or there is a JVM bug.

          I would a reactor bug to show up a lot more and with a lot more things are not working sort of way, and the reactor graph code has not changed in 2+ years.

          I am at a bit of a loss here oleg_nenashev / timja can you see anything awary with my analysis?

          at the least the full startup logs may reveal something, and adding startup logging with te following JVM system properties
          jenkins.model.Jenkins.initLogLevel=INFO and jenkins.model.Jenkins.logStartupPerformance=true

          James Nord added a comment - the call to persist in the stack in the comment is from here https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3342 and that is inside the final reactor stage https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3301-L3356 loading the config is called from the first stage https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3256-L3269 and sets the flag to true https://github.com/jenkinsci/jenkins/blob/ae88291731ddeb0e77c1944d997c4d60a77bfd87/core/src/main/java/jenkins/model/Jenkins.java#L3148-L3166 The variable is never set to false afterwards. The variable is `volatile` and so the different reactor tasks even when done in different threads should see the end result. so the only thing I can think of is that the reactor ordering / blocking is completely broken or there is a JVM bug. I would a reactor bug to show up a lot more and with a lot more things are not working sort of way, and the reactor graph code has not changed in 2+ years. I am at a bit of a loss here oleg_nenashev / timja can you see anything awary with my analysis? at the least the full startup logs may reveal something, and adding startup logging with te following JVM system properties jenkins.model.Jenkins.initLogLevel=INFO and jenkins.model.Jenkins.logStartupPerformance=true

          Oleg Nenashev added a comment -

          +1 for full startup log.

          No, no idea what happened here. I could imagine the field with "false" being de-serialized from the disk, but even in this case the flag should have been set to true afterwards in the loadConfig call. 

           

           

          Oleg Nenashev added a comment - +1 for full startup log. No, no idea what happened here. I could imagine the field with "false" being de-serialized from the disk, but even in this case the flag should have been set to true afterwards in the loadConfig call.     

          Alexander Dobetsberger added a comment - - edited

          Just tired the update again from 2.264 to 2.274 but i could not reproduce the execption anymore.

          I did update all plugins beforehand so maybe there was a fix in there somewhere.

           

          The problem i face is that i still can't login after the update, but i guess thats a different problem, i will create a new issue for that.

          Alexander Dobetsberger added a comment - - edited Just tired the update again from 2.264 to 2.274 but i could not reproduce the execption anymore. I did update all plugins beforehand so maybe there was a fix in there somewhere.   The problem i face is that i still can't login after the update, but i guess thats a different problem, i will create a new issue for that.

          Albert Jankowski added a comment - - edited

          I tried to upgrade Jenkins again and ran into the same issue. I ended up doing the following to fix the issue:

          • Downgraded Jenkins to 2.249
          • Updated Role-based Authorization Strategy to 3.1.1
          • Upgrade Jenkins to the latest (2.299)

          That fixed my issue. I was able to capture the log just in case it is needed:

          2021-06-23 13:35:37.681+0000 [id=34]	WARNING	hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created /var/lib/jenkins/plugins/jquery-ui/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness
          2021-06-23 13:35:37.682+0000 [id=34]	WARNING	hudson.ClassicPluginStrategy#createPluginWrapper: encountered /var/lib/jenkins/plugins/jquery-ui.hpi under a nonstandard name; expected jquery-ui.jpi
          2021-06-23 13:35:37.684+0000 [id=34]	INFO	hudson.PluginManager$1$3$1#isDuplicate: Ignoring /var/lib/jenkins/plugins/jquery-ui.hpi because /var/lib/jenkins/plugins/jquery-ui.jpi is already loaded
          2021-06-23 13:35:37.699+0000 [id=32]	INFO	jenkins.InitReactorRunner$1#onAttained: Listed all plugins
          2021-06-23 13:35:41.741+0000 [id=28]	INFO	o.j.h.p.backup.BackupPluginImpl#loadConfiguration: Loading configuration...
          2021-06-23 13:35:41.742+0000 [id=28]	SEVERE	o.j.h.p.backup.BackupPluginImpl#start: Error loading configuration file : /var/lib/jenkins/backup.xml
          2021-06-23 13:35:55.732+0000 [id=28]	INFO	jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
          2021-06-23 13:36:00.805+0000 [id=32]	INFO	jenkins.InitReactorRunner$1#onAttained: Started all plugins
          2021-06-23 13:36:00.842+0000 [id=34]	INFO	jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
          2021-06-23 13:36:00.879+0000 [id=31]	WARNING	jenkins.model.Jenkins$5#runTask: Loading global config failed perhaps due to plugin dependency issues
          java.lang.NoSuchMethodError: com.thoughtworks.xstream.io.HierarchicalStreamReader.peekNextChild()Ljava/lang/String;
          	at com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy$ConverterImpl.unmarshal(RoleBasedAuthorizationStrategy.java:593)
          	at hudson.util.XStream2$AssociatedConverterImpl.unmarshal(XStream2.java:466)
          	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
          	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
          	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
          	at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:443)
          	at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:345)
          	at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:284)
          	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72)
          	at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72)
          	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66)
          	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50)
          	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134)
          	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
          	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1431)
          	at hudson.util.XStream2.unmarshal(XStream2.java:161)
          	at hudson.util.XStream2.unmarshal(XStream2.java:132)
          	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1411)
          	at hudson.XmlFile.unmarshal(XmlFile.java:180)
          Caused: java.io.IOException: Unable to read /var/lib/jenkins/config.xml
          	at hudson.XmlFile.unmarshal(XmlFile.java:183)
          	at hudson.XmlFile.unmarshal(XmlFile.java:163)
          	at jenkins.model.Jenkins.loadConfig(Jenkins.java:3234)
          	at jenkins.model.Jenkins.access$1200(Jenkins.java:340)
          	at jenkins.model.Jenkins$12.run(Jenkins.java:3336)
          	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
          	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
          	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160)
          	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
          	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          2021-06-23 13:36:00.889+0000 [id=31]	INFO	jenkins.InitReactorRunner$1#onAttained: System config loaded
          2021-06-23 13:36:01.570+0000 [id=30]	INFO	jenkins.InitReactorRunner$1#onAttained: System config adapted
          2021-06-23 13:36:02.683+0000 [id=27]	INFO	jenkins.InitReactorRunner$1#onAttained: Loaded all jobs
          2021-06-23 13:36:02.685+0000 [id=34]	INFO	jenkins.InitReactorRunner$1#onAttained: Configuration for all jobs updated
          2021-06-23 13:36:02.705+0000 [id=51]	INFO	hudson.model.AsyncPeriodicWork#lambda$doRun$0: Started Download metadata
          2021-06-23 13:36:02.760+0000 [id=51]	INFO	hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Download metadata. 53 ms
          2021-06-23 13:36:02.786+0000 [id=29]	SEVERE	jenkins.model.Jenkins#save: An attempt to save Jenkins'' global configuration before it has been loaded has been made during milestone Configuration for all jobs updated.  This is indicative of a bug in the caller and may lead to full or partial loss of configuration.
          java.lang.IllegalStateException: call trace
          	at jenkins.model.Jenkins.save(Jenkins.java:3452)
          	at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3480)
          	at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2707)
          	at jenkins.model.Jenkins$15.run(Jenkins.java:3423)
          	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
          	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
          	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160)
          	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
          	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          2021-06-23 13:36:02.789+0000 [id=29]	SEVERE	jenkins.InitReactorRunner$1#onTaskFailed: Failed Finalizing set up
          java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded
          	at jenkins.model.Jenkins.save(Jenkins.java:3461)
          	at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3480)
          	at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2707)
          	at jenkins.model.Jenkins$15.run(Jenkins.java:3423)
          	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
          	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
          	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160)
          	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
          	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          2021-06-23 13:36:02.790+0000 [id=21]	SEVERE	hudson.util.BootFailure#publish: Failed to initialize Jenkins
          java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded
          	at jenkins.model.Jenkins.save(Jenkins.java:3461)
          	at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3480)
          	at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2707)
          	at jenkins.model.Jenkins$15.run(Jenkins.java:3423)
          	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
          	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
          	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160)
          	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
          	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          Caused: org.jvnet.hudson.reactor.ReactorException
          	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282)
          	at jenkins.InitReactorRunner.run(InitReactorRunner.java:50)
          	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1193)
          	at jenkins.model.Jenkins.<init>(Jenkins.java:991)
          	at hudson.model.Hudson.<init>(Hudson.java:86)
          	at hudson.model.Hudson.<init>(Hudson.java:82)
          	at hudson.WebAppMain$3.run(WebAppMain.java:298)
          Caused: hudson.util.HudsonFailedToLoad
          	at hudson.WebAppMain$3.run(WebAppMain.java:315)
          2021-06-23 13:36:02.796+0000 [id=21]	INFO	jenkins.model.Jenkins#cleanUp: Stopping Jenkins
          2021-06-23 13:36:02.845+0000 [id=27]	SEVERE	hudson.model.User#loadFromUserConfigFile: Failed to load /var/lib/jenkins/users/noreply_5389189741004524536/config.xml
          java.nio.channels.ClosedByInterruptException
          	at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
          	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:164)
          	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65)
          	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109)
          	at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
          	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
          	at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
          	at java.io.FilterInputStream.read(FilterInputStream.java:83)
          	at java.io.PushbackInputStream.read(PushbackInputStream.java:139)
          	at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.getHeader(XmlHeaderAwareReader.java:79)
          	at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.<init>(XmlHeaderAwareReader.java:61)
          	at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65)
          Caused: com.thoughtworks.xstream.io.StreamException: 
          	at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:69)
          	at hudson.XmlFile.unmarshal(XmlFile.java:180)
          Caused: java.io.IOException: Unable to read /var/lib/jenkins/users/noreply_5389189741004524536/config.xml
          	at hudson.XmlFile.unmarshal(XmlFile.java:183)
          	at hudson.XmlFile.unmarshal(XmlFile.java:163)
          	at hudson.model.User.loadFromUserConfigFile(User.java:225)
          	at hudson.model.User.load(User.java:195)
          	at hudson.model.User.<init>(User.java:190)
          	at hudson.model.User.<init>(User.java:122)
          	at hudson.model.User$AllUsers.scanAll(User.java:1081)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          	at java.lang.reflect.Method.invoke(Method.java:498)
          	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105)
          	at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:181)
          	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
          	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160)
          	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
          	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
          	at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
          	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          	at java.lang.Thread.run(Thread.java:748)
          2021-06-23 13:36:02.850+0000 [id=27]	SEVERE	hudson.model.User#loadFromUserConfigFile: Failed to load /var/lib/jenkins/users/tomcat_5709365328938117350/config.xml
          

          Albert Jankowski added a comment - - edited I tried to upgrade Jenkins again and ran into the same issue. I ended up doing the following to fix the issue: Downgraded Jenkins to 2.249 Updated Role-based Authorization Strategy to 3.1.1 Upgrade Jenkins to the latest (2.299) That fixed my issue. I was able to capture the log just in case it is needed: 2021-06-23 13:35:37.681+0000 [id=34] WARNING hudson.ClassicPluginStrategy#createClassJarFromWebInfClasses: Created / var /lib/jenkins/plugins/jquery-ui/WEB-INF/lib/classes.jar; update plugin to a version created with a newer harness 2021-06-23 13:35:37.682+0000 [id=34] WARNING hudson.ClassicPluginStrategy#createPluginWrapper: encountered / var /lib/jenkins/plugins/jquery-ui.hpi under a nonstandard name; expected jquery-ui.jpi 2021-06-23 13:35:37.684+0000 [id=34] INFO hudson.PluginManager$1$3$1#isDuplicate: Ignoring / var /lib/jenkins/plugins/jquery-ui.hpi because / var /lib/jenkins/plugins/jquery-ui.jpi is already loaded 2021-06-23 13:35:37.699+0000 [id=32] INFO jenkins.InitReactorRunner$1#onAttained: Listed all plugins 2021-06-23 13:35:41.741+0000 [id=28] INFO o.j.h.p.backup.BackupPluginImpl#loadConfiguration: Loading configuration... 2021-06-23 13:35:41.742+0000 [id=28] SEVERE o.j.h.p.backup.BackupPluginImpl#start: Error loading configuration file : / var /lib/jenkins/backup.xml 2021-06-23 13:35:55.732+0000 [id=28] INFO jenkins.InitReactorRunner$1#onAttained: Prepared all plugins 2021-06-23 13:36:00.805+0000 [id=32] INFO jenkins.InitReactorRunner$1#onAttained: Started all plugins 2021-06-23 13:36:00.842+0000 [id=34] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all extensions 2021-06-23 13:36:00.879+0000 [id=31] WARNING jenkins.model.Jenkins$5#runTask: Loading global config failed perhaps due to plugin dependency issues java.lang.NoSuchMethodError: com.thoughtworks.xstream.io.HierarchicalStreamReader.peekNextChild()Ljava/lang/ String ; at com.michelin.cio.hudson.plugins.rolestrategy.RoleBasedAuthorizationStrategy$ConverterImpl.unmarshal(RoleBasedAuthorizationStrategy.java:593) at hudson.util.XStream2$AssociatedConverterImpl.unmarshal(XStream2.java:466) at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66) at hudson.util.RobustReflectionConverter.unmarshalField(RobustReflectionConverter.java:443) at hudson.util.RobustReflectionConverter.doUnmarshal(RobustReflectionConverter.java:345) at hudson.util.RobustReflectionConverter.unmarshal(RobustReflectionConverter.java:284) at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:72) at com.thoughtworks.xstream.core.AbstractReferenceUnmarshaller.convert(AbstractReferenceUnmarshaller.java:72) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:66) at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:50) at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:134) at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1431) at hudson.util.XStream2.unmarshal(XStream2.java:161) at hudson.util.XStream2.unmarshal(XStream2.java:132) at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1411) at hudson.XmlFile.unmarshal(XmlFile.java:180) Caused: java.io.IOException: Unable to read / var /lib/jenkins/config.xml at hudson.XmlFile.unmarshal(XmlFile.java:183) at hudson.XmlFile.unmarshal(XmlFile.java:163) at jenkins.model.Jenkins.loadConfig(Jenkins.java:3234) at jenkins.model.Jenkins.access$1200(Jenkins.java:340) at jenkins.model.Jenkins$12.run(Jenkins.java:3336) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) 2021-06-23 13:36:00.889+0000 [id=31] INFO jenkins.InitReactorRunner$1#onAttained: System config loaded 2021-06-23 13:36:01.570+0000 [id=30] INFO jenkins.InitReactorRunner$1#onAttained: System config adapted 2021-06-23 13:36:02.683+0000 [id=27] INFO jenkins.InitReactorRunner$1#onAttained: Loaded all jobs 2021-06-23 13:36:02.685+0000 [id=34] INFO jenkins.InitReactorRunner$1#onAttained: Configuration for all jobs updated 2021-06-23 13:36:02.705+0000 [id=51] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Started Download metadata 2021-06-23 13:36:02.760+0000 [id=51] INFO hudson.model.AsyncPeriodicWork#lambda$doRun$0: Finished Download metadata. 53 ms 2021-06-23 13:36:02.786+0000 [id=29] SEVERE jenkins.model.Jenkins#save: An attempt to save Jenkins'' global configuration before it has been loaded has been made during milestone Configuration for all jobs updated. This is indicative of a bug in the caller and may lead to full or partial loss of configuration. java.lang.IllegalStateException: call trace at jenkins.model.Jenkins.save(Jenkins.java:3452) at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3480) at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2707) at jenkins.model.Jenkins$15.run(Jenkins.java:3423) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) 2021-06-23 13:36:02.789+0000 [id=29] SEVERE jenkins.InitReactorRunner$1#onTaskFailed: Failed Finalizing set up java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded at jenkins.model.Jenkins.save(Jenkins.java:3461) at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3480) at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2707) at jenkins.model.Jenkins$15.run(Jenkins.java:3423) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) 2021-06-23 13:36:02.790+0000 [id=21] SEVERE hudson.util.BootFailure#publish: Failed to initialize Jenkins java.lang.IllegalStateException: An attempt to save the global configuration was made before it was loaded at jenkins.model.Jenkins.save(Jenkins.java:3461) at jenkins.model.Jenkins.saveQuietly(Jenkins.java:3480) at jenkins.model.Jenkins.setSecurityRealm(Jenkins.java:2707) at jenkins.model.Jenkins$15.run(Jenkins.java:3423) at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) Caused: org.jvnet.hudson.reactor.ReactorException at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282) at jenkins.InitReactorRunner.run(InitReactorRunner.java:50) at jenkins.model.Jenkins.executeReactor(Jenkins.java:1193) at jenkins.model.Jenkins.<init>(Jenkins.java:991) at hudson.model.Hudson.<init>(Hudson.java:86) at hudson.model.Hudson.<init>(Hudson.java:82) at hudson.WebAppMain$3.run(WebAppMain.java:298) Caused: hudson.util.HudsonFailedToLoad at hudson.WebAppMain$3.run(WebAppMain.java:315) 2021-06-23 13:36:02.796+0000 [id=21] INFO jenkins.model.Jenkins#cleanUp: Stopping Jenkins 2021-06-23 13:36:02.845+0000 [id=27] SEVERE hudson.model.User#loadFromUserConfigFile: Failed to load / var /lib/jenkins/users/noreply_5389189741004524536/config.xml java.nio.channels.ClosedByInterruptException at java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:164) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109) at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read(BufferedInputStream.java:265) at java.io.FilterInputStream.read(FilterInputStream.java:83) at java.io.PushbackInputStream.read(PushbackInputStream.java:139) at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.getHeader(XmlHeaderAwareReader.java:79) at com.thoughtworks.xstream.core.util.XmlHeaderAwareReader.<init>(XmlHeaderAwareReader.java:61) at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:65) Caused: com.thoughtworks.xstream.io.StreamException: at com.thoughtworks.xstream.io.xml.AbstractXppDriver.createReader(AbstractXppDriver.java:69) at hudson.XmlFile.unmarshal(XmlFile.java:180) Caused: java.io.IOException: Unable to read / var /lib/jenkins/users/noreply_5389189741004524536/config.xml at hudson.XmlFile.unmarshal(XmlFile.java:183) at hudson.XmlFile.unmarshal(XmlFile.java:163) at hudson.model.User.loadFromUserConfigFile(User.java:225) at hudson.model.User.load(User.java:195) at hudson.model.User.<init>(User.java:190) at hudson.model.User.<init>(User.java:122) at hudson.model.User$AllUsers.scanAll(User.java:1081) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:105) at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:181) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1160) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang. Thread .run( Thread .java:748) 2021-06-23 13:36:02.850+0000 [id=27] SEVERE hudson.model.User#loadFromUserConfigFile: Failed to load / var /lib/jenkins/users/tomcat_5709365328938117350/config.xml

          IIUC this was caused by https://www.jenkins.io/blog/2020/11/10/spring-xstream/, resolved in 3.1 of the role-strategy plugin, and can therefore be closed.

          Allan BURDAJEWICZ added a comment - IIUC this was caused by https://www.jenkins.io/blog/2020/11/10/spring-xstream/ , resolved in 3.1 of the role-strategy plugin, and can therefore be closed.

            Unassigned Unassigned
            albertski Albert Jankowski
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: