Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-68752

Jenkins version upgrade fails from 2.289.3 to 2.332.3 LTS

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • core
    • production
    • 2.357, 2.346.2

      I'm kind of stuck, trying to upgrade Jenkins LTS version from 2.289.3 to 2.332.3 but after upgrade process when Jenkins restart I'm seeing below error on screen and unable to perform any function:

       Error

      java.lang.IllegalStateException: Duplicate key Digital-Deploy (attempted merging values hudson.logging.LogRecorder@75681a9b and hudson.logging.LogRecorder@75681a9b) at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:133) at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180) at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) at hudson.logging.LogRecorderManager.setRecorders(LogRecorderManager.java:105) at hudson.logging.LogRecorderManager.load(LogRecorderManager.java:146) at hudson.logging.LogRecorderManager.init(LogRecorderManager.java:261) Caused: java.lang.reflect.InvocationTargetException at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:109) Caused: java.lang.Error at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:115) at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:185) at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:305) at jenkins.model.Jenkins$5.runTask(Jenkins.java:1156) at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:222) at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:121) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused: org.jvnet.hudson.reactor.ReactorException at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:291) at jenkins.InitReactorRunner.run(InitReactorRunner.java:49) at jenkins.model.Jenkins.executeReactor(Jenkins.java:1191) at jenkins.model.Jenkins.<init>(Jenkins.java:981) at hudson.model.Hudson.<init>(Hudson.java:86) at hudson.model.Hudson.<init>(Hudson.java:82) at hudson.WebAppMain$3.run(WebAppMain.java:247) Caused: hudson.util.HudsonFailedToLoad at hudson.WebAppMain$3.run(WebAppMain.java:264)

          [JENKINS-68752] Jenkins version upgrade fails from 2.289.3 to 2.332.3 LTS

          Habib Omer added a comment -

          This is blocking me, did anyone came across this type of issue when upgrading Jenkins version?

          Habib Omer added a comment - This is blocking me, did anyone came across this type of issue when upgrading Jenkins version?

          Mark Waite added a comment -

          Never seen that message before.  It almost looks like a logging configuration on the controller might have multiple definitions for the same key 'Digital-Deploy'.  Does that string appear in one of the Jenkins configuration files?  Is it there more than once?  If you remove the duplication, does it resolve the issue?

          https://www.jenkins.io/doc/upgrade-guide/ shows that there are 9 upgrade guides that you should read so that you understand more about the changes that have been made since your original version.

          Mark Waite added a comment - Never seen that message before.  It almost looks like a logging configuration on the controller might have multiple definitions for the same key 'Digital-Deploy'.  Does that string appear in one of the Jenkins configuration files?  Is it there more than once?  If you remove the duplication, does it resolve the issue? https://www.jenkins.io/doc/upgrade-guide/ shows that there are 9 upgrade guides that you should read so that you understand more about the changes that have been made since your original version.

          Basil Crow added a comment -

          The relevant code was modified recently in JENKINS-61278 / https://github.com/jenkinsci/jenkins/pull/4538

          Basil Crow added a comment - The relevant code was modified recently in JENKINS-61278 / https://github.com/jenkinsci/jenkins/pull/4538

          Habib Omer added a comment -

          markewaite Last month I was able to upgraded Jenkins LTS version from 2.289.3 to 2.332.2 on our dev environment without any problem, but having issues when tried on prod.

          basil Should recently modified code changes fixed this issue now? 

          Habib Omer added a comment - markewaite Last month I was able to upgraded Jenkins LTS version from 2.289.3 to 2.332.2 on our dev environment without any problem, but having issues when tried on prod. basil Should recently modified code changes fixed this issue now? 

          Mark Waite added a comment -

          haalkaf  the pull request description indicates that you probably have duplicate data in your configuration as code definition files.  If it is working as expected in your development environment and not working in production, you may want to compare the configuration as code files between the two locations.

          Mark Waite added a comment - haalkaf   the pull request description indicates that you probably have duplicate data in your configuration as code definition files.  If it is working as expected in your development environment and not working in production, you may want to compare the configuration as code files between the two locations.

          Habib Omer added a comment -

          markewaite  Where I can find configuration as code definition files?

          Habib Omer added a comment - markewaite   Where I can find configuration as code definition files?

          haalkaf, by default, the Configuration as Code plugin reads jenkins.yaml in the directory of the Jenkins controller. This can be overridden in several ways:

          • "casc.jenkins.config" Java system property
          • "CASC_JENKINS_CONFIG" environment variable
          • a setting that Jenkins saves to the "io.jenkins.plugins.casc.CasCGlobalConfig.xml" file

          Kalle Niemitalo added a comment - haalkaf , by default, the Configuration as Code plugin reads jenkins.yaml in the directory of the Jenkins controller. This can be overridden in several ways: "casc.jenkins.config" Java system property "CASC_JENKINS_CONFIG" environment variable a setting that Jenkins saves to the "io.jenkins.plugins.casc.CasCGlobalConfig.xml" file

          However, I don't see any mention of "configuration as code" in https://github.com/jenkinsci/jenkins/pull/6673. It does mention "copying files around on the file system" but I think that refers to "log/*.xml" files, not to anything used by the Configuration as Code plugin. You could search for "<name>Digital-Deploy</name>" in the "log/*.xml" files.

          Kalle Niemitalo added a comment - However, I don't see any mention of "configuration as code" in https://github.com/jenkinsci/jenkins/pull/6673 . It does mention "copying files around on the file system" but I think that refers to "log/*.xml" files, not to anything used by the Configuration as Code plugin. You could search for "<name>Digital-Deploy</name>" in the "log/*.xml" files.

          Habib Omer added a comment -

          After removing "Digital-Deploy" files from /var/lib/jenkins/log, I'm successfully able to upgrade latest LTS 2.346.1 version. Thank you all for your comments.

          Habib Omer added a comment - After removing "Digital-Deploy" files from /var/lib/jenkins/log, I'm successfully able to upgrade latest LTS 2.346.1 version. Thank you all for your comments.

            timja Tim Jacomb
            haalkaf Habib Omer
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: