Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53906

Jenkins fails to start with Exception after update to 2.144

    • Icon: Bug Bug
    • Resolution: Not A Defect
    • Icon: Major Major
    • core, gradle-plugin
    • None
    • Operating system: CentOS release 6.10 x86_64
      JRE running Jenkins: OpenJDK 1.8.0_181-b13
      Jenkins version: 2.144 (updated from 2.142), installed via yum from repository
      Gradle Plugin: 1.29

      After updating Jenkins from 2.142 to 2.144 via yum, Jenkins does not successfully start up. On accessing the web UI the following exception is displayed:

      java.lang.IllegalArgumentException: Comparison method violates its general contract!
      	at java.util.ComparableTimSort.mergeHi(ComparableTimSort.java:866)
      	at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:483)
      	at java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:422)
      	at java.util.ComparableTimSort.sort(ComparableTimSort.java:222)
      	at java.util.Arrays.sort(Arrays.java:1312)
      	at java.util.Arrays.sort(Arrays.java:1506)
      	at java.util.ArrayList.sort(ArrayList.java:1462)
      	at java.util.Collections.sort(Collections.java:141)
      	at hudson.ExtensionList.sort(ExtensionList.java:400)
      	at hudson.ExtensionList.ensureLoaded(ExtensionList.java:320)
      	at hudson.ExtensionList.iterator(ExtensionList.java:172)
      	at jenkins.model.Jenkins.getDescriptorByType(Jenkins.java:1522)
      	at hudson.plugins.copyartifact.BuildSelectorParameter.initAliases(BuildSelectorParameter.java:158)
      Caused: java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:104)
      Caused: java.lang.Error
      	at hudson.init.TaskMethodFinder.invoke(TaskMethodFinder.java:110)
      	at hudson.init.TaskMethodFinder$TaskImpl.run(TaskMethodFinder.java:175)
      	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:296)
      	at jenkins.model.Jenkins$5.runTask(Jenkins.java:1066)
      	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:214)
      	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused: org.jvnet.hudson.reactor.ReactorException
      	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:282)
      	at jenkins.InitReactorRunner.run(InitReactorRunner.java:48)
      	at jenkins.model.Jenkins.executeReactor(Jenkins.java:1100)
      	at jenkins.model.Jenkins.<init>(Jenkins.java:904)
      	at hudson.model.Hudson.<init>(Hudson.java:85)
      	at hudson.model.Hudson.<init>(Hudson.java:81)
      	at hudson.WebAppMain$3.run(WebAppMain.java:233)
      Caused: hudson.util.HudsonFailedToLoad
      	at hudson.WebAppMain$3.run(WebAppMain.java:250)
      Page generated: Oct 4, 2018 12:34:51 PM CESTJenkins ver. 2.144

      In the jenkins.log log file there are many exceptions. I will attach text files with examples. Some of the exceptions seem to point to the Gradle Plugin but I may be wrong about that. I can't easily uninstall the Gradle Plugin to test, because we depend on two plugins which have the Gradle Plugin as a dependency.

      For now a Jenkins downgrade to 2.142 solves the issue.

        1. 1.txt
          6 kB
        2. 2.txt
          2 kB
        3. 3.txt
          2 kB
        4. 4.txt
          2 kB
        5. 5.txt
          3 kB

          [JENKINS-53906] Jenkins fails to start with Exception after update to 2.144

          Daniel Beck added a comment -

          sezpoz looks like the most likely culprit. I liked Groovy too, until I saw that Gradle Plugin is written in Java.

          FYI jglick

          Daniel Beck added a comment - sezpoz looks like the most likely culprit. I liked Groovy too, until I saw that Gradle Plugin is written in Java. FYI jglick

          Jesse Glick added a comment -

          SezPoz was indeed updated, though I see no link to this issue:

          java.lang.NoClassDefFoundError: com/trend/iwss/jscan/runtime/Session
          	at hudson.plugins.gradle.Messages.<clinit>(Messages.java)
          	at hudson.plugins.gradle.Gradle$DescriptorImpl.getDisplayName(Gradle.java:471)
          	at hudson.ExtensionComponent.compareTo(ExtensionComponent.java:97)
          	at hudson.ExtensionComponent.compareTo(ExtensionComponent.java:42)
          	at java.util.ComparableTimSort.binarySort(ComparableTimSort.java:262)
          	at java.util.ComparableTimSort.sort(ComparableTimSort.java:207)
          	at java.util.Arrays.sort(Arrays.java:1312)
          	at java.util.Arrays.sort(Arrays.java:1506)
          	at java.util.ArrayList.sort(ArrayList.java:1462)
          	at java.util.Collections.sort(Collections.java:141)
          	at hudson.ExtensionList.sort(ExtensionList.java:400)
          	at …
          

          which does not make any sense to me. Messages.java is generated and makes no reference to that library, whatever it is.

          Is there some way to reproduce this from scratch?

          Jesse Glick added a comment - SezPoz was indeed updated, though I see no link to this issue: java.lang.NoClassDefFoundError: com/trend/iwss/jscan/runtime/Session at hudson.plugins.gradle.Messages.<clinit>(Messages.java) at hudson.plugins.gradle.Gradle$DescriptorImpl.getDisplayName(Gradle.java:471) at hudson.ExtensionComponent.compareTo(ExtensionComponent.java:97) at hudson.ExtensionComponent.compareTo(ExtensionComponent.java:42) at java.util.ComparableTimSort.binarySort(ComparableTimSort.java:262) at java.util.ComparableTimSort.sort(ComparableTimSort.java:207) at java.util.Arrays.sort(Arrays.java:1312) at java.util.Arrays.sort(Arrays.java:1506) at java.util.ArrayList.sort(ArrayList.java:1462) at java.util.Collections.sort(Collections.java:141) at hudson.ExtensionList.sort(ExtensionList.java:400) at … which does not make any sense to me. Messages.java is generated and makes no reference to that library, whatever it is. Is there some way to reproduce this from scratch?

          Frank Shimizu added a comment - - edited

          jglick, I set up Jenkins 2.142 from scratch in Docker with jenkins/jenkins:2.142 and installed the exact same plugins as on our production system. I did no configuration or otherwise. Then I updated to 2.144 by deleting the Docker container and starting a new one with the :2.144 image. I could not reproduce the problem; Jenkins starts normally.

          Something I noticed:

          I had to manually download the hpi files for the ruby-runtime and gradle plugins because of download/checksum errors. ruby-runtime was manually installed without a problem. gradle initially failed the automatic installation because of a checksum error. I downloaded gradle plugin 1.29 from https://updates.jenkins-ci.org/download/plugins/gradle/ and indeed the checksums don't match. I still installed it manually via the Advanced -> Upload Plugin function and despite some errors it was successfully installed. Then after updating Jenkins to 2.144, there were now many Exceptions related to hudson.plugins.gradle.Messages during startup. But Jenkins 2.144 still successfully started. I will try again to reproduce this next week.

          Frank Shimizu added a comment - - edited jglick , I set up Jenkins 2.142 from scratch in Docker with jenkins/jenkins:2.142 and installed the exact same plugins as on our production system. I did no configuration or otherwise. Then I updated to 2.144 by deleting the Docker container and starting a new one with the :2.144 image. I could not reproduce the problem; Jenkins starts normally. Something I noticed: I had to manually download the hpi files for the ruby-runtime and gradle plugins because of download/checksum errors. ruby-runtime was manually installed without a problem. gradle initially failed the automatic installation because of a checksum error. I downloaded gradle plugin 1.29 from https://updates.jenkins-ci.org/download/plugins/gradle/  and indeed the checksums don't match. I still installed it manually via the Advanced -> Upload Plugin function and despite some errors it was successfully installed. Then after updating Jenkins to 2.144, there were now many Exceptions related to hudson.plugins.gradle.Messages during startup. But Jenkins 2.144 still successfully started. I will try again to reproduce this next week.

          Daniel Beck added a comment -

          ruby-runtime is a known pile of garbage I have so far been unsuccessful in removing from distribution. Seems I need to renew my efforts.

          Sorry for the false alarm jglick.

          Daniel Beck added a comment - ruby-runtime is a known pile of garbage I have so far been unsuccessful in removing from distribution. Seems I need to renew my efforts. Sorry for the false alarm jglick .

          Jesse Glick added a comment -

          No clue. Note that the ruby-runtime plugin is no longer supported.

          Jesse Glick added a comment - No clue. Note that the ruby-runtime plugin is no longer supported .

          Jesse Glick added a comment -

          Well, there is nothing in the stack traces I saw implicating ruby-runtime specifically, whereas I did see some stuff indicating something was very broken in the gradle plugin.

          Jesse Glick added a comment - Well, there is nothing in the stack traces I saw implicating ruby-runtime specifically, whereas I did see some stuff indicating something was very broken in the gradle plugin.

          Frank Shimizu added a comment -

          Thank you for all the information so far, especially about the deprecation of ruby-runtime. I was able to disable ruby-runtime since we do not seem to actually use its dependency. Which is the GitLab Hook Plugin.

           

          I also manually downgraded the gradle plugin to 1.24 which for me seems to be the latest release with matching checksums. Now the Jenkins update to 2.144 seems to have succeeded. Jenkins starts up successfully, the many Exceptions in the log are gone and we're testing whether everything works. I may also proceed to update to 2.145 since it's available now.

           

          Not sure if the issue with the gradle plugin checksums happens only to us or others too. Should I close this issue if we find no more problems with the update and create a new one concerning the gradle plugin checksums?

          Frank Shimizu added a comment - Thank you for all the information so far, especially about the deprecation of ruby-runtime. I was able to disable ruby-runtime since we do not seem to actually use its dependency. Which is the GitLab Hook Plugin.   I also manually downgraded the gradle plugin to 1.24 which for me seems to be the latest release with matching checksums. Now the Jenkins update to 2.144 seems to have succeeded. Jenkins starts up successfully, the many Exceptions in the log are gone and we're testing whether everything works. I may also proceed to update to 2.145 since it's available now.   Not sure if the issue with the gradle plugin checksums happens only to us or others too. Should I close this issue if we find no more problems with the update and create a new one concerning the gradle plugin checksums?

          Frank Shimizu added a comment -

          Update to 2.145 was successful too. Again thanks for the helpful information. I'm setting this issue to Resolved.

          Frank Shimizu added a comment - Update to 2.145 was successful too. Again thanks for the helpful information. I'm setting this issue to Resolved.

          Frank Shimizu added a comment - - edited

          Fixed by downgrading to a gradle plugin version with matching checksums.

          Frank Shimizu added a comment - - edited Fixed by downgrading to a gradle plugin version with matching checksums.

          Jesse Glick added a comment -

          Not sure what exactly happened here.

          Jesse Glick added a comment - Not sure what exactly happened here.

            wolfs Stefan Wolf
            shimizuf Frank Shimizu
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: