Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-19383

Bytecode compatibility transformer mistakenly corrupts org.apache.ivy.core.settings.IvySettings.triggers

      On two separate Jenkins servers, I'm unable to start Jenkins with the following error,

      org.jvnet.hudson.reactor.ReactorException: java.lang.VerifyError: (class: org/apache/ivy/core/settings/IvySettings, method: getTriggers signature: ()Ljava/util/List;\) Illegal type in constant pool
      	at org.jvnet.hudson.reactor.Reactor.execute(Reactor.java:246)
      	at jenkins.InitReactorRunner.run(InitReactorRunner.java:43)
      	at jenkins.model.Jenkins.executeReactor(Jenkins.java:906)
      	at jenkins.model.Jenkins.<init>(Jenkins.java:806)
      	at hudson.model.Hudson.<init>(Hudson.java:81)
      	at hudson.model.Hudson.<init>(Hudson.java:77)
      	at hudson.WebAppMain$3.run(WebAppMain.java:221)
      Caused by: java.lang.VerifyError: (class: org/apache/ivy/core/settings/IvySettings, method: getTriggers signature: ()Ljava/util/List;\) Illegal type in constant pool
      	at hudson.ivy.IvyBuildTrigger.getIvy(IvyBuildTrigger.java:234)
      	at hudson.ivy.IvyBuildTrigger.recomputeModuleDescriptor(IvyBuildTrigger.java:337)
      	at hudson.ivy.IvyBuildTrigger.getModuleDescriptor(IvyBuildTrigger.java:264)
      	at hudson.ivy.IvyBuildTrigger.buildDependencyGraph(IvyBuildTrigger.java:446)
      	at hudson.util.DescribableList.buildDependencyGraph(DescribableList.java:213)
      	at hudson.model.Project.buildDependencyGraph(Project.java:179)
      	at hudson.model.DependencyGraph.build(DependencyGraph.java:95)
      	at jenkins.model.Jenkins.rebuildDependencyGraph(Jenkins.java:3598)
      	at jenkins.model.Jenkins$20.run(Jenkins.java:2577)
      	at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:146)
      	at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:259)
      	at jenkins.model.Jenkins$7.runTask(Jenkins.java:895)
      	at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:187)
      	at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:94)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.lang.Thread.run(Unknown Source)
      

      I tried different versions, and I see this error with 1.527, 1.528, and 1.529. I've rolled back to 1.526 which is working.

      Let me know if I can provide any additional information. Thanks!

      Update:
      I should also clarify that I'm using free-style projects with "Trigger the build of other projects based on the Ivy dependency management system" post-build action.

      I installed another Jenkins system to debug this issue. On this fresh system, Jenkins started successfully. However, after each build, the verify error showed up in jenkins.err.log,

      Sep 04, 2013 9:10:47 AM SEVERE hudson.model.Executor run
      Executor threw an exception
      java.lang.VerifyError: (class: org/apache/ivy/core/settings/IvySettings, method: getTriggers signature: ()Ljava/util/List;) Illegal type in constant pool
      	at hudson.ivy.IvyBuildTrigger.getIvy(IvyBuildTrigger.java:234)
      	at hudson.ivy.IvyBuildTrigger.recomputeModuleDescriptor(IvyBuildTrigger.java:337)
      	at hudson.ivy.IvyBuildTrigger.perform(IvyBuildTrigger.java:418)
      	at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:45)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:804)
      	at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:776)
      	at hudson.model.Build$BuildExecution.cleanUp(Build.java:192)
      	at hudson.model.Run.execute(Run.java:1648)
      	at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
      	at hudson.model.ResourceController.execute(ResourceController.java:88)
      	at hudson.model.Executor.run(Executor.java:247)
      

      I turned on -verbose:class, and it looks like the right jar is being used,

      [Loaded org.apache.ivy.core.sort.SortEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.plugins.parser.ParserSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.publish.PublishEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.deliver.DeliverEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.check.CheckEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.install.InstallEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.plugins.resolver.ResolverSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.resolve.ResolveEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.retrieve.RetrieveEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.repository.RepositoryManagementEngineSettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.settings.IvySettings from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.RelativeUrlResolver from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.NormalRelativeUrlResolver from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.cache.RepositoryCacheManager from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.plugins.lock.LockStrategy from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.util.filter.Filter from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.plugins.conflict.ConflictManager from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.plugins.latest.LatestStrategy from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded org.apache.ivy.core.cache.ResolutionCacheManager from file:/C:/Program%20Files%20(x86)/Jenkins/plugins/ivy/WEB-INF/lib/ivy-2.3.0.jar]
      [Loaded java.lang.VerifyError from C:\Java\jdk1.6.0_30_x64\jre\lib\rt.jar]
      

      Also verified that the ivy-2.3.0.jar matched a freshly downloaded copy from apache.

      Stepped through in a debugger. Sadly, there's not much additional information on a VerifyError. In an expression, I was able to resolve other classes in the ivy jar but not IvySettings.

      Searching around the web, VerifyError can be frustrating to deal with. There is Good tools for debugging VerifyError? on StackOverflow. I tried using ASM as suggested in an answer, and no errors were reported. I haven't tried Krakatau yet.

      I noticed that the fresh install of Jenkins 1.529 came with Java 1.7u25. I tried running with Java 1.6(u30) and ran into the same verify error.

          [JENKINS-19383] Bytecode compatibility transformer mistakenly corrupts org.apache.ivy.core.settings.IvySettings.triggers

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          changelog.html
          core/pom.xml
          http://jenkins-ci.org/commit/jenkins/27d1a03866e468d2d9b630f636e53ec7804e4ade
          Log:
          [FIXED JENKINS-19383]

          Fixed a bug in bytecode-compatibility-transformer where class file
          format prior to 1.49 cannot have the ldc instruction pointing to a class
          constant.

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: changelog.html core/pom.xml http://jenkins-ci.org/commit/jenkins/27d1a03866e468d2d9b630f636e53ec7804e4ade Log: [FIXED JENKINS-19383] Fixed a bug in bytecode-compatibility-transformer where class file format prior to 1.49 cannot have the ldc instruction pointing to a class constant.

          dogfood added a comment -

          Integrated in jenkins_main_trunk #2980
          [FIXED JENKINS-19383] (Revision 27d1a03866e468d2d9b630f636e53ec7804e4ade)

          Result = SUCCESS
          kohsuke : 27d1a03866e468d2d9b630f636e53ec7804e4ade
          Files :

          • core/pom.xml
          • changelog.html

          dogfood added a comment - Integrated in jenkins_main_trunk #2980 [FIXED JENKINS-19383] (Revision 27d1a03866e468d2d9b630f636e53ec7804e4ade) Result = SUCCESS kohsuke : 27d1a03866e468d2d9b630f636e53ec7804e4ade Files : core/pom.xml changelog.html

          John McCarthy added a comment -

          Thanks Kohsuke! I'll verify and close this issue when it's released in Jenkins (looks like 1.538).

          John McCarthy added a comment - Thanks Kohsuke! I'll verify and close this issue when it's released in Jenkins (looks like 1.538).

          John McCarthy added a comment -

          Verified as fixed in 1.538. Thanks again!

          John McCarthy added a comment - Verified as fixed in 1.538. Thanks again!

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          core/src/main/java/hudson/ClassicPluginStrategy.java
          http://jenkins-ci.org/commit/jenkins/371826ac164cf4640ff20cec242c2efb79f5baaa
          Log:
          JENKINS-19383

          Escape hatch to disable bytecode transformation in case this causes other unforeseen problems.

          (cherry picked from commit f98c4627da3c21e37aff82c75c0ef7240e60b4da)

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/src/main/java/hudson/ClassicPluginStrategy.java http://jenkins-ci.org/commit/jenkins/371826ac164cf4640ff20cec242c2efb79f5baaa Log: JENKINS-19383 Escape hatch to disable bytecode transformation in case this causes other unforeseen problems. (cherry picked from commit f98c4627da3c21e37aff82c75c0ef7240e60b4da)

          Code changed in jenkins
          User: Kohsuke Kawaguchi
          Path:
          core/pom.xml
          http://jenkins-ci.org/commit/jenkins/ded8e0a61028ad212e930a8fc2032de4b0b29319
          Log:
          [FIXED JENKINS-19383]

          Fixed a bug in bytecode-compatibility-transformer where class file
          format prior to 1.49 cannot have the ldc instruction pointing to a class
          constant.

          (cherry picked from commit 27d1a03866e468d2d9b630f636e53ec7804e4ade)

          Conflicts:
          changelog.html

          Compare: https://github.com/jenkinsci/jenkins/compare/4a6b9a18bcfd...ded8e0a61028

          SCM/JIRA link daemon added a comment - Code changed in jenkins User: Kohsuke Kawaguchi Path: core/pom.xml http://jenkins-ci.org/commit/jenkins/ded8e0a61028ad212e930a8fc2032de4b0b29319 Log: [FIXED JENKINS-19383] Fixed a bug in bytecode-compatibility-transformer where class file format prior to 1.49 cannot have the ldc instruction pointing to a class constant. (cherry picked from commit 27d1a03866e468d2d9b630f636e53ec7804e4ade) Conflicts: changelog.html Compare: https://github.com/jenkinsci/jenkins/compare/4a6b9a18bcfd...ded8e0a61028

          Any chance of this being backported to the LTS stream?

          It's currently broken on 1.509.4.

          Looks like the associated bytecode transformer logic was backported to 1.509.4, but this fix hasn't been backported yet.

          Timothy Bingaman added a comment - Any chance of this being backported to the LTS stream? It's currently broken on 1.509.4. Looks like the associated bytecode transformer logic was backported to 1.509.4, but this fix hasn't been backported yet.

          Daniel Beck added a comment -

          Timothy: Current LTS development is based on 1.532 and will include this fix.

          Given that it appears to be just a dependency update, you should be able to fork Jenkins, incorporate that change in the stable-1.509 branch and build your own "1.509.5" with minimal effort.

          Daniel Beck added a comment - Timothy: Current LTS development is based on 1.532 and will include this fix. Given that it appears to be just a dependency update, you should be able to fork Jenkins, incorporate that change in the stable-1.509 branch and build your own "1.509.5" with minimal effort.

          Jesse Glick added a comment -

          @tbingaman the 1.509.x LTS stream is closed; 1.532.1 LTS, which includes the fix, is in preparation.

          Jesse Glick added a comment - @tbingaman the 1.509.x LTS stream is closed; 1.532.1 LTS, which includes the fix, is in preparation.

          ah ok, sweet, no worries then.

          Using 1.509.3 at the moment without issues.

          Will jump on 1.532.1 when it arrives

          Timothy Bingaman added a comment - ah ok, sweet, no worries then. Using 1.509.3 at the moment without issues. Will jump on 1.532.1 when it arrives

            kohsuke Kohsuke Kawaguchi
            jvmccarthy John McCarthy
            Votes:
            5 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: