Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-75339

Pipeline plugins upgrade from 2.2221 to 2.2234 breaks pipeline workflow

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • pipeline-model-definition-plugin, workflow-cps-plugin

      Reporting this as I expect failure like this could start affecting more users.

      And we are not sure yet how to solve this for our workflows.

      We do regular upgrades of jenkins plugins and versions and deploy, deploying with this version upgrade looks like it could break workflow in jenkins containers and we will have failures on every site, not sure if in addition every site would have a broken jenkins.

       

      Workflow run after Pipeline upgrades from 2.2221 to 2.2234 failed with java stack-dump in field groovy.lang.Closure.delegate, .. yadda ..,  in object of type org.jenkinsci.plugins.workflow.cps.CpsThreadGroup
      Caused: java.io.InvalidClassException: org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.AnyScript; Class does not extend stream superclass .. 

      The workflow failed after a 1 minute sleep  at end of a pipeline stage or at the start of a new pipeline stage. Apologies I am unfamiliar with digging in more to the java stack-dump, full dump below. 

      We run jenkins in a docker container and upgrades are done by stopping docker container and starting new with new image with updated scripts, jenkins plugins, etc ... The XXX-ku-upgrade.yml task does this (by spawning an upgrade container to manage switch-over).

       

      stage('ku upgrade') {
               steps {
                  script {
                      def inventory = readYaml file: 'inventory.yml'

                      if (inventory.all.children.ku.hosts.ku_container.latest_version == inventory.all.children.ku.hosts.ku_container.current_version)

      {                     echo 'No new version downloaded. Skip upgrade.'                     currentBuild.result = 'SUCCESS'                     return                 }

                      ansiblePlaybook inventory: 'inventory.d/', playbook: '/ansible_playbooks/kuku/XXX-ku-upgrade.yml'
                  }
                  sleep 60
               }
            }
            stage('Send deploy event') {
                steps {
                    sleep 60
                    script {
                          def inventory = readYaml file: 'inventory.yml'

                          if (inventory.all.children.ku.hosts.ku_container.latest_version == inventory.all.children.ku.hosts.ku_container.current_version)

      {                         echo 'No upgrade was done. Skip event.'                         currentBuild.result = 'SUCCESS'                         return                     }

                          timeout(time: 60, unit: 'MINUTES')

      {                         build job: "Event-job", propagate: false,                         parameters: [                               string(name: 'EVENT', value: "DeployedOK"),                               string(name: 'VERSION', value: inventory.all.children.ku.hosts.ku_container.latest_version),                               string(name: 'PARENT_JOB_NAME', value: "KU-workflow")                         ]                     }

                      }
                  }
              }

         }

       

      We can see warning about docker-workflow plugin (in release 2.2221) but we do not use that.

      "Warning: this is a breaking change for docker-workflow. Please update to https://github.com/jenkinsci/docker-workflow-plugin/releases/tag/599.v76126c79a_a_2d or newer.

      Also, Declarative builds running across the upgrade may fail. New builds should work."

       

      Workflow failure on upgrade, from workflow job/console:

      No debug.log produced

      [Pipeline] }
      [Pipeline] // script
      [Pipeline] sleep
      Sleeping for 1 min 0 sec
      an exception which occurred:
          in field groovy.lang.Closure.delegate
          in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@1c5abcac
          in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2
          in field groovy.lang.Closure.delegate
          in object org.jenkinsci.plugins.workflow.cps.CpsClosure2@3ed08b09
          in object of type org.jenkinsci.plugins.workflow.cps.CpsClosure2
          in object of type java.util.HashMap
          in field org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.closures
          in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@7726ffa3
          in object of type org.jenkinsci.plugins.workflow.cps.CpsThreadGroup
      Caused: java.io.InvalidClassException: org.jenkinsci.plugins.pipeline.modeldefinition.agent.impl.AnyScript; Class does not extend stream superclass
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadClassDescriptor(RiverUnmarshaller.java:1122)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1431)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:312)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:259)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1978)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1891)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1837)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1837)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1472)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:312)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:259)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1978)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1891)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1837)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1837)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1472)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:312)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:149)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:135)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.MarshallerObjectInputStream.readObjectOverride(MarshallerObjectInputStream.java:53)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverObjectInputStream.readObjectOverride(RiverObjectInputStream.java:307)
          at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:500)
          at java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:467)
          at java.base/java.util.HashMap.readObject(HashMap.java:1552)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.reflect.SerMethods.callReadObject(SerMethods.java:83)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.reflect.SerializableClass.callReadObject(SerializableClass.java:252)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1868)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1472)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:312)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:259)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.readFields(RiverUnmarshaller.java:1978)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doInitSerializable(RiverUnmarshaller.java:1891)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1472)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:312)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:244)
          at PluginClassLoader for workflow-support//org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:41)
          at PluginClassLoader for script-security//org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox.runInSandbox(GroovySandbox.java:331)
          at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.sandbox(RiverReader.java:250)
          at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverReader$SandboxedUnmarshaller.readObject(RiverReader.java:263)
          at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:879)
          at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution$2.onSuccess(CpsFlowExecution.java:872)
          at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.concurrent.Futures$1.run(Futures.java:147)
          at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.concurrent.DirectExecutor.execute(DirectExecutor.java:33)
          at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1307)
          at com.google.common.util.concurrent.AbstractFuture.addListener(AbstractFuture.java:772)
          at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.concurrent.Futures.addCallback(Futures.java:157)
          at PluginClassLoader for workflow-support//org.jenkinsci.plugins.workflow.support.concurrent.Futures.addCallback(Futures.java:97)
          at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.loadProgramAsync(CpsFlowExecution.java:869)
          at PluginClassLoader for workflow-cps//org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.onLoad(CpsFlowExecution.java:836)
          at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.getExecution(WorkflowRun.java:733)
          at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun.onLoad(WorkflowRun.java:597)
          at hudson.model.RunMap.retrieve(RunMap.java:273)
          at hudson.model.RunMap.retrieve(RunMap.java:65)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:703)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.load(AbstractLazyLoadRunMap.java:685)
          at jenkins.model.lazy.AbstractLazyLoadRunMap.getByNumber(AbstractLazyLoadRunMap.java:579)
          at hudson.model.RunMap.getById(RunMap.java:237)
          at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun$Owner.run(WorkflowRun.java:979)
          at PluginClassLoader for workflow-job//org.jenkinsci.plugins.workflow.job.WorkflowRun$Owner.get(WorkflowRun.java:991)
          at PluginClassLoader for workflow-api//org.jenkinsci.plugins.workflow.flow.FlowExecutionList$DefaultStorage.resume(FlowExecutionList.java:252)
          at PluginClassLoader for workflow-api//org.jenkinsci.plugins.workflow.flow.FlowExecutionList.resume(FlowExecutionList.java:127)
          at jenkins.security.ImpersonatingScheduledExecutorService$1.run(ImpersonatingScheduledExecutorService.java:67)
          at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
          at java.base/java.lang.Thread.run(Thread.java:840)
      ERROR: Failed to load program
      Finished: FAILURE

        1. run-jenkins.sh
          0.9 kB
        2. plugins.txt
          4 kB

          [JENKINS-75339] Pipeline plugins upgrade from 2.2221 to 2.2234 breaks pipeline workflow

            Unassigned Unassigned
            jamesc0 James Coleman
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: