Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-47105

CpsFlowExecutionTest#loaderReleased times out with jenkins-test-harness 2.25+

    XMLWordPrintable

    Details

    • Similar Issues:

      Description

      When moving to plugin parent 2.35, I discovered that CpsFlowExecutionTest#loaderReleased hangs - it loops forever with the following in the console:

      Skipped analysing class java.lang.reflect.Proxy because of java.lang.AssertionError
      [...]
      Skipped analysing class java.lang.reflect.Proxy because of java.lang.AssertionError
      Skipped analysing class hudson.model.Descriptor$PropertyType because of java.lang.AssertionError
      [...]
      Skipped analysing class hudson.model.Descriptor$PropertyType because of java.lang.AssertionError
      Failed analysing class org.kohsuke.stapler.JRebelFacet$ReloaderHook because of java.lang.NoClassDefFoundError: org/zeroturnaround/javarebel/ClassEventListener
      Failed to read field static final sun.awt.datatransfer.ToolkitThreadBlockedHandler sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher.handler because of java.lang.NoClassDefFoundError: Could not initialize class sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher
      Failed to read field public static final hudson.util.jna.GNUCLibrary hudson.util.jna.GNUCLibrary.LIBC because of java.lang.NoClassDefFoundError: Could not initialize class hudson.util.jna.GNUCLibrary
      Failed analysing class org.kohsuke.stapler.JRebelFacet$ReloaderHook because of java.lang.NoClassDefFoundError: org/zeroturnaround/javarebel/ClassEventListener
      Failed to read field private static final org.jvnet.solaris.libzfs.LibZFS hudson.os.solaris.ZFSProvisioner.libzfs because of java.lang.NoClassDefFoundError: Could not initialize class hudson.os.solaris.ZFSProvisioner
      Failed to read field public static final org.jvnet.solaris.libzfs.jna.libzfs org.jvnet.solaris.libzfs.jna.libzfs.LIBZFS because of java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.solaris.libzfs.jna.libzfs
      Failed to read field public static final java.lang.String org.jvnet.solaris.libzfs.jna.libzfs.ZFS_MOUNTPOINT_NONE because of java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.solaris.libzfs.jna.libzfs
      Failed to read field public static final java.lang.String org.jvnet.solaris.libzfs.jna.libzfs.ZFS_MOUNTPOINT_LEGACY because of java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.solaris.libzfs.jna.libzfs
      GC after allocation of size 2249
      Failed analysing class org.kohsuke.stapler.JRebelFacet$ReloaderHook because of java.lang.NoClassDefFoundError: org/zeroturnaround/javarebel/ClassEventListener
      Failed to read field static final sun.awt.datatransfer.ToolkitThreadBlockedHandler sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher.handler because of java.lang.NoClassDefFoundError: Could not initialize class sun.awt.dnd.SunDropTargetContextPeer$EventDispatcher
      Failed to read field public static final hudson.util.jna.GNUCLibrary hudson.util.jna.GNUCLibrary.LIBC because of java.lang.NoClassDefFoundError: Could not initialize class hudson.util.jna.GNUCLibrary
      Failed analysing class org.kohsuke.stapler.JRebelFacet$ReloaderHook because of java.lang.NoClassDefFoundError: org/zeroturnaround/javarebel/ClassEventListener
      Failed to read field private static final org.jvnet.solaris.libzfs.LibZFS hudson.os.solaris.ZFSProvisioner.libzfs because of java.lang.NoClassDefFoundError: Could not initialize class hudson.os.solaris.ZFSProvisioner
      Failed to read field public static final org.jvnet.solaris.libzfs.jna.libzfs org.jvnet.solaris.libzfs.jna.libzfs.LIBZFS because of java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.solaris.libzfs.jna.libzfs
      Failed to read field public static final java.lang.String org.jvnet.solaris.libzfs.jna.libzfs.ZFS_MOUNTPOINT_NONE because of java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.solaris.libzfs.jna.libzfs
      Failed to read field public static final java.lang.String org.jvnet.solaris.libzfs.jna.libzfs.ZFS_MOUNTPOINT_LEGACY because of java.lang.NoClassDefFoundError: Could not initialize class org.jvnet.solaris.libzfs.jna.libzfs
      

        Attachments

          Issue Links

            Activity

            abayer Andrew Bayer created issue -
            jglick Jesse Glick made changes -
            Field Original Value New Value
            Labels testing
            jglick Jesse Glick made changes -
            Assignee Oliver Gondža [ olivergondza ] Jesse Glick [ jglick ]
            Hide
            abayer Andrew Bayer added a comment - - edited

            So this bugger is a bit of a heisenbug. All these tests are coming from the command line, and with groovy-cps 1.19 (rather than the newest 1.20, since testing that is where I encountered this for the first time when I also bumped the plugin parent) fwiw. If you start from scratch, it consistently spits out that noise (and takes up to 75 seconds, vs the normal 10 or less seconds) with plugin parent 2.35 and jenkins-test-harness 2.27. If I run it with plugin parent 2.35/jenkins-test-harness 2.27 and then change jenkins-test-harness to 2.26, it spits out the output/takes longer again...on the first run. On the second run, or subsequent runs with that combination, it's fine. And what's more, changing jenkins-test-harness back to 2.27 after that, the first run is fine. Sometimes the second run is fine too! But eventually it goes back to noise/longer as well.

            With plugin parent 2.33, the behavior is similar, but the test actually ends up failing as well with jenkins-test-harness 2.27, with:

            [ERROR] Failures:
            [ERROR]   CpsFlowExecutionTest.loaderReleased Apparent soft references to org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c: {org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c=java.lang.Thread@48e20a51-group->
            java.lang.ThreadGroup@5b8c62df-threads->
            [Ljava.lang.Thread;@3194e300-[28]->
            java.lang.Thread@5f446887-threadLocals->
            java.lang.ThreadLocal$ThreadLocalMap@6c54de59-table->
            [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@77c7e4d-[26]->
            java.lang.ThreadLocal$ThreadLocalMap$Entry@443f4fd9-value->
            java.lang.ref.SoftReference@4119bed4-referent->
            org.codehaus.groovy.reflection.ClassInfo$LocalMap@bd22f03-cache->
            [Lorg.codehaus.groovy.reflection.ClassInfo;@5cef4aa1-[0]->
            org.codehaus.groovy.reflection.ClassInfo@24526e29-klazz->
            java.lang.Class@5f8ae2c5-<changed>->
            org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c}; apparent weak references: {org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c=private static final java.util.List org.jenkinsci.plugins.workflow.cps.CpsFlowExecutionTest.LOADERS->
            java.util.ArrayList@6cd63943-elementData->
            [Ljava.lang.Object;@19153301-[1]->
            java.lang.ref.WeakReference@73963a4b-referent->
            org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c}
            

            ...except it passes right away when I switch to jenkins-test-harness 2.26. So...yeah. No idea.

            EDIT: And now it started failing for me with plugin parent 2.33 and jenkins-test-harness 2.26 for no obvious reason.

            EDIT 2: And now it's failing with vanilla plugin parent 2.33 (i.e., with jenkins-test-harness 2.23)! What the hell...

            Show
            abayer Andrew Bayer added a comment - - edited So this bugger is a bit of a heisenbug. All these tests are coming from the command line, and with groovy-cps 1.19 (rather than the newest 1.20, since testing that is where I encountered this for the first time when I also bumped the plugin parent) fwiw. If you start from scratch, it consistently spits out that noise (and takes up to 75 seconds, vs the normal 10 or less seconds) with plugin parent 2.35 and jenkins-test-harness 2.27. If I run it with plugin parent 2.35/jenkins-test-harness 2.27 and then change jenkins-test-harness to 2.26, it spits out the output/takes longer again...on the first run. On the second run, or subsequent runs with that combination, it's fine. And what's more, changing jenkins-test-harness back to 2.27 after that, the first run is fine. Sometimes the second run is fine too! But eventually it goes back to noise/longer as well. With plugin parent 2.33, the behavior is similar, but the test actually ends up failing as well with jenkins-test-harness 2.27, with: [ERROR] Failures: [ERROR] CpsFlowExecutionTest.loaderReleased Apparent soft references to org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c: {org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c=java.lang. Thread @48e20a51-group-> java.lang. ThreadGroup @5b8c62df-threads-> [Ljava.lang. Thread ;@3194e300-[28]-> java.lang. Thread @5f446887-threadLocals-> java.lang.ThreadLocal$ThreadLocalMap@6c54de59-table-> [Ljava.lang.ThreadLocal$ThreadLocalMap$Entry;@77c7e4d-[26]-> java.lang.ThreadLocal$ThreadLocalMap$Entry@443f4fd9-value-> java.lang.ref.SoftReference@4119bed4-referent-> org.codehaus.groovy.reflection.ClassInfo$LocalMap@bd22f03-cache-> [Lorg.codehaus.groovy.reflection.ClassInfo;@5cef4aa1-[0]-> org.codehaus.groovy.reflection.ClassInfo@24526e29-klazz-> java.lang. Class @5f8ae2c5-<changed>-> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c}; apparent weak references: {org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c= private static final java.util.List org.jenkinsci.plugins.workflow.cps.CpsFlowExecutionTest.LOADERS-> java.util.ArrayList@6cd63943-elementData-> [Ljava.lang. Object ;@19153301-[1]-> java.lang.ref.WeakReference@73963a4b-referent-> org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$CleanGroovyClassLoader@25276a4c} ...except it passes right away when I switch to jenkins-test-harness 2.26. So...yeah. No idea. EDIT: And now it started failing for me with plugin parent 2.33 and jenkins-test-harness 2.26 for no obvious reason. EDIT 2: And now it's failing with vanilla plugin parent 2.33 (i.e., with jenkins-test-harness 2.23)! What the hell...
            abayer Andrew Bayer made changes -
            Environment workflow-cps-plugin 2.41-SNAPSHOT, jenkins-test-harness 2.27 workflow-cps-plugin 2.41-SNAPSHOT, jenkins-test-harness 2.27, JDK 1.8.0_131
            Hide
            abayer Andrew Bayer added a comment -

            For what it's worth, added to the environment that this is with JDK 1.8.0_131 on my Mac.

            Show
            abayer Andrew Bayer added a comment - For what it's worth, added to the environment that this is with JDK 1.8.0_131 on my Mac.
            jglick Jesse Glick made changes -
            Status Open [ 1 ] In Progress [ 3 ]
            Hide
            jglick Jesse Glick added a comment -

            I am not getting failures or even delayed collection on Linux with JDK 8u131 in current trunk after switching to parent 2.35. IIUC https://github.com/jenkinsci/workflow-cps-plugin/pull/177/commits/c482e6fd40210a3cc3bd32c328c565b98ed89561 is your workaround for now.

            Show
            jglick Jesse Glick added a comment - I am not getting failures or even delayed collection on Linux with JDK 8u131 in current trunk after switching to parent 2.35. IIUC https://github.com/jenkinsci/workflow-cps-plugin/pull/177/commits/c482e6fd40210a3cc3bd32c328c565b98ed89561  is your workaround for now.
            Hide
            abayer Andrew Bayer added a comment -

            Jesse Glick fwiw, try running it a few times. Like I said, it's bizarrely inconsistent in when it decides to start having issues. I'm trying concurrency experiments on a testbed I've got right now, since I know I'm pretty memory constrained on my laptop so that could be a trigger...

            ...and yeah, now I'm seeing some delayed collections with parent 2.35, though still not every time, and a run with parent 2.34 going at the same time is fine.

            Show
            abayer Andrew Bayer added a comment - Jesse Glick fwiw, try running it a few times. Like I said, it's bizarrely inconsistent in when it decides to start having issues. I'm trying concurrency experiments on a testbed I've got right now, since I know I'm pretty memory constrained on my laptop so that could be a trigger... ...and yeah, now I'm seeing some delayed collections with parent 2.35, though still not every time, and a run with parent 2.34 going at the same time is fine.
            Hide
            jglick Jesse Glick added a comment -

            Saw a failure when running the whole test suite, as opposed to the single test case. Right now I am trying to track down why LoggerRule is not working.

            Show
            jglick Jesse Glick added a comment - Saw a failure when running the whole test suite, as opposed to the single test case. Right now I am trying to track down why LoggerRule is not working.
            Hide
            jglick Jesse Glick added a comment -

            I think I have a lead: the test fails iff trustedShell is run first.

            Show
            jglick Jesse Glick added a comment - I think I have a lead: the test fails iff trustedShell is run first.
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jglick Jesse Glick made changes -
            Status In Review [ 10005 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 179 (Web Link)" [ 17775 ]
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            jglick Jesse Glick made changes -
            Status In Review [ 10005 ] In Progress [ 3 ]
            jglick Jesse Glick made changes -
            Remote Link This issue links to "PR 178 (Web Link)" [ 17777 ]
            jglick Jesse Glick made changes -
            Summary CpsFlowExecutionTest#loaderReleased times out with jenkins-test-harness 2.27 CpsFlowExecutionTest#loaderReleased times out with jenkins-test-harness 2.25+
            jglick Jesse Glick made changes -
            Status In Progress [ 3 ] In Review [ 10005 ]
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Jesse Glick
            Path:
            Jenkinsfile
            pom.xml
            http://jenkins-ci.org/commit/workflow-cps-plugin/942402131d84b961a268a2100a945b3897f121ff
            Log:
            JENKINS-47105 Checking effect of updated parent.

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Jesse Glick Path: Jenkinsfile pom.xml http://jenkins-ci.org/commit/workflow-cps-plugin/942402131d84b961a268a2100a945b3897f121ff Log: JENKINS-47105 Checking effect of updated parent.
            Hide
            scm_issue_link SCM/JIRA link daemon added a comment -

            Code changed in jenkins
            User: Sam Van Oort
            Path:
            pom.xml
            src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java
            src/test/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecutionMemoryTest.java
            src/test/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecutionTest.java
            http://jenkins-ci.org/commit/workflow-cps-plugin/bbf99da6739b37bbaaba21993f4b8a3604224969
            Log:
            Merge pull request #178 from jglick/parent-JENKINS-47105

            JENKINS-47105 Fixing test failure in loaderReleased

            Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/aa4c4e78ecf7...bbf99da6739b

            Show
            scm_issue_link SCM/JIRA link daemon added a comment - Code changed in jenkins User: Sam Van Oort Path: pom.xml src/main/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecution.java src/test/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecutionMemoryTest.java src/test/java/org/jenkinsci/plugins/workflow/cps/CpsFlowExecutionTest.java http://jenkins-ci.org/commit/workflow-cps-plugin/bbf99da6739b37bbaaba21993f4b8a3604224969 Log: Merge pull request #178 from jglick/parent- JENKINS-47105 JENKINS-47105 Fixing test failure in loaderReleased Compare: https://github.com/jenkinsci/workflow-cps-plugin/compare/aa4c4e78ecf7...bbf99da6739b
            Hide
            abayer Andrew Bayer added a comment -

            Jesse Glick So should this be resolved?

            Show
            abayer Andrew Bayer added a comment - Jesse Glick So should this be resolved?
            jglick Jesse Glick made changes -
            Resolution Fixed [ 1 ]
            Status In Review [ 10005 ] Resolved [ 5 ]

              People

              Assignee:
              jglick Jesse Glick
              Reporter:
              abayer Andrew Bayer
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: