Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55146

PCT sometimes crashes on Java 11 due to module read errors when using the docker image

    XMLWordPrintable

Details

    Description

      https://issues.apache.org/jira/browse/SUREFIRE-1496

       

      Command: // custom image is used for error propagation

      docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -e ARTIFACT_ID=ssh-slaves -e VERSION=ssh-slaves-1.24 -e JDK_VERSION=11 onenashev/pct

       

      Output:

       

      + echo 'ERROR: PCT failed with code 1. Will check for Maven Surefire dumps if it crashed' ERROR: PCT failed with code 1. Will check for Maven Surefire dumps if it crashed + read file + find /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream Found Maven Surefire dump file: /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream + echo 'Found Maven Surefire dump file: /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream' + cat /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream # Created on 2018-12-12T11:08:55.538 Corrupted stdin stream in forked JVM 1. Stream 'Error occurred during initialization of boot layer'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'Error occurred during initialization of boot layer'. at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:469) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:191) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:158) at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:87) at java.lang.Thread.run(Thread.java:748) # Created on 2018-12-12T11:08:55.541 Corrupted stdin stream in forked JVM 1. Stream 'java.lang.module.FindException: Error reading /pct/jdk11-libs/javax.activation.jar'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'java.lang.module.FindException: Error reading /pct/jdk11-libs/javax.activation.jar'. at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:469) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:191) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:158) at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:87) at java.lang.Thread.run(Thread.java:748) # Created on 2018-12-12T11:08:55.542 Corrupted stdin stream in forked JVM 1. Stream 'Caused by: java.util.zip.ZipException: zip END header not found'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'Caused by: java.util.zip.ZipException: zip END header not found'. at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:469) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:191) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:158) at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:87) at java.lang.Thread.run(Thread.java:748)

       

      Acceptance criteria

      • Figure out if it's failing under other conditions
      • PCT runs with Java 11 and docker image

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. JENKINS-55296 Add support of running PCT main executable with Java 11

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-55147 PCT Docker Image should analyze Java crashes in Maven Surefire

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. JENKINS-55162 Maven JAR Plugin should be updated to 3.1.1 in Plugin POM and Jenkins POM

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            Ascending order - Click to sort in descending order
            oleg_nenashev Oleg Nenashev added a comment -

            According to https://github.com/junit-team/junit5/issues/855 , this is actually a valid issue in Maven Surefire Plugin which is fixed in 2.21: https://github.com/apache/maven-surefire/commit/7176d3c17b438c60e48a525ef96e6812f00bec46 . Please correct me if I am wrong tibor17

             

            We cannot easily update to 3.0.0-M1 (see https://github.com/jenkinsci/plugin-pom/pull/131 from jglick), but probably we could pick up the 2.21 patch in Plugin POM. CC batmat

            oleg_nenashev Oleg Nenashev added a comment - According to https://github.com/junit-team/junit5/issues/855  , this is actually a valid issue in Maven Surefire Plugin which is fixed in 2.21: https://github.com/apache/maven-surefire/commit/7176d3c17b438c60e48a525ef96e6812f00bec46  . Please correct me if I am wrong tibor17   We cannot easily update to 3.0.0-M1 (see https://github.com/jenkinsci/plugin-pom/pull/131  from jglick ), but probably we could pick up the 2.21 patch in Plugin POM. CC batmat
            tibor17 Tibor Digana added a comment -

            oleg_nenashev
            For the external users it must be very uncertain to read if we mix unrelated issues.
            Yes, SUREFIRE-855 was fixed in 2.21.0.
            The another issue https://issues.apache.org/jira/browse/SUREFIRE-1496 is about Jigsaw modularity, see my comment in SUREFIRE-1496.
            And third unrelated cause error is in your description in this issue:
            Caused by: java.util.zip.ZipException: zip END header not found
            The Surefire is not packaging classes in JAR file. It is the responsibility of maven-jar-plugin.
            Please see the release 3.1.1 of JAR Plugin we did yesterday, I guess.
            https://issues.apache.org/jira/browse/MJAR-241
            After seen deeper, the path is pointing to JDK, this means the file /pct/jdk11-libs/javax.activation.jar and I would say that Oracle or OpenJDK has to be asked.

            Check it out again with new version of JAR Plugin and possibly Surefire plugin 3.0.0-M2, but you may try to also switch to another JDK to come over the ZIP error.

            tibor17 Tibor Digana added a comment - oleg_nenashev For the external users it must be very uncertain to read if we mix unrelated issues. Yes, SUREFIRE-855 was fixed in 2.21.0 . The another issue https://issues.apache.org/jira/browse/SUREFIRE-1496 is about Jigsaw modularity, see my comment in SUREFIRE-1496. And third unrelated cause error is in your description in this issue: Caused by: java.util.zip.ZipException: zip END header not found The Surefire is not packaging classes in JAR file. It is the responsibility of maven-jar-plugin . Please see the release 3.1.1 of JAR Plugin we did yesterday, I guess. https://issues.apache.org/jira/browse/MJAR-241 After seen deeper, the path is pointing to JDK, this means the file /pct/jdk11-libs/javax.activation.jar and I would say that Oracle or OpenJDK has to be asked. Check it out again with new version of JAR Plugin and possibly Surefire plugin 3.0.0-M2, but you may try to also switch to another JDK to come over the ZIP error.
            oleg_nenashev Oleg Nenashev added a comment -

            tibor17 thanks for the clarification!

            oleg_nenashev Oleg Nenashev added a comment - tibor17 thanks for the clarification!
            oleg_nenashev Oleg Nenashev added a comment - - edited

            Apparently I cannot reproduce the issue outside Docker, so probably it's another flavor of SUREFIRE-1588 fun

            oleg_nenashev Oleg Nenashev added a comment - - edited java.activation error is gone after updating to the recent release of javarta.aactivation: https://repo1.maven.org/maven2/com/sun/activation/jakarta.activation/1.2.1/  . This is good, but we will need to update it everywhere in Docker packaging I'd guess Maven JAR Plugin surely needs to be updated. https://github.com/jenkinsci/plugin-pom/blob/1f78cd35f318480942f82af836ed31ab98b27a87/pom.xml#L318  is where it is defined for Jenkins plugins. Likely won't help in the case of our tests taking issues referenced in https://github.com/jenkinsci/remoting/pull/315  and other core tests by jglick   Apparently I cannot reproduce the issue outside Docker, so probably it's another flavor of SUREFIRE-1588 fun
            oleg_nenashev Oleg Nenashev added a comment -

            Also CC batmat

            oleg_nenashev Oleg Nenashev added a comment - Also CC batmat
            oleg_nenashev Oleg Nenashev added a comment -

            https://github.com/jenkinsci/plugin-compat-tester/pull/97/commits/1299954b4f83f72409d5ad1a4db6efd01c2fbbef helps for the java.activation issue, but I really really doubt we want to update production packages for now. Will create follow-ups

            oleg_nenashev Oleg Nenashev added a comment - https://github.com/jenkinsci/plugin-compat-tester/pull/97/commits/1299954b4f83f72409d5ad1a4db6efd01c2fbbef  helps for the java.activation issue, but I really really doubt we want to update production packages for now. Will create follow-ups
            oleg_nenashev Oleg Nenashev added a comment -

            I had to revert the patch. It would be applicable until the Jakarta Team fixes https://github.com/eclipse-ee4j/jaf/issues/13 and https://github.com/eclipse-ee4j/jaxb-ri/issues/1236 

            Unassigning it for now so that others from Java 11 support team can pick it up

             

            oleg_nenashev Oleg Nenashev added a comment - I had to revert the patch. It would be applicable until the Jakarta Team fixes https://github.com/eclipse-ee4j/jaf/issues/13  and https://github.com/eclipse-ee4j/jaxb-ri/issues/1236   Unassigning it for now so that others from Java 11 support team can pick it up  
            oleg_nenashev Oleg Nenashev added a comment -

            Summarized the result in https://github.com/eclipse-ee4j/jaf/issues/13 . One of possible solutions would be to update the entire PCT to run with Java 11 (not tested, but it apparently does not work according to kearls3).

            oleg_nenashev Oleg Nenashev added a comment - Summarized the result in https://github.com/eclipse-ee4j/jaf/issues/13  . One of possible solutions would be to update the entire PCT to run with Java 11 (not tested, but it apparently does not work according to kearls3 ).
            oleg_nenashev Oleg Nenashev added a comment -

            I updated PCT to support running the entire codebase with JDK 11 in JENKINS-55296 . It does not resolve the issue. So we will need something better to run PCT in Docker

            oleg_nenashev Oleg Nenashev added a comment - I updated PCT to support running the entire codebase with JDK 11 in  JENKINS-55296  . It does not resolve the issue. So we will need something better to run PCT in Docker
            batmat Baptiste Mathus added a comment -

            Looks like https://issues.apache.org/jira/browse/SUREFIRE-1601

            batmat Baptiste Mathus added a comment - Looks like https://issues.apache.org/jira/browse/SUREFIRE-1601
            tibor17 Tibor Digana added a comment - - edited

            We will use tcp/ip sockets in Surefire/Failsafe (3.0.0-M4 or M5) instead of interprocess pipes which is nowadays sollution for interprocess communication.

            tibor17 Tibor Digana added a comment - - edited We will use tcp/ip sockets in Surefire/Failsafe (3.0.0-M4 or M5) instead of interprocess pipes which is nowadays sollution for interprocess communication.
            oleg_nenashev Oleg Nenashev added a comment -

            It was actually caused by a glitch in the Docker packaging. And it was fixed by alecharp in https://github.com/jenkinsci/plugin-compat-tester/pull/111 . I still confirm there is similar behavior outside Docker rarely, but I believe it is related to another issue

            oleg_nenashev Oleg Nenashev added a comment - It was actually caused by a glitch in the Docker packaging. And it was fixed by alecharp in https://github.com/jenkinsci/plugin-compat-tester/pull/111  . I still confirm there is similar behavior outside Docker rarely, but I believe it is related to another issue
            tibor17 Tibor Digana added a comment - - edited

            oleg_nenashev

            Regarding the Alpine/BusyBox we made three fixes

            https://issues.apache.org/jira/browse/SUREFIRE-1702

            https://issues.apache.org/jira/browse/SUREFIRE-1703

            https://issues.apache.org/jira/browse/SUREFIRE-1704

            and we are finishing the last few issues to cut a new release version 3.0.0-M4.

            tibor17 Tibor Digana added a comment - - edited oleg_nenashev Regarding the Alpine/BusyBox we made three fixes https://issues.apache.org/jira/browse/SUREFIRE-1702 https://issues.apache.org/jira/browse/SUREFIRE-1703 https://issues.apache.org/jira/browse/SUREFIRE-1704 and we are finishing the last few issues to cut a new release version 3.0.0-M4.

            People

              alecharp Adrien Lecharpentier
              oleg_nenashev Oleg Nenashev
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: