Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55146

PCT sometimes crashes on Java 11 due to module read errors when using the docker image

XMLWordPrintable

      https://issues.apache.org/jira/browse/SUREFIRE-1496

       

      Command: // custom image is used for error propagation

      docker run --rm -v maven-repo:/root/.m2 -v $(pwd)/out:/pct/out -e ARTIFACT_ID=ssh-slaves -e VERSION=ssh-slaves-1.24 -e JDK_VERSION=11 onenashev/pct

       

      Output:

       

      + echo 'ERROR: PCT failed with code 1. Will check for Maven Surefire dumps if it crashed' ERROR: PCT failed with code 1. Will check for Maven Surefire dumps if it crashed + read file + find /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream Found Maven Surefire dump file: /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream + echo 'Found Maven Surefire dump file: /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream' + cat /pct/tmp/work/ssh-slaves/target/surefire-reports/2018-12-12T11-08-54_738-jvmRun1.dumpstream # Created on 2018-12-12T11:08:55.538 Corrupted stdin stream in forked JVM 1. Stream 'Error occurred during initialization of boot layer'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'Error occurred during initialization of boot layer'. at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:469) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:191) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:158) at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:87) at java.lang.Thread.run(Thread.java:748) # Created on 2018-12-12T11:08:55.541 Corrupted stdin stream in forked JVM 1. Stream 'java.lang.module.FindException: Error reading /pct/jdk11-libs/javax.activation.jar'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'java.lang.module.FindException: Error reading /pct/jdk11-libs/javax.activation.jar'. at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:469) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:191) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:158) at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:87) at java.lang.Thread.run(Thread.java:748) # Created on 2018-12-12T11:08:55.542 Corrupted stdin stream in forked JVM 1. Stream 'Caused by: java.util.zip.ZipException: zip END header not found'. java.lang.IllegalArgumentException: Stream stdin corrupted. Expected comma after third character in command 'Caused by: java.util.zip.ZipException: zip END header not found'. at org.apache.maven.plugin.surefire.booterclient.output.ForkClient$OperationalData.<init>(ForkClient.java:469) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.processLine(ForkClient.java:191) at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.consumeLine(ForkClient.java:158) at org.apache.maven.plugin.surefire.booterclient.output.ThreadedStreamConsumer$Pumper.run(ThreadedStreamConsumer.java:87) at java.lang.Thread.run(Thread.java:748)

       

      Acceptance criteria

      • Figure out if it's failing under other conditions
      • PCT runs with Java 11 and docker image

          [JENKINS-55146] PCT sometimes crashes on Java 11 due to module read errors when using the docker image

          Oleg Nenashev added a comment -

          According to https://github.com/junit-team/junit5/issues/855 , this is actually a valid issue in Maven Surefire Plugin which is fixed in 2.21: https://github.com/apache/maven-surefire/commit/7176d3c17b438c60e48a525ef96e6812f00bec46 . Please correct me if I am wrong tibor17

           

          We cannot easily update to 3.0.0-M1 (see https://github.com/jenkinsci/plugin-pom/pull/131 from jglick), but probably we could pick up the 2.21 patch in Plugin POM. CC batmat

          Oleg Nenashev added a comment - According to https://github.com/junit-team/junit5/issues/855  , this is actually a valid issue in Maven Surefire Plugin which is fixed in 2.21: https://github.com/apache/maven-surefire/commit/7176d3c17b438c60e48a525ef96e6812f00bec46  . Please correct me if I am wrong tibor17   We cannot easily update to 3.0.0-M1 (see https://github.com/jenkinsci/plugin-pom/pull/131  from jglick ), but probably we could pick up the 2.21 patch in Plugin POM. CC batmat

          Tibor Digana added a comment -

          oleg_nenashev
          For the external users it must be very uncertain to read if we mix unrelated issues.
          Yes, SUREFIRE-855 was fixed in 2.21.0.
          The another issue https://issues.apache.org/jira/browse/SUREFIRE-1496 is about Jigsaw modularity, see my comment in SUREFIRE-1496.
          And third unrelated cause error is in your description in this issue:
          Caused by: java.util.zip.ZipException: zip END header not found
          The Surefire is not packaging classes in JAR file. It is the responsibility of maven-jar-plugin.
          Please see the release 3.1.1 of JAR Plugin we did yesterday, I guess.
          https://issues.apache.org/jira/browse/MJAR-241
          After seen deeper, the path is pointing to JDK, this means the file /pct/jdk11-libs/javax.activation.jar and I would say that Oracle or OpenJDK has to be asked.

          Check it out again with new version of JAR Plugin and possibly Surefire plugin 3.0.0-M2, but you may try to also switch to another JDK to come over the ZIP error.

          Tibor Digana added a comment - oleg_nenashev For the external users it must be very uncertain to read if we mix unrelated issues. Yes, SUREFIRE-855 was fixed in 2.21.0 . The another issue https://issues.apache.org/jira/browse/SUREFIRE-1496 is about Jigsaw modularity, see my comment in SUREFIRE-1496. And third unrelated cause error is in your description in this issue: Caused by: java.util.zip.ZipException: zip END header not found The Surefire is not packaging classes in JAR file. It is the responsibility of maven-jar-plugin . Please see the release 3.1.1 of JAR Plugin we did yesterday, I guess. https://issues.apache.org/jira/browse/MJAR-241 After seen deeper, the path is pointing to JDK, this means the file /pct/jdk11-libs/javax.activation.jar and I would say that Oracle or OpenJDK has to be asked. Check it out again with new version of JAR Plugin and possibly Surefire plugin 3.0.0-M2, but you may try to also switch to another JDK to come over the ZIP error.

          Oleg Nenashev added a comment -

          tibor17 thanks for the clarification!

          Oleg Nenashev added a comment - tibor17 thanks for the clarification!

          Oleg Nenashev added a comment - - edited

          Apparently I cannot reproduce the issue outside Docker, so probably it's another flavor of SUREFIRE-1588 fun

          Oleg Nenashev added a comment - - edited java.activation error is gone after updating to the recent release of javarta.aactivation: https://repo1.maven.org/maven2/com/sun/activation/jakarta.activation/1.2.1/  . This is good, but we will need to update it everywhere in Docker packaging I'd guess Maven JAR Plugin surely needs to be updated. https://github.com/jenkinsci/plugin-pom/blob/1f78cd35f318480942f82af836ed31ab98b27a87/pom.xml#L318  is where it is defined for Jenkins plugins. Likely won't help in the case of our tests taking issues referenced in https://github.com/jenkinsci/remoting/pull/315  and other core tests by jglick   Apparently I cannot reproduce the issue outside Docker, so probably it's another flavor of SUREFIRE-1588 fun

          Oleg Nenashev added a comment -

          Also CC batmat

          Oleg Nenashev added a comment - Also CC batmat

          Oleg Nenashev added a comment -

          https://github.com/jenkinsci/plugin-compat-tester/pull/97/commits/1299954b4f83f72409d5ad1a4db6efd01c2fbbef helps for the java.activation issue, but I really really doubt we want to update production packages for now. Will create follow-ups

          Oleg Nenashev added a comment - https://github.com/jenkinsci/plugin-compat-tester/pull/97/commits/1299954b4f83f72409d5ad1a4db6efd01c2fbbef  helps for the java.activation issue, but I really really doubt we want to update production packages for now. Will create follow-ups

          Oleg Nenashev added a comment -

          I had to revert the patch. It would be applicable until the Jakarta Team fixes https://github.com/eclipse-ee4j/jaf/issues/13 and https://github.com/eclipse-ee4j/jaxb-ri/issues/1236 

          Unassigning it for now so that others from Java 11 support team can pick it up

           

          Oleg Nenashev added a comment - I had to revert the patch. It would be applicable until the Jakarta Team fixes https://github.com/eclipse-ee4j/jaf/issues/13  and https://github.com/eclipse-ee4j/jaxb-ri/issues/1236   Unassigning it for now so that others from Java 11 support team can pick it up  

          Oleg Nenashev added a comment -

          Summarized the result in https://github.com/eclipse-ee4j/jaf/issues/13 . One of possible solutions would be to update the entire PCT to run with Java 11 (not tested, but it apparently does not work according to kearls3).

          Oleg Nenashev added a comment - Summarized the result in https://github.com/eclipse-ee4j/jaf/issues/13  . One of possible solutions would be to update the entire PCT to run with Java 11 (not tested, but it apparently does not work according to kearls3 ).

          Oleg Nenashev added a comment -

          I updated PCT to support running the entire codebase with JDK 11 in JENKINS-55296 . It does not resolve the issue. So we will need something better to run PCT in Docker

          Oleg Nenashev added a comment - I updated PCT to support running the entire codebase with JDK 11 in  JENKINS-55296  . It does not resolve the issue. So we will need something better to run PCT in Docker

          Baptiste Mathus added a comment -

          Looks like https://issues.apache.org/jira/browse/SUREFIRE-1601

          Baptiste Mathus added a comment - Looks like https://issues.apache.org/jira/browse/SUREFIRE-1601

          Tibor Digana added a comment - - edited

          We will use tcp/ip sockets in Surefire/Failsafe (3.0.0-M4 or M5) instead of interprocess pipes which is nowadays sollution for interprocess communication.

          Tibor Digana added a comment - - edited We will use tcp/ip sockets in Surefire/Failsafe (3.0.0-M4 or M5) instead of interprocess pipes which is nowadays sollution for interprocess communication.

          Oleg Nenashev added a comment -

          It was actually caused by a glitch in the Docker packaging. And it was fixed by alecharp in https://github.com/jenkinsci/plugin-compat-tester/pull/111 . I still confirm there is similar behavior outside Docker rarely, but I believe it is related to another issue

          Oleg Nenashev added a comment - It was actually caused by a glitch in the Docker packaging. And it was fixed by alecharp in https://github.com/jenkinsci/plugin-compat-tester/pull/111  . I still confirm there is similar behavior outside Docker rarely, but I believe it is related to another issue

          Tibor Digana added a comment - - edited

          oleg_nenashev

          Regarding the Alpine/BusyBox we made three fixes

          https://issues.apache.org/jira/browse/SUREFIRE-1702

          https://issues.apache.org/jira/browse/SUREFIRE-1703

          https://issues.apache.org/jira/browse/SUREFIRE-1704

          and we are finishing the last few issues to cut a new release version 3.0.0-M4.

          Tibor Digana added a comment - - edited oleg_nenashev Regarding the Alpine/BusyBox we made three fixes https://issues.apache.org/jira/browse/SUREFIRE-1702 https://issues.apache.org/jira/browse/SUREFIRE-1703 https://issues.apache.org/jira/browse/SUREFIRE-1704 and we are finishing the last few issues to cut a new release version 3.0.0-M4.

            alecharp Adrien Lecharpentier
            oleg_nenashev Oleg Nenashev
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: