Ramon Leon added a comment - 2019-02-15 10:11 - edited I've compared the results of the Jenkins CI. One comparison on Jenkins 2.164 and latest ATH, and the other between Jenkins 2.164 and latest ATH and Jenkins 2.156 and ATH with params (latest at that moment).  All the info is in the comparison spreadsheet , for reference here: The comparison has been made between these two executions on Jenkins CI: https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/128/testReport/ Jenkins 2.164 (no need of arguments: enable..., modules), ATH without params: https://ci.jenkins.io/job/Core/job/acceptance-test-harness/job/master/124/testReport/ Jenkins 2.156 (need arguments), ATH with params: env JAVA_OPTS="-p /home/ath-user/jdk11-libs/jaxb-api.jar:/home/ath-user/jdk11-libs/javax.activation.jar --add-modules java.xml.bind,java.activation -cp /home/ath-user/jdk11-libs/jaxb-impl.jar:/home/ath-user/jdk11-libs/jaxb-core.jar" JENKINS_OPTS="--enable-future-java" ./run.sh firefox latest -DforkCount=1 -Dmaven.test.failure.ignore=true -B -Dtest=... The oldest execution throws 38 differences between Java8 and Java11. The latest execution throws 1 difference between Java8 and Java11. And are the same differences between Java11 in this execution and Java8 in the oldest execution. The difference is JobDslPluginTest.should_fail_on_missing_plugin "Expected: a string containing ""Warning: (script, line 3) plugin 'chucknorris' needs to be installed"" but: was ""Started by user unknown or anonymous Building in workspace /mnt/agent-workspace/workspace/e_acceptance-test-harness_master/target/jenkins11653942047609402639home/workspace/Seed Processing provided DSL script Warning: (NativeMethodAccessorImpl.java) plugin 'chucknorris' needs to be installed Added items: GeneratedJob{name='New_Job'} Build step 'Process Job DSLs' changed build result to UNSTABLE Finished: UNSTABLE""" Explanation : The test execution is the same on both environments, but the text printed on console is different and the Assert checking the output fails. Is there any difference on the libraries parsing the DSL between Java8 and Java11? Assert failing: https://github.com/jenkinsci/acceptance-test-harness/blob/4c989fb384929b4ff019195122966ef694fbabcf/src/test/java/plugins/JobDslPluginTest.java#L473

Ramon Leon added a comment - 2019-02-15 10:16 I don't understand the blocked link because we don't need the docker image to run the ATH with Jenkins 2.164 and to compare the results. If we need to check the results running on the docker image, then we need to have the new docker image. olivergondza oleg_nenashev

Oleg Nenashev added a comment - 2019-03-28 14:30 Closing that since Java 11 RC testing has been performed before the 2.164.1 release