Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-55356

Some workflow jobs fail after restart on Java 11 server

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • workflow-job-plugin
    • Jenkins JDK 11 docker image and current plugins
      Multibranch Pipeline builds for git plugin, git client plugin, and platformlabeler plugin

      While running Java 11 based Jenkins in a docker container using a pre-release of the workflow support plugin which includes the fix for the null pointer exception, a

      jenkins-url/safeRestart
      

      will cause several of the Pipeline jobs that were running to fail when Jenkins tries to resume the jobs.

      Build log output from the failed builds has included messages like (seems to be more common in windows, but visible in Linux agents as well):

      07:40:50 [INFO] Running org.jenkinsci.plugins.gitclient.PushTest
      Resuming build at Fri Dec 28 07:48:06 MST 2018 after Jenkins restart
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #6: Waiting for next available executor on ‘coleen-pc2-ssh’
      Ready to run at Fri Dec 28 07:48:49 MST 2018
      07:48:49 Timeout set to expire in 17 min
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] }
      07:48:50 Failed in branch windows-8-2.150.1
      [Pipeline] // parallel
      [Pipeline] }
      [Pipeline] // timestamps
      [Pipeline] End of Pipeline
      ERROR: missing workspace C:\J\S\workspace\der_git-client-pipeline_beta-3.0 on coleen-pc2-ssh
      

      and this (seems to fail on Windows and on Linux):

      08:48:33 [INFO] --------------------------------[ hpi ]---------------------------------
      Resuming build at Fri Dec 28 08:51:46 MST 2018 after Jenkins restart
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #7: Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #7: ???
      ???
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #7: ???
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #7: ???
      Ready to run at Fri Dec 28 08:52:07 MST 2018
      08:52:07 Timeout set to expire in 54 min
      08:52:07 Timeout set to expire in 54 min
      08:52:07 Timeout set to expire in 54 min
      08:52:07 Timeout set to expire in 54 min
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] // timeout
      [Pipeline] }
      08:52:08 Failed in branch windows-8
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] }
      08:52:08 Failed in branch windows-8-2.150.1
      08:57:14 process apparently never started in /home/mwaite/testing-a.markwaite.net-agent/workspace/der_git-client-pipeline_beta-3.0@tmp/durable-3af9d572
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      08:57:14 process apparently never started in /home/mwaite/testing-a.markwaite.net-agent/workspace/der_git-client-pipeline_beta-3.0@tmp/durable-1ba13955
      [Pipeline] // node
      [Pipeline] }
      08:57:14 Failed in branch linux-8
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] }
      08:57:14 Failed in branch linux-8-2.150.1
      [Pipeline] // parallel
      [Pipeline] }
      [Pipeline] // timestamps
      [Pipeline] End of Pipeline
      ERROR: missing workspace C:\J\S\workspace\der_git-client-pipeline_beta-3.0 on coleen-pc2-ssh
      

      and this (windows and linux):

      09:12:13 [INFO] --- maven-help-plugin:3.1.1:evaluate (default-cli) @ git-client ---
      Resuming build at Fri Dec 28 09:13:35 MST 2018 after Jenkins restart
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #8: ???
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #8: ???
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #8: ???
      Waiting to resume part of Git Client Plugin Folder » Git Client Branches - Jenkinsfile » beta-3.0 #8: ???
      Ready to run at Fri Dec 28 09:13:51 MST 2018
      09:13:51 Timeout set to expire in 55 min
      09:13:51 Timeout set to expire in 55 min
      09:13:51 Timeout set to expire in 55 min
      09:13:51 Timeout set to expire in 55 min
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] // withEnv
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] // stage
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] // node
      [Pipeline] // timeout
      [Pipeline] }
      09:13:52 Failed in branch windows-8-2.150.1
      [Pipeline] }
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] // node
      [Pipeline] }
      09:13:52 Failed in branch linux-8
      [Pipeline] }
      09:13:52 Failed in branch windows-8
      09:18:58 process apparently never started in /home/mwaite/testing-a.markwaite.net-agent/workspace/r_git-client-pipeline_beta-3.0_2@tmp/durable-9d0a69fe
      [Pipeline] }
      [Pipeline] // withEnv
      [Pipeline] }
      [Pipeline] // stage
      [Pipeline] }
      [Pipeline] // timeout
      [Pipeline] }
      [Pipeline] // node
      [Pipeline] }
      09:18:58 Failed in branch linux-8-2.150.1
      [Pipeline] // parallel
      [Pipeline] }
      [Pipeline] // timestamps
      [Pipeline] End of Pipeline
      ERROR: missing workspace C:\J\S\workspace\der_git-client-pipeline_beta-3.0 on mark-pc4-ssh
      

      The process never started message and the missing workspace message are visible in both the failed git client plugin builds and in the failed git plugin builds.

      The problem does not seem to repeat on a Java 8 environment, just on a Java 11 environment.
      The problem does seem to repeat less frequently on a Java 11 environment running on a larger computer. The failing computer has 8 GB RAM with an older Intel i5 processor, while the less frequently failing computer has 32 GB RAM and a newer Intel i5 processor. The 32 GB machine has shown the failure multiple times as well as the smaller computer. That failure was during a restart while the agents and the server were very busy.

      The Docker image includes all the plugins that were used in the failure case. I've dupilicated the failures on at least two different machines.

            dnusbaum Devin Nusbaum
            markewaite Mark Waite
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: