Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-53888

Batch step running on a node other than the master fails

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • Windows Server 2012 R2 (both master and agent)
      Jenkins 2.138.1
      Pipeline 2.6
      Durable task plugin 1.26
      Pipeline Nodes and Processes 2.22

      Running a batch command on another node takes several minutes and then fails. Attached example (all Windows, the echo command won't print):

      stage('1') {
        node('uitest') {
          bat 'echo something'
        }
      }
      

      After 10 minutes the console output prompts this and the build fails:

      ERROR: script apparently exited with code 0 but asynchronous notification was lost

      In addition the system log gets these exceptions:

      • java.lang.NoClassDefFoundError: Could not initialize class hudson.slaves.SlaveComputer
      • hudson.remoting.Channel$CallSiteStackTrace: Remote call to JNLP4-connect connection from

          [JENKINS-53888] Batch step running on a node other than the master fails

          Bhushan Shah added a comment -

          For me this bug happens also with the pipeline running on linux node and with sh instead of bat.

          Bhushan Shah added a comment - For me this bug happens also with the pipeline running on linux node and with sh instead of bat.

          Jesse Glick added a comment -

          For what it is worth, I am unable to reproduce any such issue with a Windows 10 agent running a simple bat script. Possibly there are special conditions triggering it.

          Jesse Glick added a comment - For what it is worth, I am unable to reproduce any such issue with a Windows 10 agent running a simple bat script. Possibly there are special conditions triggering it.

          Jesse Glick added a comment -

          Anyone who is encountering the asynchronous notification was lost error: assuming you do not know how to reproduce the issue from scratch, please create a custom logger tracking org.jenkinsci.plugins.workflow.steps.durable_task and org.jenkinsci.plugins.durabletask at FINE and report details. Installing the support-core plugin is ideal as it allows these logs and other things to be recorded as a single ZIP file.

          Jesse Glick added a comment - Anyone who is encountering the asynchronous notification was lost error: assuming you do not know how to reproduce the issue from scratch, please create a custom logger tracking org.jenkinsci.plugins.workflow.steps.durable_task and org.jenkinsci.plugins.durabletask at FINE and report details. Installing the support-core plugin is ideal as it allows these logs and other things to be recorded as a single ZIP file.

          Jesse Glick added a comment -

          Also be sure to pick up the workflow-api 2.31 release with the purported fix of JENKINS-54073.

          Note that workflow-durable-task-step 2.25 has disabled watch mode by default, so if you have accepted that update and wish to help test this, please run with: -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=true

          Jesse Glick added a comment - Also be sure to pick up the workflow-api 2.31 release with the purported fix of JENKINS-54073 . Note that workflow-durable-task-step 2.25 has disabled watch mode by default, so if you have accepted that update and wish to help test this, please run with: -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=true

          About worklow-api (Pipeline API plugin) I was still in 2.30 when this problem occured. For now, I can plan to upgrade to last plugins version and re-enable watch mode next week.

          About "special conditions triggering it", in my case it's huge logs files (example in this report was working perfectly). See : https://issues.jenkins-ci.org/browse/JENKINS-54081?focusedCommentId=352021&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-352021

          Florian Ramillien added a comment - About worklow-api (Pipeline API plugin) I was still in 2.30 when this problem occured. For now, I can plan to upgrade to last plugins version and re-enable watch mode next week. About "special conditions triggering it", in my case it's huge logs files (example in this report was working perfectly). See : https://issues.jenkins-ci.org/browse/JENKINS-54081?focusedCommentId=352021&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-352021

          Jesse Glick added a comment -

          framillien

          in my case it's huge logs files

          Then you may rather have hit a symptom of JENKINS-54073, whereas other reporters (bshah, ericlouvard, shahmishal, but again excluding the false initial report by towel) seem to have hit something very different.

          Jesse Glick added a comment - framillien in my case it's huge logs files Then you may rather have hit a symptom of JENKINS-54073 , whereas other reporters ( bshah , ericlouvard , shahmishal , but again excluding the false initial report by towel ) seem to have hit something very different.

          Jesse Glick added a comment -

          Anyone still seeing this using up-to-date plugins and -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=true? If so, please make sure you have custom loggers set up as in my comment of 2018-10-25 and see if the problem can be reproduced in a clean environment.

          Jesse Glick added a comment - Anyone still seeing this using up-to-date plugins and -Dorg.jenkinsci.plugins.workflow.steps.durable_task.DurableTaskStep.USE_WATCHING=true ? If so, please make sure you have custom loggers set up as in my comment of 2018-10-25 and see if the problem can be reproduced in a clean environment.

          Shriram Datar added a comment -

          I am seeing the same issue with the latest Jenkins+ all up to date plugins. I am running only echo 1234 in the batch file. I have exact same logs as attached to this ticket

           

          Shriram Datar added a comment - I am seeing the same issue with the latest Jenkins+ all up to date plugins. I am running only echo 1234 in the batch file. I have exact same logs as attached to this ticket  

          milo6 added a comment -

          I change from 32 bit to 64 bit Java and Increased JVM heap size to 4GB seems to be working . 

          milo6 added a comment - I change from 32 bit to 64 bit Java and Increased JVM heap size to 4GB seems to be working . 

          Jesse Glick added a comment -

          shriramd if you are seeing the java.lang.NoClassDefFoundError: Could not initialize class hudson.slaves.SlaveComputer then, like the original reporter, this suggests that you were using an incompatible JRE for the agent—a problem which the switch to watching mode might have incidentally triggered, but not really caused. amol_malokar’s issue sounds similar—possibly the use of 32-bit Java led to some incompatibility with JNA that would up crashing a lot of class loading. Hard to know without being able to reproduce from scratch, including details of the Java installation packages used.

          Jesse Glick added a comment - shriramd if you are seeing the java.lang.NoClassDefFoundError: Could not initialize class hudson.slaves.SlaveComputer then, like the original reporter, this suggests that you were using an incompatible JRE for the agent—a problem which the switch to watching mode might have incidentally triggered, but not really caused. amol_malokar ’s issue sounds similar—possibly the use of 32-bit Java led to some incompatibility with JNA that would up crashing a lot of class loading. Hard to know without being able to reproduce from scratch, including details of the Java installation packages used.

            Unassigned Unassigned
            towel Yoav Miles
            Votes:
            7 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated: