Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59838

durable-task plugin v1.30 and v1.31 - process aparently never started in /var/lib/jenkins/workspace/<jobname>

    XMLWordPrintable

Details

    Description

      I have a pipeline job and in a stage I ssh to another server which executes "Salt orchestration task" which takes 1 hour and a half of so. 

       

      stage('Deploying services with Salt') { 
       when { 
        expression { params.DeploySalt == 'Yes' } 
       } 
       steps { 
        script { 
         STAGE=env.STAGE_NAME 
        } 
        sh label: 'Deploying required software', script: "ssh -o StrictHostKeyChecking=no ${SALTMASTERLOGIN} 'sudo salt-run state.orchestrate orch.deploy --state-output=mixed'" 
       }
      }

       

      The thing is that I am always getting the following error:

       

      [Pipeline] sh (Deploying required software)
      00:07:41.080 + ssh -o StrictHostKeyChecking=no root@saltmaster sudo salt-run state.orchestrate orch.deploy --state-output=mixed
      00:32:47.371 process apparently never started in /var/lib/jenkins/workspace/local-cloud-regression-test@tmp/durable-6becef1a
      00:32:47.371 (running Jenkins temporarily with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true might make the problem clearer)[Pipeline] }

       

      The "Salt" job is still executing on the remote server but Jenkins Pipeline fails.

       

      I have searched for information about this but I can´t figure out how to fix this.

       

      Thank you.

       

       

      Attachments

        Activity

          carroll Carroll Chiou added a comment - - edited

          Can you try running your Jenkins with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true? That might help us since the logging right now is not really telling us anything. RIght now it could be anything from a bug in the script, in the environment, or the plugin.

          Also we are at 1.33 now, so you might try upgrading to that since 1.31 and 1.32 are known to have some issues.

          carroll Carroll Chiou added a comment - - edited Can you try running your Jenkins with -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true? That might help us since the logging right now is not really telling us anything. RIght now it could be anything from a bug in the script, in the environment, or the plugin. Also we are at 1.33 now, so you might try upgrading to that since 1.31 and 1.32 are known to have some issues.
          joudaon Jon Udaondo added a comment - - edited

          How can I run that?

          I have tried running it like this because I read here (https://stackoverflow.com/questions/50067372/where-to-set-dorg-jenkinsci-plugins-durabletask-bourneshellscript-heartbeat-che). But nothing happened.

           

          stage('Deploying services with Salt') { 
            when { 
              expression { params.DeploySalt == 'Yes' } 
            } 
            steps { 
              script { 
                STAGE=env.STAGE_NAME 
                System.setProperty("org.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS", "true");
              } 
              sh label: 'Deploying required software', script: "ssh -o StrictHostKeyChecking=no ${SALTMASTERLOGIN} 'sudo salt-run state.orchestrate orch.deploy --state-output=mixed'" 
            } 
          }
          

           

          I am running Jenkins as a service, should I place that in the "/etc/default/jenkins" file in the JAVA_ARGS property like this?

           

          # Allow graphs etc. to work even when an X server is present
          JAVA_ARGS="-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless=true -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS=true"
          
          

           

          After updating that, where can I see the logs, in /var/log/jenkins/jenkins.log? 

           

          I saw that v1.33 was released but in Jenkins plugins manager is not available yet. Should I update it manually? 

           

          Thank you.

          joudaon Jon Udaondo added a comment - - edited How can I run that? I have tried running it like this because I read here ( https://stackoverflow.com/questions/50067372/where-to-set-dorg-jenkinsci-plugins-durabletask-bourneshellscript-heartbeat-che ). But nothing happened.   stage( 'Deploying services with Salt' ) { when { expression { params.DeploySalt == 'Yes' } } steps { script { STAGE=env.STAGE_NAME System .setProperty( "org.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS" , " true " ); } sh label: 'Deploying required software' , script: "ssh -o StrictHostKeyChecking=no ${SALTMASTERLOGIN} 'sudo salt-run state.orchestrate orch.deploy --state-output=mixed' " } }   I am running Jenkins as a service, should I place that in the "/etc/default/jenkins" file in the JAVA_ARGS property like this?   # Allow graphs etc. to work even when an X server is present JAVA_ARGS= "-Xmx2048m -XX:MaxPermSize=512m -Djava.awt.headless= true -Dorg.jenkinsci.plugins.durabletask.BourneShellScript.LAUNCH_DIAGNOSTICS= true "   After updating that, where can I see the logs, in /var/log/jenkins/jenkins.log?    I saw that v1.33 was released but in Jenkins plugins manager is not available yet. Should I update it manually?    Thank you.
          carroll Carroll Chiou added a comment -

          So you can pass in that system property multiple ways. I would recommend passing it in the JVM startup args for however you run Jenkins, or at the script console found in the manage jenkins setting. The JVM startup is where I would recommend you do it. Once that is done, I can't say to where your logs would be, because that would depend on your system and how you installed Jenkins.

          As for 1.33, i'm not sure why it wouldn't appear in the update center. I know you mentioned that you have issues in 1.30, but if you don't see 1.33, i would recommend downgrading to 1.30. in the plugin manager UI.

          Could you open a mailing list thread on the jenkinsci-users mailing list for this question? You should add the output once you enable LAUNCH_DIAGNOSTICS, and some details about whether this was working for you previously (i.e. what version of Jenkins you are running, what OS you are using on the Jenkins master and the agent being used by the shell step, etc.). The bug tracker is not the right place for general investigation of issues like this. I am going to close this issue for now, if it looks like there is a bug after you provide more info on the mailing list, we’ll reopen the ticket

          carroll Carroll Chiou added a comment - So you can pass in that system property multiple ways. I would recommend passing it in the JVM startup args for however you run Jenkins, or at the script console found in the manage jenkins setting. The JVM startup is where I would recommend you do it. Once that is done, I can't say to where your logs would be, because that would depend on your system and how you installed Jenkins. As for 1.33, i'm not sure why it wouldn't appear in the update center. I know you mentioned that you have issues in 1.30, but if you don't see 1.33, i would recommend downgrading to 1.30. in the plugin manager UI. Could you open a mailing list thread on the jenkinsci-users mailing list for this question? You should add the output once you enable LAUNCH_DIAGNOSTICS, and some details about whether this was working for you previously (i.e. what version of Jenkins you are running, what OS you are using on the Jenkins master and the agent being used by the shell step, etc.). The bug tracker is not the right place for general investigation of issues like this. I am going to close this issue for now, if it looks like there is a bug after you provide more info on the mailing list, we’ll reopen the ticket

          People

            carroll Carroll Chiou
            joudaon Jon Udaondo
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: