Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-43587

Pipeline fails to resume after master restart/plugin upgrade

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • pipeline
    • None
    • Jenkins 2.46.1, Latest version of pipeline plugins (pipeline-build-step: 2.5, pipeline-rest-api: 2.6, pipeline-stage-step: 2.2, etc)
    • durable-task 1.18

      During a recent Jenkins plugin upgrade and master restart, it seems that Jenkins failed to resume at least two Pipeline jobs. The pipeline was in the middle of a sh() step when the master was restarted. Both jobs have output similar to the following in the console:

      Resuming build at Thu Apr 13 15:01:50 EDT 2017 after Jenkins restart
Waiting to resume part of <job name...>: ???
Ready to run at Thu Apr 13 15:01:51 EDT 2017

       

      However this text has been displayed for several minutes now with no obvious indication on what the job is waiting for. We can see that the pipeline is still running on the correct executor that it was running on pre-restart however, if we log into the server, there is no durable task or process of the script that the sh() step was running. From logging of the script that we were running, we can tell that the command did finish successfully but can't understand how Jenkins lost track of it. From the logging, the time when the command finished was around the same time when the master was restarting (it is difficult to pinpoint exactly). 

        1. jenkins.log
          46 kB
        2. image-2019-05-17-16-09-05-599.png
          image-2019-05-17-16-09-05-599.png
          15 kB

          [JENKINS-43587] Pipeline fails to resume after master restart/plugin upgrade

          Erik Lattimore created issue -
          Erik Lattimore made changes -
          Link New: This issue relates to JENKINS-39552 [ JENKINS-39552 ]
          Erik Lattimore made changes -
          Description Original: During a recent Jenkins plugin upgrade and master restart, it seems that Jenkins failed to resume at least one Pipeline job. The pipeline was in the middle of a sh() step when the master was restarted. In the console output after the restart we see the following:
          {noformat}
          Resuming build at Thu Apr 13 15:01:50 EDT 2017 after Jenkins restart
          Waiting to resume part of <job name...>: ???
          Ready to run at Thu Apr 13 15:01:51 EDT 2017{noformat}
           

          However this text has been displayed for several minutes now with no obvious indication on what the job is waiting for. We can see that the pipeline is still running on the correct executor that it was running on pre-restart however, if we log into the server, there is no durable task or process of the script that the sh() step was running. From logging of the script that we were running, we can tell that the command did finish successfully but can't understand how Jenkins lost track of it. From the logging, the time when the command finished was around the same time when the master was restarting (it is difficult to pinpoint exactly).            		New: During a recent Jenkins plugin upgrade and master restart, it seems that Jenkins failed to resume at least two Pipeline jobs. The pipeline was in the middle of a sh() step when the master was restarted. Both jobs have output similar to the following in the console:
          {noformat}
          Resuming build at Thu Apr 13 15:01:50 EDT 2017 after Jenkins restart
          Waiting to resume part of <job name...>: ???
          Ready to run at Thu Apr 13 15:01:51 EDT 2017{noformat}
           

          However this text has been displayed for several minutes now with no obvious indication on what the job is waiting for. We can see that the pipeline is still running on the correct executor that it was running on pre-restart however, if we log into the server, there is no durable task or process of the script that the sh() step was running. From logging of the script that we were running, we can tell that the command did finish successfully but can't understand how Jenkins lost track of it. From the logging, the time when the command finished was around the same time when the master was restarting (it is difficult to pinpoint exactly). 
          Elifarley made changes -
          Link New: This issue is related to JENKINS-46961 [ JENKINS-46961 ]
          Sam Van Oort made changes -
          Assignee New: Sam Van Oort [ svanoort ]
          Sam Van Oort made changes -
          Assignee Original: Sam Van Oort [ svanoort ] New: Alex Taylor [ ataylor ]
          Mircea-Andrei Albu made changes -
          Attachment New: image-2019-05-17-16-09-05-599.png [ 47408 ]
          papanito made changes -
          Attachment New: jenkins.log [ 51178 ]
          Alex Taylor made changes -
          Released As New: durable-task 1.18
          Resolution New: Fixed [ 1 ]
          Status Original: Open [ 1 ] New: Fixed but Unreleased [ 10203 ]
          Alex Taylor made changes -
          Status Original: Fixed but Unreleased [ 10203 ] New: Resolved [ 5 ]
          papanito made changes -
          Link New: This issue is related to JENKINS-62248 [ JENKINS-62248 ]

            ataylor Alex Taylor
            elatt Erik Lattimore
            Votes:
            13 Vote for this issue
            Watchers:
            29 Start watching this issue

              Created:
              Updated:
              Resolved: