Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59824

Ignore timeout while provisioning

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Duplicate
    • Component/s: _unsorted
    • Labels:
      None
    • Similar Issues:

      Description

      We've set up a new custom image which takes a lot of time provisioning (40+ minutes).

      I've set the deployment timeout to 5000 seconds but after around 40 minutes the VM will get deleted cause of this exception:

       

      AzureVMCloud: createProvisionedAgent: Deployment win10pp16-1017121639027 not yet finished (Running): Microsoft.Compute/virtualMachines:win10pp-725b70 - waited 2,310 seconds
      Oct 17, 2019 12:56:06 PM SEVERE com.microsoft.azure.vmagent.AzureVMCloud$2 callFailure creating provisioned agent 'win10pp-725b70'
      com.microsoft.azure.vmagent.exceptions.AzureCloudException: AzureVMCloud: createProvisionedAgent: Deployment Failed: Microsoft.Compute/virtualMachines:win10pp-725b70 - Conflict - {status=Failed, error={code=ResourceDeploymentFailure, message=The resource operation completed with terminal provisioning state 'Failed'., details=[{code=OSProvisioningTimedOut, message=OS Provisioning for VM 'win10pp-725b70' did not finish in the allotted time. The VM may still finish provisioning successfully. Please check provisioning state later. Also, make sure the image has been properly prepared (generalized).
       * Instructions for Windows: https://azure.microsoft.com/documentation/articles/virtual-machines-windows-upload-image/ 
       * Instructions for Linux: https://azure.microsoft.com/documentation/articles/virtual-machines-linux-capture-image/ }]}}
      	at com.microsoft.azure.vmagent.exceptions.AzureCloudException.create(AzureCloudException.java:37)
      	at com.microsoft.azure.vmagent.AzureVMCloud.createProvisionedAgent(AzureVMCloud.java:584)
      	at com.microsoft.azure.vmagent.AzureVMCloud$2.call(AzureVMCloud.java:830)
      	at com.microsoft.azure.vmagent.AzureVMCloud$2.call(AzureVMCloud.java:808)
      	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
      	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
      	at java.util.concurrent.FutureTask.run(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at java.lang.Thread.run(Unknown Source)
      
      
      

      Is there a way to ignore this exception? The VM actually works after waiting another 10-20 minutes (it then switches from status creating to running).

      Right now it will delete the VM and start creating a new one which also runs into the timout.

        Attachments

          Activity

          Hide
          jieshe Jie Shen added a comment -

          According to discussion at https://github.com/Azure/azure-sdk-for-node/issues/1491, it seems to be a service side issue. I also have not figured out how to change it in the java sdk. So it is possible to improve the image itself or try to deploy to different locations?

          Show
          jieshe Jie Shen added a comment - According to discussion at  https://github.com/Azure/azure-sdk-for-node/issues/1491 , it seems to be a service side issue. I also have not figured out how to change it in the java sdk. So it is possible to improve the image itself or try to deploy to different locations?
          Hide
          orangepie Mike K added a comment -

          I was already troubleshooting the used image with the MS Azure support.
          I'm using a custom generalized Win10 image with software and frameworks preinstalled. The reinstallation of that apps just takes a lot of time and sadly can't be speed up.

          The VM is actually provisioning fine it just exceeds the 40 minutes after which Azure seems to throw the timeout exception.

          As the support told me there is no way to increase the timeout on Azure I would expect the vm agent plugin to respect my set deploy timeout value.
          The thrown exception can be clearly identified as timeout exception (by it's title and message. I can provide a log entry if needed) so I'd suggest to not abort the wait process if the timeout exception is thrown before the plugins deploy timeout is exceeded.

          Show
          orangepie Mike K added a comment - I was already troubleshooting the used image with the MS Azure support. I'm using a custom generalized Win10 image with software and frameworks preinstalled. The reinstallation of that apps just takes a lot of time and sadly can't be speed up. The VM is actually provisioning fine it just exceeds the 40 minutes after which Azure seems to throw the timeout exception. As the support told me there is no way to increase the timeout on Azure I would expect the vm agent plugin to respect my set deploy timeout value. The thrown exception can be clearly identified as timeout exception (by it's title and message. I can provide a log entry if needed) so I'd suggest to not abort the wait process if the timeout exception is thrown before the plugins deploy timeout is exceeded.
          Hide
          timja Tim Jacomb added a comment -

          All issues have been transferred to GitHub.

          See https://github.com/jenkinsci/azure-vm-agents-plugin/issues

          Search the issue title to find it.

          (This is a bulk comment and can't link to the specific issue)

          Show
          timja Tim Jacomb added a comment - All issues have been transferred to GitHub. See https://github.com/jenkinsci/azure-vm-agents-plugin/issues Search the issue title to find it. (This is a bulk comment and can't link to the specific issue)

            People

            Assignee:
            jieshe Jie Shen
            Reporter:
            orangepie Mike K
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: