Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-59522

Takes long time to deploy custom extentions only for windows 10 custome managed image

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Duplicate
    • Component/s: _unsorted
    • Labels:
      None
    • Environment:
      Jenkins 2.176.1
      Azure vm plugin 1.2.1 and 1.2.2
    • Similar Issues:

      Description

      Hi,

      We are using Custom Managed image based on Windows 10 RS5 for one of our cloud configuration. 

      It was working fine but since last week it has started to show us the problems. 

      Sometimes it takes more than 25 minutes to spin up one agent while all other agents (which are Windows server, windows server core or ubuntu16 custom manged images) come up in 2 to 4 minutes. 

      I have created new Windows 10 RS3 and Rs5 images but  nothing seems to help. It still takes 15/17 minutes to spin up an agent. When I see at "deployments" progress in Azure, It is obvious that the most time is taken while installing the "custom extentions" The VM is created in 3 minutes and can be logged in to, but the deployment of custom extentions takes 10 or more minutes. What could be the possible reason  ?

      I tried with version 1.1.1 too but with no apparant success. Any clues ?

       

        Attachments

          Activity

          Hide
          faizan Muhammad Faizan ul haq added a comment -

          If the Custom Managed Image has "windows test mode" enabled, the deployment takes 25 minutes and then fails with an error: 

          { "status": "Canceled", "error": { "code": "ResourceDeploymentFailure", "message": "The resource operation completed with terminal provisioning state 'Canceled'.", "details": [

          { "code": "OperationPreempted", "message": "The operation has been preempted by a more recent operation." }

          ] } }

          Show
          faizan Muhammad Faizan ul haq added a comment - If the Custom Managed Image has "windows test mode" enabled, the deployment takes 25 minutes and then fails with an error:  { "status": "Canceled", "error": { "code": "ResourceDeploymentFailure", "message": "The resource operation completed with terminal provisioning state 'Canceled'.", "details": [ { "code": "OperationPreempted", "message": "The operation has been preempted by a more recent operation." } ] } }
          Hide
          faizan Muhammad Faizan ul haq added a comment -

          I created a fresh image using packer, using the following code: 

          {
          "builders": [{
          "type": "azure-arm",

          "client_id": "",
          "client_secret": "",
          "tenant_id": "",
          "subscription_id": "",

          "managed_image_resource_group_name": "",
          "managed_image_name": "",
          "os_type": "Windows",
          "image_publisher": "MicrosoftWindowsDesktop",
          "image_offer": "Windows-10",
          "image_sku": "RS5-PRO",
          "communicator": "winrm",
          "winrm_use_ssl": true,
          "winrm_insecure": true,
          "winrm_timeout": "5m",
          "winrm_username": "packer",
          "azure_tags":

          { "dept": "", "task": "" }

          ,
          "location": "North Europe",
          "vm_size": "Standard_DS2_v2"
          }],
          "provisioners": [

          { "type": "powershell", "inline": [ "Set-Service RdAgent -StartupType Disabled", "Set-Service WindowsAzureTelemetryService -StartupType Disabled", "Set-Service WindowsAzureGuestAgent -StartupType Disabled", "remove-itemproperty -Path 'HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\SysPrepExternal\\Generalize' -Name '*'", "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml )\{ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force}

          ",
          "& $env:SystemRoot\\System32\\Sysprep
          Sysprep.exe /oobe /generalize /quiet /quit",
          "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup
          State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE')

          { Write-Output $imageState.ImageState; Start-Sleep -s 10 }

          else { break } }"
          ]

          }]
          }
          ~

          and it took 9 minutes for deployment. Kindly suggest if we are doing something wrong. this is only for windows-10 Desktop

          Show
          faizan Muhammad Faizan ul haq added a comment - I created a fresh image using packer, using the following code:  { "builders": [{ "type": "azure-arm", "client_id": "", "client_secret": "", "tenant_id": "", "subscription_id": "", "managed_image_resource_group_name": "", "managed_image_name": "", "os_type": "Windows", "image_publisher": "MicrosoftWindowsDesktop", "image_offer": "Windows-10", "image_sku": "RS5-PRO", "communicator": "winrm", "winrm_use_ssl": true, "winrm_insecure": true, "winrm_timeout": "5m", "winrm_username": "packer", "azure_tags": { "dept": "", "task": "" } , "location": "North Europe", "vm_size": "Standard_DS2_v2" }], "provisioners": [ { "type": "powershell", "inline": [ "Set-Service RdAgent -StartupType Disabled", "Set-Service WindowsAzureTelemetryService -StartupType Disabled", "Set-Service WindowsAzureGuestAgent -StartupType Disabled", "remove-itemproperty -Path 'HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup\\SysPrepExternal\\Generalize' -Name '*'", "if( Test-Path $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml )\{ rm $Env:SystemRoot\\windows\\system32\\Sysprep\\unattend.xml -Force} ", "& $env:SystemRoot\\System32\\Sysprep Sysprep.exe /oobe /generalize /quiet /quit", "while($true) { $imageState = Get-ItemProperty HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Setup State | Select ImageState; if($imageState.ImageState -ne 'IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE') { Write-Output $imageState.ImageState; Start-Sleep -s 10 } else { break } }" ] }] } ~ and it took 9 minutes for deployment. Kindly suggest if we are doing something wrong. this is only for windows-10 Desktop
          Hide
          nilesha Nilesh Anchan added a comment -

          Even we are facing the issue when custom image is taking longer time to load in Jenkins.

          Show
          nilesha Nilesh Anchan added a comment - Even we are facing the issue when custom image is taking longer time to load in Jenkins.
          Hide
          faizan Muhammad Faizan ul haq added a comment -

          I realized that only custom extension was taking longer time and the thing that is done in "custom extension" is the installation of SSH daemon. We pre installed the image with ssh and exclude the ssh installation in cloud config. 

          Show
          faizan Muhammad Faizan ul haq added a comment - I realized that only custom extension was taking longer time and the thing that is done in "custom extension" is the installation of SSH daemon. We pre installed the image with ssh and exclude the ssh installation in cloud config. 
          Hide
          jieshe Jie Shen added a comment -

          Muhammad Faizan ul haq Thanks for your shared information!

          Show
          jieshe Jie Shen added a comment - Muhammad Faizan ul haq Thanks for your shared information!
          Hide
          timja Tim Jacomb added a comment -

          All issues have been transferred to GitHub.

          See https://github.com/jenkinsci/azure-vm-agents-plugin/issues

          Search the issue title to find it.

          (This is a bulk comment and can't link to the specific issue)

          Show
          timja Tim Jacomb added a comment - All issues have been transferred to GitHub. See https://github.com/jenkinsci/azure-vm-agents-plugin/issues Search the issue title to find it. (This is a bulk comment and can't link to the specific issue)

            People

            Assignee:
            jieshe Jie Shen
            Reporter:
            faizan Muhammad Faizan ul haq
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: