Uploaded image for project: 'Jenkins'
  1. Jenkins
  2. JENKINS-67604

WMI Windows Agent will no longer connect after Microsoft KB5004442 security update

    • Icon: Bug Bug
    • Resolution: Won't Fix
    • Icon: Critical Critical
    • windows-slaves-plugin
    • None
    • jenkins controller: 2.319.2 on centos7, adoptium jdk-11.0.13+8, Windows Agents v1.8 plugin
      jenkins build node: Windows 10 Enterprise, 10.0.19042 Build 19042, includes updates through 2022-01-13

      • A connection attempt to windows build node generates the following EventViewer system log message on the build node:
        • source: DistributedCOM
        • Event ID: 10036
        • Level: Error
          The server-side authentication level policy does not allow the user ***** from address ***** to activate DCOM server. Please raise the activation authentication level at least to RPC_C_AUTHN_LEVEL_PKT_INTEGRITY in client application.

       

          [JENKINS-67604] WMI Windows Agent will no longer connect after Microsoft KB5004442 security update

          • Snippet from jenkins node log:
            [2022-01-15 11:08:29] [windows-agents] Connecting to *****
            ERROR: Message not found for errorCode: 0x80010111
            org.jinterop.dcom.common.JIException: Message not found for errorCode: 0x80010111
            	at org.jinterop.dcom.core.JIComServer.init(JIComServer.java:580)
            	at org.jinterop.dcom.core.JIComServer.initialise(JIComServer.java:481)
            	at org.jinterop.dcom.core.JIComServer.<init>(JIComServer.java:445)
            	at org.jvnet.hudson.wmi.WMI.connect(WMI.java:59)
            	at hudson.os.windows.ManagedWindowsServiceLauncher.launch(ManagedWindowsServiceLauncher.java:208)
            	at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:293)
            	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
            	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80)
            	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
            	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
            	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
            	at java.lang.Thread.run(Thread.java:748)
            Caused by: org.jinterop.dcom.common.JIRuntimeException: Message not found for errorCode: 0x80010111
            	at org.jinterop.dcom.core.JIRemActivation.read(JIRemActivation.java:192)
            	at ndr.NdrObject.decode(NdrObject.java:36)
            	at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137)
            	at rpc.Stub.call(Stub.java:113)
            	at org.jinterop.dcom.core.JIComServer.init(JIComServer.java:568)
            	... 11 more

             

          Mike Butterfield added a comment - Snippet from jenkins node log: [2022-01-15 11:08:29] [windows-agents] Connecting to ***** ERROR: Message not found for errorCode: 0x80010111 org.jinterop.dcom.common.JIException: Message not found for errorCode: 0x80010111 at org.jinterop.dcom.core.JIComServer.init(JIComServer.java:580) at org.jinterop.dcom.core.JIComServer.initialise(JIComServer.java:481) at org.jinterop.dcom.core.JIComServer.<init>(JIComServer.java:445) at org.jvnet.hudson.wmi.WMI.connect(WMI.java:59) at hudson.os.windows.ManagedWindowsServiceLauncher.launch(ManagedWindowsServiceLauncher.java:208) at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:293) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:80) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.jinterop.dcom.common.JIRuntimeException: Message not found for errorCode: 0x80010111 at org.jinterop.dcom.core.JIRemActivation.read(JIRemActivation.java:192) at ndr.NdrObject.decode(NdrObject.java:36) at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:137) at rpc.Stub.call(Stub.java:113) at org.jinterop.dcom.core.JIComServer.init(JIComServer.java:568) ... 11 more  

          Alex Earl added a comment -

          I highly recommend moving to using SSH even for Windows agents. It is much more stable. Do you have a specific requirement to use DCOM?

          Alex Earl added a comment - I highly recommend moving to using SSH even for Windows agents. It is much more stable. Do you have a specific requirement to use DCOM?

          I have converted my jenkins windows nodes to use SSH, and have not experienced any problems, so this is no longer an issue for our me. 

          The SSH feature is already included in Windows Server 2019 and Windows 10 and can be enabled via Optional Features. See https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview for further info.
          For earlier Windows versions, OpenSSH can be downloaded and installed. See above microsoft link or visit https://github.com/PowerShell/OpenSSH-Portable  

          For those who have not yet switched,  the '...enabled by default...' update is scheduled for 2022-06-14 (as of 2022-05-06), unless Microsoft quietly chooses to slide the timeline dates again, as they did earlier this year.

          Mike Butterfield added a comment - I have converted my jenkins windows nodes to use SSH, and have not experienced any problems, so this is no longer an issue for our me.  The SSH feature is already included in Windows Server 2019 and Windows 10 and can be enabled via Optional Features. See https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview for further info. For earlier Windows versions, OpenSSH can be downloaded and installed. See above microsoft link or visit https://github.com/PowerShell/OpenSSH-Portable   For those who have not yet switched,  the '...enabled by default...' update is scheduled for 2022-06-14 (as of 2022-05-06), unless Microsoft quietly chooses to slide the timeline dates again, as they did earlier this year.

          Alex Earl added a comment -

          Another option is to use the Windows Cloud plugin (https://plugins.jenkins.io/windows-cloud/), it uses WinRM to launch jobs on agents. SSH is definitely the best option in my opinion, but if someone doesn't want to use SSH, that plugin may be an option. 

          Alex Earl added a comment - Another option is to use the Windows Cloud plugin ( https://plugins.jenkins.io/windows-cloud/ ), it uses WinRM to launch jobs on agents. SSH is definitely the best option in my opinion, but if someone doesn't want to use SSH, that plugin may be an option. 

          The timeline for https://support.microsoft.com/en-us/topic/kb5004442-manage-changes-for-windows-dcom-server-security-feature-bypass-cve-2021-26414-f1400b52-c141-43d2-941e-37ed901c769c was updated on 10/19/22 and now includes an update on 2022-11-08:

           

          June 8, 2021 Hardening changes disabled by default but with the ability to enable them using a registry key.
          June 14, 2022 Hardening changes enabled by default but with the ability to disable them using a registry key.
          November 8, 2022 This update will automatically raise authentication level for all non-anonymous activation requests from DCOM clients to RPC_C_AUTHN_LEVEL_PKT_INTEGRITY if it is below Packet Integrity. With this change, most Windows DCOM client applications will automatically work with DCOM hardening change on server side without any modification to the DCOM client applications.
          March 14, 2023 Hardening changes enabled by default with no ability to disable them. By this point, you must resolve any compatibility issues with the hardening changes and applications in your environment.

           

          As noted above, our team has universally abandoned use of this plugin and are now using the 'Launch agents via SSH' plugin. 

          To configure windows for SSH:

          Mike Butterfield added a comment - The timeline for https://support.microsoft.com/en-us/topic/kb5004442-manage-changes-for-windows-dcom-server-security-feature-bypass-cve-2021-26414-f1400b52-c141-43d2-941e-37ed901c769c was updated on 10/19/22 and now includes an update on 2022-11-08:   June 8, 2021 Hardening changes disabled by default but with the ability to enable them using a registry key. June 14, 2022 Hardening changes enabled by default but with the ability to disable them using a registry key. November 8, 2022 This update will automatically raise authentication level for all non-anonymous activation requests from DCOM clients to RPC_C_AUTHN_LEVEL_PKT_INTEGRITY if it is below Packet Integrity. With this change, most Windows DCOM client applications will automatically work with DCOM hardening change on server side without any modification to the DCOM client applications. March 14, 2023 Hardening changes enabled by default with no ability to disable them. By this point, you must resolve any compatibility issues with the hardening changes and applications in your environment.   As noted above, our team has universally abandoned use of this plugin and are now using the ' Launch agents via SSH ' plugin.  To configure windows for SSH: For Windows versions at or above Windows Server 2019 and Windows 10, see https://learn.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview For Windows Versions PRIOR to Server 2019 or Windows 10 use portable SSH:  Fohttps://github.com/PowerShell/Win32-OpenSSH/releases

          IIUC, Microsoft's upcoming 2022-11-08 update changes the DCOM client implementation in Windows. The WMI Windows Agents plugin uses portable libraries (https://github.com/jenkinsci/extras-jinterop-wmi and https://sourceforge.net/projects/j-interop/), which implement the network protocol in Java instead of calling Windows components via JNI, so the 2022-11-08 update will not help this plugin.

          Kalle Niemitalo added a comment - IIUC, Microsoft's upcoming 2022-11-08 update changes the DCOM client implementation in Windows. The WMI Windows Agents plugin uses portable libraries ( https://github.com/jenkinsci/extras-jinterop-wmi and https://sourceforge.net/projects/j-interop/ ), which implement the network protocol in Java instead of calling Windows components via JNI, so the 2022-11-08 update will not help this plugin.

          Mark Waite added a comment -

          Won't be fixed. See JENKINS-67604 for the details of the deprecation of agents started by WMI calls using DCOM.

          Mark Waite added a comment - Won't be fixed. See JENKINS-67604 for the details of the deprecation of agents started by WMI calls using DCOM.

            escoem Emilio Escobar
            mb_austex Mike Butterfield
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: