[Solved] All Hyper-V VM are Saved State. An error occurs when trying to change the state of a virtual machine

Some of the virtual machines went into the Saved state (and on some servers all virtual machines), and when you try to start them (execute Start), we get an error message:

SERVER-NAME failed to restore virtual machine state.
Virtual machine SERVER-NAME could not be started because the hypervisor is not running.

On older hypervisors, the error is:
SERVER-NAME failed to change state.

Screenshot of virtual machine states:


Error message:


Errors found in system logs:

System Error 
Source: Hyper-V-Hypervisor
Error ID: 80
Hypervisor launch failed; The operating systems boot loader failed with error 0xC00000BB.

Log Name: Microsoft-Windows-Hyper-V-VMMS/Admin
Source: Hyper-V-VMMS
Error ID: 15160
'SERVER-NAME' failed to restore virtual machine state. 

Log Name: Microsoft-Windows-Hyper-V-VMMS/Admin
Source: Hyper-V-VMMS
Error ID: 15350
The virtualization infrastructure driver (VID) is not running.

Log Name: Microsoft-Windows-Hyper-V-VMMS/Admin
Source: Hyper-V-VMMS
Error ID: 20148
Virtual machine 'SERVER-NAME' could not be started because the hypervisor is not running (Virtual machine ID 53E42B3E-32CA-4F38-BF0F-C22D8CCED5A7). The following actions may help you resolve the problem: 1) Verify that the processor of the physical computer has a supported version of hardware-assisted virtualization. 2) Verify that hardware-assisted virtualization and hardware-assisted data execution protection are enabled in the BIOS of the physical computer.  (If you edit the BIOS to enable either setting, you must turn off the power to the physical computer and then turn it back on.  Resetting the physical computer is not sufficient.) 3) If you have made changes to the Boot Configuration Data store, review these changes to ensure that the hypervisor is configured to launch automatically. 

Solution:

The issue is related to the Monthly Rollup KB5009624 
KB5009624 (Monthly Rollup)

To confirm this, find the line in the update history:
2022-01 Security Monthly Quality Rollup for Windows Server 2012 R2 x64-based System (KB5009624)



Next, go to Control Panel - Add or Remove Programs - Installed Updates:


Finding an update in the list Security Update for Microsoft Windows (KB5009624):


Remove this update:


After uninstalling the update, the server will reboot.
Please note that virtual machines will not start automatically after restarting the hypervisor. You will need to manually change the state of virtual machines (start).
Temporarily disable the installation of system updates or specifically updates: 2022-01 Security Monthly Quality Rollup for Windows Server 2012 R2 x64-based System (KB5009624):



Colleagues report that security update KB5009624 on Windows Server 2012 R2 breaks network folder access from scanners. When you try to scan to a network folder, for example, from an HP MFP, DNS drops, the domain controller reports that Windows has found an error and a minute later it goes into reboot.

Official Microsoft comments about issues in this update (KB5009624):
After installing this update on a device by using Unified Extensible Firmware Interface (UEFI), virtual machines (VMs) in Hyper-V might not start.
We are currently investigating and will provide an update in an upcoming release.
UPDATE:
For solve this issue Microsoft provided the update KB5010794: Out-of-band update for Windows 8.1 and Windows Server 2012 R2.
This update resolves the following issues: 
• Virtual machines (VMs) located on a server that has Unified Extensible Firmware Interface (UEFI) enabled fail to start after installing update KB5009624.
• Active Directory attributes are not written correctly during a Lightweight Directory Access Protocol (LDAP) modify operation with multiple specific attribute changes.
• Windows Servers might restart unexpectedly after installing update KB5009624 on domain controllers (DCs).

You need to install this update:





(с) Ella S.
If you liked the article, please like, repost or leave a comment. If you have any comments, also write comments.

9 комментариев:

  1. We have received official comments from Microsoft about problems starting virtual machines after installing update (KB5009624). Updated the text of the article.

    ОтветитьУдалить
  2. Colleagues also report that there is an issue on Windows Server 2019 with update KB5009557, due to which the domain controller can go into "perpetual reboot", which ultimately leads to network problems.

    ОтветитьУдалить
  3. Known issues in this update KB5009624 (Monthly Rollup):
    After installing this update on domain controllers (DCs), affected versions of Windows Server might restart unexpectedly.
    Note: On Windows Server 2016 and later, you are more likely to be affected when DCs use Shadow Principals in Enhanced Security Admin Environment (ESAE) or environments that use Privileged Identity Management (PIM).

    ОтветитьУдалить
  4. Thank you! Really defective update. As, however, very often with Microsoft

    ОтветитьУдалить
  5. We have experienced this issue in a cluster, currently all VMs are running on one host. We have uninstalled the mentioned updated, and installed the fixed update KB5010794. Virtualization is enabled from BIOS, and also powered cycle the server however the Microsoft Hyper-V Virtualization Driver is not found in Device Manager. Without this driver Hyper-V is not aware of the CPU Virtualization. We have also disabled the feature and enabled it again, power cycle the server but to no available. When trying to scan for hardware change in Device Manager nothing is being found.

    ОтветитьУдалить
    Ответы
    1. Try adding the device as legacy hardware. select System devices and choose microsoft as comany then find the drive. in our case it took around 5 mins to complete the install. it will look as stuck but in the end resolved our issue.

      Удалить