Bug 2375049

Summary: Additional memory assigned by Hyper-V dynamic memory not visible by system after initial allocation
Product: [Fedora] Fedora Reporter: Luca Cavana <luca.cavana>
Component: kernelAssignee: Vitaly Kuznetsov <vkuznets>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 42CC: acaringi, adscvr, airlied, hdegoede, hpa, jforbes, josef, kernel-maint, linville, masami256, mchehab, mmuzila, ptalbert, steved, suraj.ghimire7, vkuznets
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-6.15.4-200.fc42 kernel-6.15.4-100.fc41 Doc Type: ---
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2025-06-30 02:21:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Ansible playbook to create systemd-udevd rules none

Description Luca Cavana 2025-06-26 20:19:50 UTC 
Virtual Machines running under Hyper-V with the Dynamic Memory feature turned on are unable to use the additional memory granted by Hyper-V.

Example:
A VM configured to use:
- 1024 MB of RAM as "Startup"
- 512 MB of RAM as "Minimum"
- 8192 MB of RAM as "Maximum"
- 30% Memory buffer

Will always show the output of the free -h command as having 883MB of TOTAL memory.
When applications inside any of these VMs begin putting memory pressure on the host, Hyper-V will allocate additional memory to the VM but the system will never see it; free -h command will always display 883MB of totale memory.

Reproducible: Always

Steps to Reproduce:
1. Install a fresh Fedora 42 system on Hyper-V (Windows Server 2022)
2. Configure the VM to use dynamic memory
3. Put some memory pressure on the VM.
Actual Results:
The memory allocated to the VM by the hypervisor will grow but the memory available inside the VM will not.

Expected Results:
The memory allocated to the VM by the hypervisor will grow as it will grow the memory available to the VM, as reported by the free -h command.

Additional Information:
With applications that put more stress than others on the memory (as it is the case with databases) the situation escalates quite rapidly in Hyper-V reaching the "Maximum" limit, but the VM still sees their meager 883MB.
Comment 1 Vitaly Kuznetsov 2025-06-27 08:41:07 UTC 
The problem is that we lost CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y as upstream it was replaced with CONFIG_MHP_DEFAULT_ONLINE_TYPE_* setting.

A quick workaround is to add a udev rule for memory onlining:

SUBSYSTEM=="memory", ACTION=="add", ATTR{state}=="offline", ATTR{state}="online"
Comment 2 Luca Cavana 2025-06-27 14:22:30 UTC 
Created attachment 2095404 [details]
Ansible playbook to create systemd-udevd rules

The workaround proposed by Vitaly works, I've put together a simple Ansible playbook that takes care of applying the workaround at scale for Fedora 42 hosts while there is a more permanent fix in the code.
Comment 3 Fedora Update System 2025-06-27 19:12:47 UTC 
FEDORA-2025-7feed8b25a (kernel-6.15.4-200.fc42 and linux-firmware-20250627-1.fc42) has been submitted as an update to Fedora 42.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-7feed8b25a
Comment 4 Fedora Update System 2025-06-27 19:13:04 UTC 
FEDORA-2025-f6f8526a43 (kernel-6.15.4-100.fc41 and linux-firmware-20250627-1.fc41) has been submitted as an update to Fedora 41.
https://bodhi.fedoraproject.org/updates/FEDORA-2025-f6f8526a43
Comment 5 Justin M. Forbes 2025-06-27 19:16:15 UTC 
(In reply to Vitaly Kuznetsov from comment #1)
> The problem is that we lost CONFIG_MEMORY_HOTPLUG_DEFAULT_ONLINE=y as
> upstream it was replaced with CONFIG_MHP_DEFAULT_ONLINE_TYPE_* setting.

I fixed stable Fedora with your MR + a couple of fixes. See https://gitlab.com/cki-project/kernel-ark/-/commit/1d496a80392fa333f57e39b95cde0f19864034c1

We do still need your MR for kernel-ark os-build though so that rawhide can get the fix.  As it changes configs in common, it will require RHEL acks.
Comment 6 Fedora Update System 2025-06-28 02:09:02 UTC 
FEDORA-2025-7feed8b25a has been pushed to the Fedora 42 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-7feed8b25a`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-7feed8b25a

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 7 Fedora Update System 2025-06-28 02:33:54 UTC 
FEDORA-2025-f6f8526a43 has been pushed to the Fedora 41 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2025-f6f8526a43`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2025-f6f8526a43

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.
Comment 8 Luca Cavana 2025-06-28 08:02:07 UTC 
(In reply to Fedora Update System from comment #6)
> FEDORA-2025-7feed8b25a has been pushed to the Fedora 42 testing repository.
> Soon you'll be able to install the update with the following command:
> `sudo dnf upgrade --enablerepo=updates-testing --refresh
> --advisory=FEDORA-2025-7feed8b25a`
> You can provide feedback for this update here:
> https://bodhi.fedoraproject.org/updates/FEDORA-2025-7feed8b25a
> 
> See also https://fedoraproject.org/wiki/QA:Updates_Testing for more
> information on how to test updates.

I tried the fix on a few test systems, after undoing the systemd-udevd rules, and the dynamic memory feature has restarted working correctly.
It is allocating and removing memory as expected.

Thank you very much for the quick answer and fix.
Comment 9 Fedora Update System 2025-06-30 02:21:59 UTC 
FEDORA-2025-7feed8b25a (kernel-6.15.4-200.fc42 and linux-firmware-20250627-1.fc42) has been pushed to the Fedora 42 stable repository.
If problem still persists, please make note of it in this bug report.
Comment 10 Fedora Update System 2025-06-30 02:46:20 UTC 
FEDORA-2025-f6f8526a43 (kernel-6.15.4-100.fc41 and linux-firmware-20250627-1.fc41) has been pushed to the Fedora 41 stable repository.
If problem still persists, please make note of it in this bug report.