Bug 1408813

Summary:	PCI: Reserve MMIO space over 4G for PCI hotplug
Product:	Red Hat Enterprise Linux 7	Reporter:	Marcel Apfelbaum <marcel>
Component:	libvirt	Assignee:	Erik Skultety <eskultet>
Status:	CLOSED NOTABUG	QA Contact:	Jingjing Shao <jishao>
Severity:	low	Docs Contact:
Priority:	low
Version:	7.4	CC:	ailan, chayang, dyuan, jinzhao, jsuchane, juzhang, lhuang, mtessun, rbalakri, virt-maint, xuzhang
Target Milestone:	rc	Keywords:	FutureFeature
Target Release:	7.4
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Enhancement
Doc Text:		Story Points:	---
Clone Of:	1390346
Clones:	1410580 1410589 (view as bug list)		Environment:
Last Closed:	2017-04-10 11:35:33 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1390346
Bug Blocks:	1410580, 1410589

Description Marcel Apfelbaum 2016-12-27 12:14:54 UTC

+++ This bug was initially created as a clone of Bug #1390346 +++

QEMU reserves only 32-bit memory range for hotplug. The range can be rather limited and not enough to hot-plug devices with large BARs.

Add a parameter to QEMU to reserve a 64-bit range in CRS
starting after the memory ranged reserved for memory hotplug.

Check that all the range is addressable by VM's CPU.

Add libvirt support for the new command line parameters.

Comment 3 Marcel Apfelbaum 2017-01-25 13:28:15 UTC

Hi,

In order to be able to hot-plug PCI devices, QEMU needs to reserve some address space to be mapped to PCI devices registers.

QEMU today reserves space only on 32-bit area (< 4G) memory. The problem is the "window" reserved is not always enough, especially if the VM has a lot of devices (they also use the same pool).
However (>4G) memory space is huge and the most part is not actually used, the only limit is the CPU addressable bits.

The question is how much space to reserve? Reserving too much can limit the migration only to the host where the VMs can support the same addressable bits.

Who can "guess" how much reservation we "need"? I suppose libvirt/nova, they can query the pyhsical PCI devices that may be attached in the future, the hosts CPU limitations and so on.

However, this is too low level and we want to come up with a solution that does not involve libvirt/nova, at least we will try.

Thanks,
Marcel

Comment 4 Jaroslav Suchanek 2017-04-10 11:35:33 UTC

(In reply to Marcel Apfelbaum from comment #3)
> 
> However, this is too low level and we want to come up with a solution that
> does not involve libvirt/nova, at least we will try.

Ok, please reopen if there is anything for libvirt.

Thanks,
J.