Bug 1414081
Summary: | qemu-kvm-ma: Emulated GICv3 doesn't support MSI | ||||||||
---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 7 | Reporter: | Richard W.M. Jones <rjones> | ||||||
Component: | qemu-kvm-ma | Assignee: | Eric Auger <eric.auger> | ||||||
Status: | CLOSED NOTABUG | QA Contact: | Virtualization Bugs <virt-bugs> | ||||||
Severity: | unspecified | Docs Contact: | |||||||
Priority: | unspecified | ||||||||
Version: | 7.4 | CC: | abologna, chayang, drjones, eric.auger, hhuang, jcm, juzhang, laine, libvirt-maint, rbalakri, virt-maint | ||||||
Target Milestone: | rc | ||||||||
Target Release: | --- | ||||||||
Hardware: | aarch64 | ||||||||
OS: | Unspecified | ||||||||
Whiteboard: | |||||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||||
Doc Text: | Story Points: | --- | |||||||
Clone Of: | Environment: | ||||||||
Last Closed: | 2018-02-05 15:17:59 UTC | Type: | Bug | ||||||
Regression: | --- | Mount Type: | --- | ||||||
Documentation: | --- | CRM: | |||||||
Verified Versions: | Category: | --- | |||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||
Embargoed: | |||||||||
Bug Depends On: | |||||||||
Bug Blocks: | 910269, 1173755 | ||||||||
Attachments: |
|
Description
Richard W.M. Jones
2017-01-17 17:13:00 UTC
qemu-2.7.0-7.fc25.aarch64 Created attachment 1241897 [details]
Log file.
Created attachment 1241899 [details]
guestfs-6e3g0js3v3zp0l1e.log
qemu command line and error
Not surprisingly, this can be reproduced by creating an aarch64 guest on an x86 host. switching back to virtio-mmio addresses of course eliminates the error. The problem is -machine virt-2.7,...,gic-version=3 libvirt automatically adds <feature> <gic version='3'/> </feature> to the guest XML passed in by libguestfs because QEMU advertises emulated GICv3 support: $ /usr/libexec/qemu-kvm -M none -qmp stdio {"QMP": {"version": {"qemu": {"micro": 0, "minor": 8, "major": 2}, "package": "(qemu-kvm-rhev-2.8.0-2.el7)"}, "capabilities": []}} {"execute": "qmp_capabilities"} {"return": {}} {"execute": "query-gic-capabilities"} {"return": [{"emulated": true, "version": 3, "kernel": false}, {"emulated": true, "version": 2, "kernel": true}]} but apparently the emulated GICv3, unlike the in-kernel one, doesn't support MSI. Adding <feature> <gic version='2'/> </feature> to the guest XML when using TCG should work around the issue without having to give up PCI, but the proper way to fix it would be for QEMU to grow MSI support in the emulated GICv3. A minimal reproducer that doesn't require libguestfs or libvirt is: $ /usr/libexec/qemu-kvm \ -nographic -nodefaults \ -M virt,accel=tcg,gic-version=3 \ -device ioh3420 Reassigning to qemu-kvm-rhev. The MSI controller used along with GICv3 is the GICv3 ITS. Pure QEMU emulated model is not supported at the moment. Only KVM ITS in-kernel model is supported. If this is an important feature this is a task I could carry on. In case you use GICv2, MSI should be supported since the MSI controller being used is different: it is a GICv2M. Thanks Eric (In reply to Eric Auger from comment #6) > The MSI controller used along with GICv3 is the GICv3 ITS. Pure QEMU > emulated model is not supported at the moment. Only KVM ITS in-kernel model > is supported. Is it though? $ /usr/libexec/qemu-kvm \ -nographic -nodefaults \ -M virt,accel=kvm,gic-version=3 -cpu host \ -device ioh3420 qemu-kvm: -device ioh3420: MSI is not supported by interrupt controller qemu-kvm: -device ioh3420: Device initialization failed This is on a GICv3-equipped ThunderX host. I was under the impression such a combination would work, but that doesn't seem to be the case after all. It should but you need at least a 4.8 kernel (in-kernel ITS) and a QEMU 2.8 (ITS KVM device + related ACPI tables) Thanks Eric (In reply to Eric Auger from comment #8) > It should but you need at least a 4.8 kernel (in-kernel ITS) and a QEMU 2.8 > (ITS KVM device + related ACPI tables) I tried with kernel 4.9, QEMU master and libvirt 3.0.0 on a GICv3 machine and I didn't have any problem running a guest, so it seems the KVM situation will sort itself out as newer packages make it into distributions. For TCG guests though, we'll have to either stick with the emulated GICv2 or implement MSI support in the emulated GICv3. Eric, any update on the progress? Bug 1449837 has been reported yesterday and it's an example of people being affected by the lack of GICv3 MSI controller emulation in QEMU. See my comment there for how we will probably end up working around the issue in the short run. Job not started as of now as I work on the smmu emulation model at the moment. Developing the emulation code for ITS is a huge work and we should not expect this to come soon. Okay, thanks for the update. When the MSI controller will eventually be implemented, will it show up in the device list like arm-gicv2m already does? I'm asking because libvirt will want to decide between using GICv2 and GICv3 based on the availability of the MSI controller, so it will need a way to figure out whether the QEMU binary supports it, and it showing up in the device list would do the trick. I guess so. I don't see any reason why it wouldn't. No longer a bug? If the reproducer in comment 5 is correct, then I believe the bug still exists even upstream: $ /usr/bin/qemu-system-aarch64 -nographic -nodefaults -M virt,accel=tcg,gic-version=3 -device ioh3420 qemu-system-aarch64: -device ioh3420: MSI is not supported by interrupt controller Package: qemu-system-aarch64-core-2.10.0-0.4.rc4.fc28.aarch64 Host: Linux mustang.home.annexia.org 4.13.0-0.rc6.git0.1.fc28.aarch64 #1 SMP Mon Aug 21 14:16:58 UTC 2017 aarch64 aarch64 aarch64 GNU/Linux For completeness however I also tried the reproducer on RHEL 7.4 with the batcave repo enabled: $ /usr/libexec/qemu-kvm \ > -nographic -nodefaults \ > -M virt,accel=tcg,gic-version=3 \ > -device ioh3420 qemu-kvm: -device ioh3420: MSI is not supported by interrupt controller Package: qemu-kvm-rhev-2.9.0-22.el7a.aarch64 Host: Linux harm.home.annexia.org 4.5.0-15.4.2.el7.aarch64 #1 SMP Wed Mar 22 15:58:27 EDT 2017 aarch64 aarch64 aarch64 GNU/Linux Nothing has changed on QEMU side and there is still no emulated device for ITS. So if you launch qemu directly with virt,gic_version=3 that's normal you still hit the issue. I think the problem was worked around at libvirt level by choosing by default gic_version=2 and hence using gicv2m MSI controller instead. This latter has some emulated code. For reference, the libvirt bug used to track the workaround is Bug 1450433. We decided to close this BZ as there is a workaround at libvirt level, consisting in using GICv2 + GICv2m MSI controller and it is TCG only. The main limitation when using GICv2m is number of vcpus is capped at 8. |