Bug 1795651
Summary: | Can't start or migrate s390x domains using old (~3-4 years) machine types | ||
---|---|---|---|
Product: | [Community] Virtualization Tools | Reporter: | Christian Ehrhardt <paelzer> |
Component: | libvirt | Assignee: | Jiri Denemark <jdenemar> |
Status: | CLOSED NEXTRELEASE | QA Contact: | |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | unspecified | CC: | cohuck, dhildenb, jdenemar, libvirt-maint, tburke |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | s390x | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | libvirt-6.1.0 | Doc Type: | If docs needed, set a value |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2020-02-07 08:32:38 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: |
Description
Christian Ehrhardt
2020-01-28 14:53:06 UTC
Libvirt should only add <cpu mode='host-model'/> default CPU to the domain definition if query-machines QMP command returns "host-s390x-cpu" as default-cpu-type for the machine type used by the domain. Can you please check what QEMU started as /usr/bin/qemu-system-s390x -machine none,accel=kvm -nodefaults -nographic tells about s390-ccw-virtio-2.5 machine type when you call the query-machines QMP command? I guess it says "default-cpu-type": "host-s390x-cpu". If this is the case, we either need to fix QEMU not to advertise default CPUs for machine type for which using CPU models would not work or we need to add some kind of hack to libvirt. I'd prefer if QEMU could stop reporting the default cpu as it has the best knowledge about machine types which support CPU modeling. (In reply to Jiri Denemark from comment #1) > Libvirt should only add <cpu mode='host-model'/> default CPU to the domain > definition if query-machines QMP command returns "host-s390x-cpu" as > default-cpu-type for the machine type used by the domain. > > Can you please check what QEMU started as > > /usr/bin/qemu-system-s390x -machine none,accel=kvm -nodefaults -nographic > > tells about s390-ccw-virtio-2.5 machine type when you call the query-machines > QMP command? > > I guess it says "default-cpu-type": "host-s390x-cpu". If this is the case, we > either need to fix QEMU not to advertise default CPUs for machine type for > which using CPU models would not work or we need to add some kind of hack to > libvirt. I'd prefer if QEMU could stop reporting the default cpu as it has > the > best knowledge about machine types which support CPU modeling. Well ... "-cpu host" works just fine under any machine (even without cpu model support). So I don't think "default-cpu-type=host-s390x-cpu" is wrong for older machine types. I guess the issue here is that there is no way to identify if a QEMU machine supports CPU models - except trying to start something very basic like "-cpu z900" and getting told that it does not work. Nobody ever stumbled over this, because with the old QEMU versions/machines, I guess there weren't any real enterprise users. Migration without CPU model support is not guaranteed to work either way - especially when migrating between different HW/Hypervisors. Having that said, I don't think introducing new interfaces (e.g., exporting if a QEMU machine supports CPU models) is worth it. We could blacklist the affected QEMU machines in libvirt, but yeah, that's always hacky. Oh, and the lack of CPU model support in KVM on rather old kernels adds another level of complexity. Any QEMU machine won't be able to start anything besides "-cpu host" in case KVM misses the right interfaces. I think the discussion went already further with David explaining why we might just consider this unsupported/Won't-Fix in his opinion - but for completeness let me report the info that Jiri asked for. Just running on QMP: { "execute": "qmp_capabilities" } { "execute": "query-machines" } And reporting the section about s390-ccw-virtio-2.5 used in the reported case. Old qemu 2.5: {"name": "s390-ccw-virtio-2.5", "cpu-max": 255} New qemu 4.2: {"hotpluggable-cpus": true, "name": "s390-ccw-virtio-2.5", "numa-mem-supported": false, "default-cpu-type": "host-s390x-cpu", "cpu-max": 248, "deprecated": false} So the assumption that the old type now reports "default-cpu-type": "host-s390x-cpu" was correct. @David: while "Migration without CPU model support is not guaranteed to work either way - especially when migrating between different HW/Hypervisors" is true, it was working quite well the last few years as long as you stayed on the same machine (which is common and trivial on s390x due to the LPARs as you know). And about "weren't any real enterprise users", you know how this works on the mainframe - you have very few early adopters and if you burn them too much no one is left :-) I agree that the current QMP answer of "host-s390x-cpu" seems correct and adding a new interface definitely seems too much for this. But if a quirk in libvirt to detect the set of older machine-types to handle them differently wouldn't be too hard I'd appreciate that solution (vs a Won't Fix) for sure. Yeah, that's what I figured too. The current "host-s390x-cpu" answer is correct as using -cpu host will work and thus changing this in QEMU looks inappropriate. On the other hand, I believe libvirt is the only consumer of this API :-) Anyway, the question we need to answer is whether we should use -cpu host or -cpu $HOST_MODEL for a given machine type. Do you have a complete list of machine types which do not work with host-model? Oh, that question was for me, missed it :) Basically everything older than 2.8 s390-ccw-virtio-2.4 s390-ccw-virtio-2.5 s390-ccw-virtio-2.6 s390-ccw-virtio-2.7 + some distro forks of that (Christian might want to comment on that). The "missing KVM support" is harder to catch. It can strike with any machine (on fairly old kernels). Ack - the list above is complete for the upstream source - thanks David. On downstreams we would have to also match that to s390-ccw-virtio-xenial (and other distros might have other cases), but that obviously would not be part of an upstream commit. And further for downstreams - Ubuntu as an example - >2.8 will actually turn into 2.11 as that is the next version still available and supported. Patches sent upstream for review: https://www.redhat.com/archives/libvir-list/2020-February/msg00241.html This is now fixed by commit c6ff3d1535480ef7fac8c9911ad7715f7d0e2acb Refs: v6.0.0-322-gc6ff3d1535 Author: Jiri Denemark <jdenemar> AuthorDate: Thu Feb 6 10:22:23 2020 +0100 Commit: Jiri Denemark <jdenemar> CommitDate: Fri Feb 7 09:19:02 2020 +0100 qemu_capabilities: Disable CPU models on old s390 machine types Starting a KVM domain on s390 with old machine type (such as s390-ccw-virtio-2.5) and without any guest CPU model configured fails with CPU models are not available: KVM doesn't support CPU models QEMU error. This is cause by libvirt using host-model CPU as the default CPU based on QEMU reporting "host" CPU model as being the default one (see commit v5.9.0-402-g24d8202294: qemu: Use host-model CPU on s390 by default). However, even though both QEMU and KVM support CPU models on s390 and QEMU can give us the host-model CPU, we can't use it with old machine types which only support -cpu host. https://bugzilla.redhat.com/show_bug.cgi?id=1795651 Reported-by: Christian Ehrhardt <paelzer> Signed-off-by: Jiri Denemark <jdenemar> Reviewed-by: Ján Tomko <jtomko> Tests: - start old common type s390-ccw-virtio-2.5 - start old ubunut type s390-ccw-virtio-xenial - migrate from old installation that was pre 2.8 Comparing: a) libvirt 6.0 (6.0.0-0ubuntu2) b) libvirt 6.0 + this series (6.0.0-0ubuntu3~test1) a) failed in all cases with the expected qemu-system-s390x: CPU models are not available: KVM doesn't support CPU models b) all three cases worked fine now Special case: If I tried to start the formerly defined "breakme" domains they got added <cpu mode='host-model' check='partial'/> Therefore they now fail with: error: Failed to start domain breakme error: unsupported configuration: CPU mode 'host-model' for s390x kvm domain on s390x host is not supported by hypervisor If I undefine, and define again fromt the template as reported in the BZ <domain type='kvm'> <name>breakme</name> <memory unit='KiB'>524288</memory> <os> <type arch='s390x' machine='s390-ccw-virtio-2.5'>hvm</type> </os> </domain> I now get after define: <cpu mode='host-passthrough' check='none'/> So it detected things correctly detecting the old type. A cross check using a new type like s390-ccw-virtio-4.0 or s390-ccw-virtio-eoan worked fine and gave me the epxected <cpu mode='host-model' check='partial'/> Replying to the list with Tested-by. Thanks for the patches. |