Description of problem: cpu feature policy do not work as expected with cpu mode=passthrough Version-Release number of selected component (if applicable): libvirt-3.9.0-2.el7.x86_64 qemu-kvm-rhev-2.9.0-16.el7_4.12.x86_64 How reproducible: 100% Steps to Reproduce: forbid 1. On host support avx 2. Edit the vm's xml with below section # virsh edit rhel7 <cpu mode='host-passthrough' check='full'> <feature policy='forbid' name='avx'/> </cpu> 3. vm can start successfully # virsh start rhel7 Domain rhel7 started # ps aux | grep rhel7 ...-machine pc-i440fx-rhel7.4.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -cpu host,avx=off... require 1. on intel host do not support cr8legacy # cat /proc/cpuinfo | grep cr8legacy ===> no output 2. # virsh dumpxml rhel7 | grep /cpu -B2 <cpu mode='host-passthrough' check='full'> <feature policy='require' name='cr8legacy'/> </cpu> 3. # virsh start rhel7 Domain rhel7 started # ps aux | grep rhel7 ...-cpu host,cr8legacy=on... Actual results: For forbid, the guest can start but it support avx. For require, the guest can start but it do not support cr8legacy. Expected results: For forbid, the guest should not start as it support avx. For require, the guest should not start as it do not support cr8legacy. Or for host-passthrough do not support require and forbid? better to document somewhere. Additional info:
This bug is going to be addressed in next major release.
Hi jiri I also found the following issue; could you please help to have a look whether it is a problem. Version: libvirt-4.5.0-29.module+el8.1.0+3567+07fd3313.x86_64 qemu-kvm-2.12.0-80.module+el8.1.0+3572+48154135.x86_64 kernel-4.18.0-109.el8.x86_64 Steps: S1: Start VM with "full" check policy and "force" feature policy for host-model mode on host does not support cr8legacy # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive |grep "<cpu" -A3 <cpu mode='host-model' check='full'> <model fallback='allow'/> <feature policy='force' name='cr8legacy'/> </cpu> # virsh start vm1 error: Failed to start domain vm1 error: operation failed: guest CPU doesn't match specification: missing features: cr8legacy I am not sure whether the VM should start successfully. but in S2, it starts successfully. S2: Start VM with "full" check policy and "force" feature policy for host-passthrough mode on host does not support cr8legacy # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive |grep "<cpu" -A2 <cpu mode='host-passthrough' check='full'> <feature policy='force' name='cr8legacy'/> </cpu> # virsh start vm1 Domain vm1 started # virsh dumpxml vm1 |grep "<cpu" -A20 <cpu mode='host-passthrough' check='full'> <feature policy='force' name='cr8legacy'/> </cpu> # ps -ef |grep vm1 qemu 31414 1 30 01:32 ? -cpu host,cr8legacy=on # virsh console vm1 # lscpu |grep cr8legacy No output What is the difference in S1 and S2?
It looks like the check='full' attribute is not actually honored for host-passthrough.
Patches sent upstream for review: https://www.redhat.com/archives/libvir-list/2020-March/msg00426.html
Fixed upstream by commit ac36a824641862dcac057c6403b27ab1e91874f5 Refs: v6.1.0-175-gac36a82464 Author: Jiri Denemark <jdenemar> AuthorDate: Mon Mar 9 14:14:04 2020 +0100 Commit: Jiri Denemark <jdenemar> CommitDate: Tue Mar 17 22:53:53 2020 +0100 cpu: Honor check='full' for host-passthrough CPUs The check attribute was completely ignored for host-passthrough CPUs even if they explicitly requested some features to be enabled. For example, a domain with the following CPU definition <cpu mode='host-passthrough' check='full'> <feature policy='require' name='svm'/> </cpu> would happily start even when 'svm' cannot be enabled. Let's call virCPUArchUpdateLive for host-passthrough CPUs with VIR_CPU_CHECK_FULL to make sure the architecture specific code can validate the provided virtual CPU against the desired definition. https://bugzilla.redhat.com/show_bug.cgi?id=1515677 Signed-off-by: Jiri Denemark <jdenemar> Reviewed-by: Ján Tomko <jtomko>
Version: libvirt-6.0.0-17.module+el8.3.0+6423+e4cb6418.x86_64 kernel-4.18.0-202.el8.x86_64 qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64 Steps: S1: Start VM with host-model cpu conf, require policy and cr8legacy cpu feature (not supported by the host) # lscpu | grep cr8legacy No output # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive | grep "<cpu" -A2 <cpu mode='host-model' check='full'> <feature policy='require' name='cr8legacy'/> </cpu> # virsh start vm1 error: Failed to start domain vm1 error: operation failed: guest CPU doesn't match specification: missing features: mpx,cr8legacy S2: Start VM with host-passthrough cpu conf, require policy and cr8legacy cpu feature (not supported by the host) ==> Not expected here. # lscpu | grep cr8legacy No output # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive | grep "<cpu" -A2 <cpu mode='host-passthrough' check='full'> <feature policy='require' name='cr8legacy'/> </cpu> # virsh start vm1 Domain vm1 started # ps -ef | grep vm1 -cpu host,cr8legacy=on S3. Upgrade libvirt and retest S2 # yum update libvirt* -y # systemctl restart libvirtd # rpm -qa libvirt libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 # lscpu | grep avx Flags: ... avx # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive | grep "<cpu" -A2 <cpu mode='host-passthrough' check='full'> <feature policy='require' name='cr8legacy'/> </cpu> # virsh start vm1 error: Failed to start domain vm1 error: operation failed: guest CPU doesn't match specification: missing features: cr8legacy ==> Passed Testing the scenario that starting VM with host-passthrough cpu conf, require policy and cr8legacy cpu feature (not supported by the host) with libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 passed.
Version: libvirt-6.0.0-17.module+el8.3.0+6423+e4cb6418.x86_64 kernel-4.18.0-202.el8.x86_64 qemu-kvm-5.0.0-0.module+el8.3.0+6620+5d5e1420.x86_64 Steps: S1: Start VM with host-model cpu conf, forbid policy and avx cpu feature (supported by the host) # lscpu | grep avx Flags: ... avx # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive | grep "<cpu" -A2 <cpu mode='host-model' check='full'> <feature policy='forbid' name='avx'/> </cpu> # virsh start vm1 error: Failed to start domain vm1 error: operation failed: guest CPU doesn't match specification: missing features: mpx S2: Start VM with host-passthrough cpu conf, forbid policy and avx cpu feature (supported by the host) ==> Not expected here. # lscpu | grep avx Flags: ... avx # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive | grep "<cpu" -A2 <cpu mode='host-passthrough' check='full'> <feature policy='forbid' name='avx'/> </cpu> # virsh start vm1 Domain vm1 started # ps -ef | grep vm1 -cpu host,avx=off S3. Upgrade libvirt and retest S2 # yum update libvirt* -y # systemctl restart libvirtd # rpm -qa libvirt libvirt-6.3.0-1.module+el8.3.0+6478+69f490bb.x86_64 # lscpu | grep avx Flags: ... avx # virsh domstate vm1 shut off # virsh dumpxml vm1 --inactive | grep "<cpu" -A2 <cpu mode='host-passthrough' check='full'> <feature policy='forbid' name='avx'/> </cpu> # virsh start vm1 Domain vm1 started # ps -ef | grep vm1 -cpu host,avx=off ==> Test result here is same with S2 which is not expected.
Hi Jiri Could you pls help to check whether the test scenario in comment 11 has been fixed? Thank you. ：）
No the scenario with policy='forbid' is not fixed by the patches created for this bug. The forbid policy is not very useful, but apparently it does not work the way it's documented. You may want to file a new bz for this.
Track the issue in comment 11 with https://bugzilla.redhat.com/show_bug.cgi?id=1840770. Based on the test results in comment 10, move this bug to be verified.
I can reproduce the bug with latest versions - libvirt-daemon-6.6.0-6.module+el8.3.0+8125+aefcf088.x86_64 qemu-kvm-5.1.0-12.module+el8.3.0+8338+cbcb1a4b.x86_64 kernel-4.18.0-240.5.el8.x86_64 In intel host without supporting cr8legacy 1. # cat /proc/cpuinfo | grep cr8legacy ===> no output 2. # virsh dumpxml vt-vm1 | grep /cpu -B2 <cpu mode='host-passthrough' check='full' migratable='on'> <feature policy='require' name='cr8legacy'/> <numa> <cell id='0' cpus='0-1' memory='5242880' unit='KiB'/> </numa> </cpu> 3. # virsh start vt-vm1 Domain vt-vm1 started # ps aux | grep vt-vm1 ...-cpu host,cr8legacy=on. From the comments above, the issue should have been fixed in latest builds. Can you please help to confirm it? Thanks.
Seems the step 1 has a little issue. cat /proc/cpuinfo |grep cr8legacy should be "cr8_legacy". So actually the machine has the "cr8_legacy"feature supported. Found another machine with latest libvirt version and tested it and the issue is fixed. # virsh start vt-vm1 error: Failed to start domain vt-vm1 error: operation failed: guest CPU doesn't match specification: missing features: cr8legacy
Right, looks like you figured it out yourself.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (virt:8.3 bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:5137