Description of problem: VM can't be migrated if CPU model was not specified. Version-Release number of selected component (if applicable): CNV-1.3 How reproducible: Steps to Reproduce: 1. Create VM without "cpu.model" tag in yaml file 2. Run migration 3. Actual results: Migration fails, new pod has an "error" status: # oc get pod virt-launcher-vmi-migratable-46vcp -o wide NAME READY STATUS RESTARTS AGE IP NODE virt-launcher-vmi-migratable-46vcp 1/2 Error 0 7m 10.130.0.86 cnv-executor-ipinto-node2.example.com There is an error message in pod logs: {"component":"virt-launcher","level":"error","msg":"operation failed: guest CPU doesn't match specification: missing features: mpx,avx512f,avx512dq,clflushopt,clwb,avx512cd,avx512bw,avx512vl,pku,spec-ctrl,xsavec,xgetbv1","pos":"virCPUx86UpdateLive:2755","subcomponent":"libvirt","timestamp":"2018-11-02T19:32:41.765000Z"} Expected results: Migration completed, no CPU error messages in logs Additional info: Workaround - specify the CPU model in VMI, for example: spec: domain: cpu: model: Nehalem In this case new pod created well and migration completed.
Please provide kubectl get -o yaml vmi …
# kubectl get -o yaml vmi vmi-migratable apiVersion: kubevirt.io/v1alpha2 kind: VirtualMachineInstance metadata: creationTimestamp: 2018-11-05T16:20:16Z finalizers: - foregroundDeleteVirtualMachine generation: 1 labels: kubevirt.io/migrationTargetNodeName: cnv-executor-ipinto-node2.example.com kubevirt.io/nodeName: cnv-executor-ipinto-node1.example.com special: vmi-migratable name: vmi-migratable namespace: default resourceVersion: "4079601" selfLink: /apis/kubevirt.io/v1alpha2/namespaces/default/virtualmachineinstances/vmi-migratable uid: ae839663-e116-11e8-99f9-fa163ecc89c7 spec: domain: devices: disks: - disk: bus: virtio name: registrydisk volumeName: registryvolume interfaces: - bridge: {} name: default features: acpi: enabled: true firmware: uuid: 6e3b0e5b-4c9d-4dcb-a21a-89f05824b316 machine: type: q35 resources: requests: memory: 64M networks: - name: default pod: {} terminationGracePeriodSeconds: 0 volumes: - name: registryvolume registryDisk: image: kubevirt/cirros-registry-disk-demo:latest status: interfaces: - ipAddress: 10.129.0.185 migrationState: migrationUid: e858c716-e117-11e8-99f9-fa163ecc89c7 sourceNode: cnv-executor-ipinto-node1.example.com startTimestamp: 2018-11-05T16:29:18Z targetNode: cnv-executor-ipinto-node2.example.com targetNodeAddress: 10.130.0.48:43208 nodeName: cnv-executor-ipinto-node1.example.com phase: Running
Can you please attach the domxml of the VM too? Just so we know what CPU model was set by kubevirt as it might have defaulted to host-model for example.
# cat /etc/libvirt/qemu/default_vmi-migratable.xml <domain type='kvm'> <name>default_vmi-migratable</name> <uuid>0bcbe651-2a5f-4c91-8a74-11c037c23840</uuid> <metadata> <kubevirt xmlns="http://kubevirt.io"> <uid>e81913cb-f32b-11e8-8e67-fa163e46d324</uid> <graceperiod> <deletionGracePeriodSeconds>0</deletionGracePeriodSeconds> </graceperiod> </kubevirt> </metadata> <memory unit='KiB'>62500</memory> <currentMemory unit='KiB'>62500</currentMemory> <vcpu placement='static'>1</vcpu> <iothreads>1</iothreads> <sysinfo type='smbios'> <system> <entry name='uuid'>0bcbe651-2a5f-4c91-8a74-11c037c23840</entry> </system> </sysinfo> <os> <type arch='x86_64' machine='pc-q35-2.12'>hvm</type> <boot dev='hd'/> </os> <features> <acpi/> </features> <cpu mode='host-model' check='partial'> <model fallback='allow'/> </cpu> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='raw' cache='none'/> <source file='/var/run/kubevirt-ephemeral-disks/registry-disk-data/default/vmi-migratable/disk_registryvolume/disk-image.raw'/> <target dev='vda' bus='virtio'/> <alias name='ua-registrydisk'/> <address type='pci' domain='0x0000' bus='0x03' slot='0x00' function='0x0'/> </disk> <controller type='usb' index='0' model='none'/> <controller type='sata' index='0'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1f' function='0x2'/> </controller> <controller type='pci' index='0' model='pcie-root'/> <controller type='virtio-serial' index='0'> <address type='pci' domain='0x0000' bus='0x02' slot='0x00' function='0x0'/> </controller> <controller type='pci' index='1' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='1' port='0x10'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0' multifunction='on'/> </controller> <controller type='pci' index='2' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='2' port='0x11'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x1'/> </controller> <controller type='pci' index='3' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='3' port='0x12'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x2'/> </controller> <controller type='pci' index='4' model='pcie-root-port'> <model name='pcie-root-port'/> <target chassis='4' port='0x13'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x3'/> </controller> <interface type='bridge'> <mac address='0a:58:0a:81:00:2f'/> <source bridge='k6t-eth0'/> <model type='virtio'/> <alias name='ua-default'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> <serial type='unix'> <source mode='bind' path='/var/run/kubevirt-private/e81913cb-f32b-11e8-8e67-fa163e46d324/virt-serial0'/> <target type='isa-serial' port='0'> <model name='isa-serial'/> </target> </serial> <console type='unix'> <source mode='bind' path='/var/run/kubevirt-private/e81913cb-f32b-11e8-8e67-fa163e46d324/virt-serial0'/> <target type='serial' port='0'/> </console> <channel type='unix'> <target type='virtio' name='org.qemu.guest_agent.0'/> <address type='virtio-serial' controller='0' bus='0' port='1'/> </channel> <input type='mouse' bus='ps2'/> <input type='keyboard' bus='ps2'/> <graphics type='vnc' socket='/var/run/kubevirt-private/e81913cb-f32b-11e8-8e67-fa163e46d324/virt-vnc'> <listen type='socket' socket='/var/run/kubevirt-private/e81913cb-f32b-11e8-8e67-fa163e46d324/virt-vnc'/> </graphics> <video> <model type='vga' vram='16384' heads='1' primary='yes'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/> </video> <memballoon model='none'/> </devices> </domain>
Quick research showed, that adding CPU model "Nehalem" to yaml file resolved the problem: spec: domain: cpu: model: Nehalem
So the default cpu mode='host-model' means the VM uses most of the features of the node it was started at. Migrating the VM to any lesser host will end up like in your report. Model Nehalem solves that for you only because it is compatible with both of your hosts. We are working on adding support for CPU model compatibility based scheduling and expect to have it soon: https://github.com/kubevirt/kubevirt/pull/1721
Moving this to MODIFIED as PR 1721 was merged upstream before Releases 0.12 and 0.13 were branched.
Verified on latest d/s build, fixed
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0417