Description of problem: Kubevirt labels are applied to worker nodes having no support for virtualization. Version-Release number of selected component (if applicable): How reproducible: Always Steps to Reproduce: 1. Install OCP 4.10 and CNV 4.10.0 to vSphere 2. Copy and customize RHCOS template to enable: Hardware virtualization, IOMMU, Performance Counters 3. Create a 2nd worker machineset referencing this new template 4. Scale up to 1 and check for existance of kubvirt labels on all nodes oc get nodes -l kubevirt.io/schedulable=true -L kubevirt.io/schedulable Actual results: * Many kubevirt.io labels are present on all non-infra worker nodes. E.g. schedulable. oc get nodes -l kubevirt.io/schedulable=true -L kubevirt.io/schedulable NAME STATUS ROLES AGE VERSION SCHEDULABLE hub-7vxwj-cnv-drhkz Ready worker 2d19h v1.23.5+9ce5071 true hub-7vxwj-worker-5s4fm Ready worker 80d v1.23.5+9ce5071 true hub-7vxwj-worker-9kvf4 Ready worker 54d v1.23.5+9ce5071 true hub-7vxwj-worker-dd62d Ready worker 80d v1.23.5+9ce5071 true * Only worker "hub-7vxwj-cnv-drhkz" has nested virtualization enabled. oc get nodes NAME STATUS ROLES AGE VERSION hub-7vxwj-cnv-drhkz Ready worker 2d18h v1.23.5+9ce5071 <---- Has VMX hub-7vxwj-master-0 Ready master 80d v1.23.5+9ce5071 hub-7vxwj-master-1 Ready master 80d v1.23.5+9ce5071 hub-7vxwj-master-2 Ready master 80d v1.23.5+9ce5071 hub-7vxwj-store-1-mhf5g Ready infra,worker 80d v1.23.5+9ce5071 hub-7vxwj-store-2-4xdl2 Ready infra,worker 80d v1.23.5+9ce5071 hub-7vxwj-store-3-b42rl Ready infra,worker 80d v1.23.5+9ce5071 hub-7vxwj-worker-5s4fm Ready worker 80d v1.23.5+9ce5071 hub-7vxwj-worker-9kvf4 Ready worker 54d v1.23.5+9ce5071 hub-7vxwj-worker-dd62d Ready worker 80d v1.23.5+9ce5071 * The /proc/cpuinfo confirms absence of vmx on all but 1 host for pod in $(oc get pods -n openshift-machine-config-operator -l k8s-app=machine-config-daemon -o name --no-headers); do echo "# $pod"; oc rsh -c machine-config-daemon $pod grep -m 1 -E 'vmx|svm' /proc/cpuinfo 2>/dev/null; done # pod/machine-config-daemon-2v7f4 # pod/machine-config-daemon-2vkrk # pod/machine-config-daemon-5gxx7 # pod/machine-config-daemon-92rrf # pod/machine-config-daemon-dbclr # pod/machine-config-daemon-kczbw # pod/machine-config-daemon-kx99g # pod/machine-config-daemon-msk2b # pod/machine-config-daemon-nbtx8 flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap xsaveopt arat md_clear flush_l1d arch_capabilities # pod/machine-config-daemon-vjjfg * Additionally, even though all nodes are VMs on a single Intel Xeon ESXi host the CPU detection is seemingly inconsistent oc get nodes -l cpu-vendor.node.kubevirt.io/Intel=true NAME STATUS ROLES AGE VERSION hub-7vxwj-cnv-drhkz Ready worker 2d19h v1.23.5+9ce5071 oc get nodes -l cpu-vendor.node.kubevirt.io/AMD=true NAME STATUS ROLES AGE VERSION hub-7vxwj-worker-5s4fm Ready worker 80d v1.23.5+9ce5071 hub-7vxwj-worker-9kvf4 Ready worker 54d v1.23.5+9ce5071 hub-7vxwj-worker-dd62d Ready worker 80d v1.23.5+9ce5071 Expected results: Only nodes capable of virtualization have any kubevirt.io labels, particularly 'kubevirt.io/schedulable'. Additional info:
If i understood the problem correctly it's about marking the node unschedulable when virtualization is not supported. One way to verify if virtualization is supported is to verify that `svm` or `vmx` are supported or in other words check that the output of : `egrep -o '(vmx|svm)' /proc/cpuinfo | sort | uniq` isn't empty.
More than marking the virt-incompatible nodes kubevirt unschedulable, I believe the problem is there should be no kubevirt labels applied to such nodes. I'll have to repro my past observations in a nested virt vSphere lab, but I just checked a 4.11.5 cluster on AWS with a pair of baremetal workers along side ec2 workers. I do not see this problem there. Only the metal instances have kubevirt labels.
Closing it as not a bug since this field is internal and shouldn't be used by users. Although, since it's truly confusing, I've opened this issue for changing the label's name: https://github.com/kubevirt/kubevirt/issues/8583 For more info: https://github.com/kubevirt/kubevirt/pull/8554#issuecomment-1272909490