Fedora Account System
Red Hat Associate
Red Hat Customer
Description of problem: Add template validation rule which should be skipped if a field is missing. { "name": "core-limits", "valid": "jsonpath::.spec.domain.cpu.cores", "path": "jsonpath::.spec.domain.cpu.cores", "rule": "integer", "message": "cpu cores must be limited", "min": 1, "max": 8 } If a VM is created without the validated field (spec.domain.cpu.cores), template validator does not skip the rule and VM creation fails. Version-Release number of selected component (if applicable): CNV 2.4.1 Template validator 2.4.1-2 How reproducible: 100% Steps to Reproduce: 1. Create a custom template 2. Add validation rule on a field (for example: cpu.cores) 3. Create a VM from the template (VM should not have cpu.cores) Actual results: VM creation fails The request is invalid: .spec.domain.cpu.cores: cpu cores must be limited: Some of values [0] are not in interval [1, 8] {"component":"kubevirt-template-validator","level":"info","msg":"evalution summary for win-16-1:\nminimal-required-memory applied: OK, All values [4294967296] are in interval [536870912, N/A]\nwindows-virtio-bus SKIPPED: not appliable\nwindows-disk-bus SKIPPED: not appliable\nwindows-cd-bus SKIPPED: not appliable\ncore-limits applied: FAIL, Some of values [0] are not in interval [1, 8]\n\nsucceeded=false","pos":"admission.go:42","timestamp":"2020-08-20T10:10:26.104309Z"} When the rule contains wildcard usage ("valid": "jsonpath::.spec.domain.cpu.cores[*]") The rule is skipped, however wildcards should apply on slices (i.e items with fields) and not fields. Expected results: VM creation should succeed, the rule should be skipped: {"component":"kubevirt-template-validator","level":"info","msg":"evalution summary for win-16-1:\nminimal-required-memory applied: OK, All values [4294967296] are in interval [536870912, N/A]\nwindows-virtio-bus SKIPPED: not appliable\nwindows-disk-bus SKIPPED: not appliable\nwindows-cd-bus SKIPPED: not appliable\ncore-limits SKIPPED: not appliable\n\nsucceeded=true","pos":"admission.go:42","timestamp":"2020-08-20T10:09:31.777678Z"} Additional info: ===================================================== $ oc get -n openshift template -n openshift windows-server-medium-v0.11.3-new -oyaml apiVersion: template.openshift.io/v1 kind: Template metadata: annotations: defaults.template.kubevirt.io/disk: rootdisk defaults.template.kubevirt.io/network: default description: Template for Microsoft Windows Server 2012 R2 VM or newer. A PVC with the Windows disk image must be available. iconClass: icon-windows name.os.template.kubevirt.io/win2k12r2: Microsoft Windows Server 2012 R2 name.os.template.kubevirt.io/win2k16: Microsoft Windows Server 2016 name.os.template.kubevirt.io/win2k19: Microsoft Windows Server 2019 openshift.io/display-name: Microsoft Windows Server 2012 R2+ VM openshift.io/documentation-url: https://github.com/kubevirt/common-templates openshift.io/provider-display-name: KubeVirt openshift.io/support-url: https://github.com/kubevirt/common-templates/issues tags: hidden,kubevirt,virtualmachine,windows template.kubevirt.io/editable: | /objects[0].spec.template.spec.domain.cpu.cores /objects[0].spec.template.spec.domain.resources.requests.memory /objects[0].spec.template.spec.domain.devices.disks /objects[0].spec.template.spec.volumes /objects[0].spec.template.spec.networks template.kubevirt.io/version: v1alpha1 template.openshift.io/bindable: "false" validations: | [ { "name": "minimal-required-memory", "path": "jsonpath::.spec.domain.resources.requests.memory", "rule": "integer", "message": "This VM requires more memory.", "min": 536870912 }, { "name": "windows-virtio-bus", "path": "jsonpath::.spec.domain.devices.disks[*].disk.bus", "valid": "jsonpath::.spec.domain.devices.disks[*]", "rule": "enum", "message": "virto disk bus type has better performance, install virtio drivers in VM and change bus type", "values": ["virtio"], "justWarning": true }, { "name": "windows-disk-bus", "path": "jsonpath::.spec.domain.devices.disks[*].disk.bus", "valid": "jsonpath::.spec.domain.devices.disks[*]", "rule": "enum", "message": "disk bus has to be either virtio or sata", "values": ["virtio", "sata"] }, { "name": "windows-cd-bus", "path": "jsonpath::.spec.domain.devices.disks[*].cdrom.bus", "valid": "jsonpath::.spec.domain.devices.disks[*].cdrom.bus", "rule": "enum", "message": "cd bus has to be sata", "values": ["sata"] }, { "name": "core-limits", "valid": "jsonpath::.spec.domain.cpu.cores", "path": "jsonpath::.spec.domain.cpu.cores", "rule": "integer", "message": "cpu cores must be limited", "min": 1, "max": 8 } ] creationTimestamp: "2020-08-20T08:24:03Z" labels: flavor.template.kubevirt.io/medium: "true" os.template.kubevirt.io/win2k12r2: "true" os.template.kubevirt.io/win2k16: "true" os.template.kubevirt.io/win2k19: "true" template.kubevirt.io/type: base template.kubevirt.io/version: v0.11.3 workload.template.kubevirt.io/server: "true" managedFields: - apiVersion: template.openshift.io/v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:ownerReferences: .: {} k:{"uid":"e0330a3c-028d-4ff0-ae9a-baff858a44ac"}: .: {} f:apiVersion: {} f:kind: {} f:name: {} f:uid: {} manager: Swagger-Codegen operation: Update time: "2020-08-20T07:14:23Z" - apiVersion: template.openshift.io/v1 fieldsType: FieldsV1 fieldsV1: f:metadata: f:annotations: .: {} f:defaults.template.kubevirt.io/disk: {} f:defaults.template.kubevirt.io/network: {} f:description: {} f:iconClass: {} f:name.os.template.kubevirt.io/win2k12r2: {} f:name.os.template.kubevirt.io/win2k16: {} f:name.os.template.kubevirt.io/win2k19: {} f:openshift.io/display-name: {} f:openshift.io/documentation-url: {} f:openshift.io/provider-display-name: {} f:openshift.io/support-url: {} f:tags: {} f:template.kubevirt.io/editable: {} f:template.kubevirt.io/version: {} f:template.openshift.io/bindable: {} f:validations: {} f:labels: .: {} f:flavor.template.kubevirt.io/medium: {} f:os.template.kubevirt.io/win2k12r2: {} f:os.template.kubevirt.io/win2k16: {} f:os.template.kubevirt.io/win2k19: {} f:template.kubevirt.io/type: {} f:template.kubevirt.io/version: {} f:workload.template.kubevirt.io/server: {} f:objects: {} f:parameters: {} manager: oc operation: Update time: "2020-08-20T10:10:23Z" name: windows-server-medium-v0.11.3-new namespace: openshift resourceVersion: "173734" selfLink: /apis/template.openshift.io/v1/namespaces/openshift/templates/windows-server-medium-v0.11.3-new uid: b83a89ec-633e-49f6-bc62-9df58ae23c93 objects: - apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: labels: app: ${NAME} vm.kubevirt.io/template: windows-server-medium-v0.11.3-new vm.kubevirt.io/template.revision: "1" vm.kubevirt.io/template.version: v0.11.3 name: ${NAME} spec: running: false template: metadata: labels: kubevirt.io/domain: ${NAME} kubevirt.io/size: medium spec: domain: clock: timer: hpet: present: false hyperv: {} pit: tickPolicy: delay rtc: tickPolicy: catchup utc: {} cpu: cores: 1 sockets: 1 threads: 1 devices: disks: - disk: bus: sata name: rootdisk inputs: - bus: usb name: tablet type: tablet interfaces: - masquerade: {} model: e1000e name: default features: acpi: {} apic: {} hyperv: relaxed: {} spinlocks: spinlocks: 8191 vapic: {} resources: requests: memory: 4Gi evictionStrategy: LiveMigrate networks: - name: default pod: {} terminationGracePeriodSeconds: 3600 volumes: - name: rootdisk persistentVolumeClaim: claimName: ${PVCNAME} parameters: - description: VM name from: windows-[a-z0-9]{6} generate: expression name: NAME - description: Name of the PVC with the disk image name: PVCNAME required: true ===================================================== VM.yaml: apiVersion: v1 items: - apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: labels: app: win-16-1 vm.kubevirt.io/template: windows-server-medium-v0.11.3-new vm.kubevirt.io/template.namespace: openshift vm.kubevirt.io/template.revision: "1" vm.kubevirt.io/template.version: v0.11.3 name: win-16-1 spec: running: false template: metadata: labels: kubevirt.io/domain: win-16-1 kubevirt.io/size: medium spec: domain: clock: timer: hpet: present: false hyperv: {} pit: tickPolicy: delay rtc: tickPolicy: catchup utc: {} cpu: sockets: 1 threads: 1 devices: inputs: - bus: usb name: tablet type: tablet interfaces: - masquerade: {} model: e1000e name: default features: acpi: {} apic: {} hyperv: relaxed: {} spinlocks: spinlocks: 8191 vapic: {} machine: type: pc-q35-rhel8.2.0 resources: requests: memory: 4Gi evictionStrategy: LiveMigrate networks: - name: default pod: {} terminationGracePeriodSeconds: 3600 volumes: - name: rootdisk persistentVolumeClaim: claimName: win-16-1 kind: List metadata: {}
The ".spec.domain.cpu.cores" field is defined in go struct as int32, so it cannot be missing, and is always defined. By default it's value is 0, which is interpreted by kubevirt as 1. I could change the template-validator so it treats value 0 as 1 in this case. It may be confusing, if the cores is set to 0 in the yaml, but kubevirt handles it correctly.
Waiting for Andrej to verify the fix with the DS container
OCP/CNV Info: $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.6.1 True False 30h Cluster version is 4.6.1 $ oc -n openshift-cnv get csv NAME DISPLAY VERSION REPLACES PHASE kubevirt-hyperconverged-operator.v2.5.0 OpenShift Virtualization 2.5.0 kubevirt-hyperconverged-operator.v2.4.2 Succeeded $ oc -n openshift-cnv get deployments -o wide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR cdi-apiserver 1/1 1 1 29h cdi-apiserver registry.redhat.io/container-native-virtualization/virt-cdi-apiserver@sha256:1ea3e3ef6d2aef53597144431c51e8e929d7796833d638a327958ebb5d17eead cdi.kubevirt.io=cdi-apiserver cdi-deployment 1/1 1 1 29h cdi-controller registry.redhat.io/container-native-virtualization/virt-cdi-controller@sha256:73fccb5b2cbf31bdfcb7deaf4c204ad090a47a2466576bd6a981dcb4ded66173 app=containerized-data-importer cdi-operator 1/1 1 1 29h cdi-operator registry.redhat.io/container-native-virtualization/virt-cdi-operator@sha256:d0c82410e9d3a9988038f17d06bf008fab9f559942928f28e1a2fe1b5d6e5746 name=cdi-operator,operator.cdi.kubevirt.io= cdi-uploadproxy 1/1 1 1 29h cdi-uploadproxy registry.redhat.io/container-native-virtualization/virt-cdi-uploadproxy@sha256:baed09d6e85d2f714c7f2ad33298998e96e218d95df9ebed22786bd8b6084dc6 cdi.kubevirt.io=cdi-uploadproxy cluster-network-addons-operator 1/1 1 1 29h cluster-network-addons-operator registry.redhat.io/container-native-virtualization/cluster-network-addons-operator@sha256:f6758fc4831c6d9e1315d965c8580f7999c69b10366b0e91b1d318e40b125e3d name=cluster-network-addons-operator hco-operator 1/1 1 1 29h hyperconverged-cluster-operator registry.redhat.io/container-native-virtualization/hyperconverged-cluster-operator@sha256:24739d6bee8db84c916f87b6df441f03e70cc15756add2fa23bdbbc08e28afce name=hyperconverged-cluster-operator hostpath-provisioner-operator 1/1 1 1 29h hostpath-provisioner-operator registry.redhat.io/container-native-virtualization/hostpath-provisioner-rhel8-operator@sha256:aa88cfb9fa0c2cc1cc8a5f0374bcf80e9a10ab52514f698304631e605f5a8d23 name=hostpath-provisioner-operator,operator.hostpath-provisioner.kubevirt.io= kubemacpool-mac-controller-manager 1/1 1 1 29h manager registry.redhat.io/container-native-virtualization/kubemacpool@sha256:942b1becf5c9f87b7ec84ca6739235643b8cf14300a2838046daa0c7ddf74f34 control-plane=mac-controller-manager,controller-tools.k8s.io=1.0 kubevirt-ssp-operator 1/1 1 1 29h kubevirt-ssp-operator registry.redhat.io/container-native-virtualization/kubevirt-ssp-operator@sha256:a7429a5b35d9a5743766637d1dc33cab924590f9586f27c2f9b189b1723b71b8 name=kubevirt-ssp-operator nmstate-webhook 2/2 2 2 29h nmstate-webhook registry.redhat.io/container-native-virtualization/kubernetes-nmstate-handler-rhel8@sha256:8d2ee0e672391c99a05c1c9181df0844e8c2b6f5aa402a9cc921e69ca1f3570e name=nmstate-webhook node-maintenance-operator 1/1 1 1 29h node-maintenance-operator registry.redhat.io/container-native-virtualization/node-maintenance-operator@sha256:24a5a82634c94736973659d6122d4bbc5f4be94c230a5092ea6da690189a8fc4 name=node-maintenance-operator virt-api 2/2 2 2 29h virt-api registry.redhat.io/container-native-virtualization/virt-api@sha256:6f157417ce8bf18750f3abf85be07b9d0f93e80193434adce54ececd11f56727 kubevirt.io=virt-api virt-controller 2/2 2 2 29h virt-controller registry.redhat.io/container-native-virtualization/virt-controller@sha256:d274faef48d0cb018369eb9e71363e2ca0d3946f12124d80e7ea3ab32afe1ea1 kubevirt.io=virt-controller virt-operator 2/2 2 2 29h virt-operator registry.redhat.io/container-native-virtualization/virt-operator@sha256:9b238d17e4166babf4f1ca5d6c8322ce47748db2bd38ee727d1a7506abd349ab kubevirt.io=virt-operator virt-template-validator 2/2 2 2 29h webhook registry.redhat.io/container-native-virtualization/kubevirt-template-validator@sha256:2f7447dd7c2d13f1ee6167db475de71e1beb87a9b4347511e9b5be72fab57497 kubevirt.io=virt-template-validator vm-import-controller 1/1 1 1 29h vm-import-controller registry.redhat.io/container-native-virtualization/vm-import-controller-rhel8@sha256:58bbf885d80d535e8069c30e3cbe27220e7a73f54a6d6d156d3c1de3a5a9ffa7 operator.v2v.kubevirt.io=,operator.v2v.kubevirt.io/createVersion=v2.5.0,v2v.kubevirt.io=vm-import-controller vm-import-operator 1/1 1 1 29h vm-import-operator registry.redhat.io/container-native-virtualization/vm-import-operator-rhel8@sha256:ee3d3f555906f876bff06e1def77d7dc57aef8e30ecfac1e9784b3cf33519a39 name=vm-import-operator,operator.v2v.kubevirt.io= Create new template with CPU core validation: $ oc create -f template-windows-server-medium-v0.12.3-new.yaml template.template.openshift.io/windows-server-medium-v0.12.3-new created $ oc get -n openshift template windows-server-medium-v0.12.3-new windows-server-medium-v0.12.3-new Template for Microsoft Windows Server 2012 R2 VM or newer. A PVC with the Win... 4 (1 blank) 1 Create VM using new template, do not specify CPU cores: $ oc create -f vm-winserver-template-test.json virtualmachine.kubevirt.io/win-bz1870541 created Rule skipped as expected: $ oc -n openshift-cnv logs virt-template-validator-9df554c9c-htgd5 | tail ... {"component":"kubevirt-template-validator","level":"info","msg":"evalution summary for win-bz1870541:\nminimal-required-memory applied: OK, All values [8589934592] are in interval [536870912, N/A]\nwindows-virtio-bus SKIPPED: not appliable\nwindows-disk-bus SKIPPED: not appliable\nwindows-cd-bus SKIPPED: not appliable\ncore-limits SKIPPED: not appliable\n\nsucceeded=true","pos":"admission.go:45","timestamp":"2020-10-26T18:15:29.229026Z"}