Created attachment 1641882 [details] old-style template namespace annotation Description of problem: The old-style template namespace annotation prevents creating VM from UI. $ oc logs virt-template-validator-7448b5dd85-5mvhn {"component":"kubevirt-template-validator","level":"warning","msg":"VM test1 has old-style template namespace annotation 'vm.kubevirt.io/template-namespace', should be updated to 'vm.kubevirt.io/template.namespace'","pos":"uplink.go:51","timestamp":"2019-12-04T05:28:15.758828Z"} Version-Release number of selected component (if applicable): 4.3.0-0.nightly-2019-11-28-103851 How reproducible: 100% Steps to Reproduce: 1. Create VM from UI 2. 3. Actual results: Expected results: Additional info: Change "vm.kubevirt.io/template-namespace" to "vm.kubevirt.io/template.namespace", use this new yaml content can create the VM.
Pleas ignore the Additional info.
(In reply to Guohua Ouyang from comment #0) > Created attachment 1641882 [details] > old-style template namespace annotation > > Description of problem: > The old-style template namespace annotation prevents creating VM from UI. > > $ oc logs virt-template-validator-7448b5dd85-5mvhn > > {"component":"kubevirt-template-validator","level":"warning","msg":"VM test1 > has old-style template namespace annotation > 'vm.kubevirt.io/template-namespace', should be updated to > 'vm.kubevirt.io/template.namespace'","pos":"uplink.go:51","timestamp":"2019- > 12-04T05:28:15.758828Z"} This is a warning and a reminder the annotation should be updated, and should not prevent the creation of the VM (otherwise it's a validator bug). See: https://github.com/fromanirh/kubevirt-template-validator/blob/master/pkg/webhooks/validating/uplink.go#L41 https://github.com/fromanirh/kubevirt-template-validator/commit/f61516c1277274bc0e577128a769fa83207b2dd9 There must be something else that causes the real error
Could you check the attachment to see if any clues or does it need some other logs?
(In reply to Guohua Ouyang from comment #3) > Could you check the attachment to see if any clues or does it need some > other logs? Yes, the error message is clear (and comes from kubernetes infra): let's check the templates, it seems there is an error there.
A failed example yaml. copied from the result page: apiVersion: kubevirt.io/v1alpha3 kind: VirtualMachine metadata: labels: app: vm1 vm.kubevirt.io/template: centos6-server-small-v0.7.0 vm.kubevirt.io/template.revision: '1' vm.kubevirt.io/template.version: v0.8.0 flavor.template.kubevirt.io/small: 'true' os.template.kubevirt.io/centos6.7: 'true' workload.template.kubevirt.io/server: 'true' vm.kubevirt.io/template-namespace: openshift name: vm1 annotations: name.os.template.kubevirt.io/centos6.7: CentOS 6.7 namespace: default spec: running: false template: metadata: labels: kubevirt.io/domain: vm1 kubevirt.io/size: small vm.kubevirt.io/name: vm1 spec: domain: cpu: cores: 1 sockets: 1 threads: 1 devices: disks: - bootOrder: 1 disk: bus: virtio name: disk0 interfaces: - bootOrder: 2 masquerade: {} model: virtio name: nic0 networkInterfaceMultiqueue: true rng: {} resources: requests: memory: 2G evictionStrategy: LiveMigrate hostname: vm1 networks: - name: nic0 pod: {} terminationGracePeriodSeconds: 0 volumes: - name: disk0 persistentVolumeClaim: claimName: upload-pvc
After playing with it for a while, this is what I found: If I have the annotation: vm.kubevirt.io/template-namespace: openshift present, it fails with: admission webhook "virt-template-admission.kubevirt.io" denied the request: invalid character ']' looking for beginning of value If I have the annotation: vm.kubevirt.io/template.namespace: openshift present, it fails with: If I remove the annotation, the validation passes. Attaching the template to where this annotation points to
Created attachment 1641934 [details] template which the VM points to
moving to ssp for further investigation
(In reply to Tomas Jelinek from comment #6) > After playing with it for a while, this is what I found: > > If I have the annotation: > vm.kubevirt.io/template-namespace: openshift > present, it fails with: > admission webhook "virt-template-admission.kubevirt.io" denied the request: > invalid character ']' looking for beginning of value > > If I have the annotation: > vm.kubevirt.io/template.namespace: openshift > present, it fails with: sorry, seems I did not finish the sentence. I wanted to say that it fails with the same error: admission webhook "virt-template-admission.kubevirt.io" denied the request: invalid character ']' looking for beginning of value > > > If I remove the annotation, the validation passes. > Attaching the template to where this annotation points to
Looks like this: validations: | [ { "name": "minimal-required-memory", "path": "jsonpath::.spec.domain.resources.requests.memory", "rule": "integer", "message": "This VM requires more memory.", "min": 1073741824 }, ] is not valid JSON - trailing comma on the last item of an array
version changed by mistake
This PR should fix it: https://github.com/kubevirt/common-templates/pull/114
@tomas, we need to correct the label "vm.kubevirt.io/template-namespace" on console side, does it need a new bug?
(In reply to Guohua Ouyang from comment #17) > @tomas, we need to correct the label "vm.kubevirt.io/template-namespace" on > console side, does it need a new bug? we should, eventually. So far it is only a warning but the API accepts both versions (template-namespace and template.namespace), so not a 4.3 material. But in 4.4 timeframe we should adapt. So please open a low priority bug on the console kubevirt plugin.
all patches merged -> MODIFIED
Great, thanks, Francesco!
The root-cause is identified correctly but other templates needs to be fixed as well. So moving back to Assigned. Please note, simple workaround on a deployed environment consisting of sort of: $ oc edit template centos6-server-tiny-v0.7.0 -n openshift -o yaml # and removing the trailing comma is not sufficient since something (operator??) keeps replacing this manually updated object with the original (broken) one.
Marek this PR (https://github.com/kubevirt/common-templates/pull/114) will fix all templates, not only centos6. SSP operator is replacing them as you mentioned. But this can be stopped by https://github.com/MarSik/kubevirt-ssp-operator/blob/master/TROUBLESHOOTING.md. We are currently working on new release of ssp operator, which should fix this.
Karel, I tried to stop the the SSP operator, as can be seen bellow, however, it seems that the wrong "," is keep put back in oc edit template centos6-server-tiny-v0.7.0 -n openshift -o yaml Is there something missed here please? oc -n openshift-cnv edit KubevirtTemplateValidator template-validator-hyperconverged-cluster # Please edit the object below. Lines beginning with a '#' will be ignored, # and an empty file will abort the edit. If an error occurs while saving this file will be # reopened with the relevant failures. # apiVersion: kubevirt.io/v1 kind: KubevirtTemplateValidator metadata: annotations: kubevirt.io/operator.paused: "true" creationTimestamp: "2019-12-04T10:57:17Z" generation: 1 labels: app: hyperconverged-cluster name: template-validator-hyperconverged-cluster namespace: openshift-cnv ownerReferences: - apiVersion: hco.kubevirt.io/v1alpha1 blockOwnerDeletion: true controller: true kind: HyperConverged name: hyperconverged-cluster uid: 3fa33cbd-0d68-43ae-809f-633974e592da resourceVersion: "3545462" selfLink: /apis/kubevirt.io/v1/namespaces/openshift-cnv/kubevirttemplatevalidators/template-validator-hyperconverged-cluster uid: 1384fec2-bbc0-4375-8aae-b1dfc1eb2dfb spec: {} status: conditions: - lastTransitionTime: "2019-12-04T10:58:20Z" message: Template-validator is progressing. reason: progressing status: "False" type: Progressing - lastTransitionTime: "2019-12-04T10:58:03Z" message: Template-validator is available. reason: available status: "True"
You have to stop it in KubevirtCommonTemplatesBundle not in KubevirtTemplateValidator CR. This bug is fixed in kubevirt-ssp-operator-container-v2.2.0-14.
Verified on "kubeVirtVersion":"v2.2.0-10" IMAGE_REFERENCE: registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-ssp-operator:v2.2.0-14 The comma that used to be located after validation is not longer there for: $ oc edit template rhel7-server-small-v0.7.0 -n openshift This was blocking us from running import VM, and now we can import VM successfully.
confirm the issue is fixed on kubevirt-ssp-operator:v2.2.0-14. $ oc get template rhel7-server-small-v0.7.0 -n openshift -o yaml | grep validation -A 10 validations: | [ { "name": "minimal-required-memory", "path": "jsonpath::.spec.domain.resources.requests.memory", "rule": "integer", "message": "This VM requires more memory.", "min": 1073741824 } ] creationTimestamp: "2019-12-10T01:13:01Z" And create VM from UI is okay.
closing this bug manually; it should have been included in https://access.redhat.com/errata/RHEA-2020:0307