Bug 1779504 - Cannot create vm on UI because trailing comma in common template
Summary: Cannot create vm on UI because trailing comma in common template
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: SSP
Version: 2.2.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 2.2.0
Assignee: Karel Šimon
QA Contact: Israel Pinto
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-12-04 05:39 UTC by Guohua Ouyang
Modified: 2020-01-30 20:02 UTC (History)
11 users (show)

Fixed In Version: kubevirt-ssp-operator-container-v2.2.0-14
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-01-30 20:02:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
old-style template namespace annotation (52.71 KB, image/png)
2019-12-04 05:39 UTC, Guohua Ouyang
no flags Details
template which the VM points to (4.13 KB, text/plain)
2019-12-04 08:47 UTC, Tomas Jelinek
no flags Details

Description Guohua Ouyang 2019-12-04 05:39:59 UTC
Created attachment 1641882 [details]
old-style template namespace annotation

Description of problem:
The old-style template namespace annotation prevents creating VM from UI.

$ oc logs virt-template-validator-7448b5dd85-5mvhn

{"component":"kubevirt-template-validator","level":"warning","msg":"VM test1 has old-style template namespace annotation 'vm.kubevirt.io/template-namespace', should be updated to 'vm.kubevirt.io/template.namespace'","pos":"uplink.go:51","timestamp":"2019-12-04T05:28:15.758828Z"}


Version-Release number of selected component (if applicable):
4.3.0-0.nightly-2019-11-28-103851

How reproducible:
100%

Steps to Reproduce:
1. Create VM from UI
2.
3.

Actual results:


Expected results:


Additional info:
Change "vm.kubevirt.io/template-namespace" to "vm.kubevirt.io/template.namespace", use this new yaml content can create the VM.

Comment 1 Guohua Ouyang 2019-12-04 06:38:37 UTC
Pleas ignore the Additional info.

Comment 2 Francesco Romani 2019-12-04 08:23:12 UTC
(In reply to Guohua Ouyang from comment #0)
> Created attachment 1641882 [details]
> old-style template namespace annotation
> 
> Description of problem:
> The old-style template namespace annotation prevents creating VM from UI.
> 
> $ oc logs virt-template-validator-7448b5dd85-5mvhn
> 
> {"component":"kubevirt-template-validator","level":"warning","msg":"VM test1
> has old-style template namespace annotation
> 'vm.kubevirt.io/template-namespace', should be updated to
> 'vm.kubevirt.io/template.namespace'","pos":"uplink.go:51","timestamp":"2019-
> 12-04T05:28:15.758828Z"}

This is a warning and a reminder the annotation should be updated, and should not prevent the creation of the VM (otherwise it's a validator bug). See:
https://github.com/fromanirh/kubevirt-template-validator/blob/master/pkg/webhooks/validating/uplink.go#L41
https://github.com/fromanirh/kubevirt-template-validator/commit/f61516c1277274bc0e577128a769fa83207b2dd9

There must be something else that causes the real error

Comment 3 Guohua Ouyang 2019-12-04 08:28:30 UTC
Could you check the attachment to see if any clues or does it need some other logs?

Comment 4 Francesco Romani 2019-12-04 08:32:52 UTC
(In reply to Guohua Ouyang from comment #3)
> Could you check the attachment to see if any clues or does it need some
> other logs?

Yes, the error message is clear (and comes from kubernetes infra): let's check the templates, it seems there is an error there.

Comment 5 Guohua Ouyang 2019-12-04 08:40:33 UTC
A failed example yaml. copied from the result page:

apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  labels:
    app: vm1
    vm.kubevirt.io/template: centos6-server-small-v0.7.0
    vm.kubevirt.io/template.revision: '1'
    vm.kubevirt.io/template.version: v0.8.0
    flavor.template.kubevirt.io/small: 'true'
    os.template.kubevirt.io/centos6.7: 'true'
    workload.template.kubevirt.io/server: 'true'
    vm.kubevirt.io/template-namespace: openshift
  name: vm1
  annotations:
    name.os.template.kubevirt.io/centos6.7: CentOS 6.7
  namespace: default
spec:
  running: false
  template:
    metadata:
      labels:
        kubevirt.io/domain: vm1
        kubevirt.io/size: small
        vm.kubevirt.io/name: vm1
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 1
          threads: 1
        devices:
          disks:
            - bootOrder: 1
              disk:
                bus: virtio
              name: disk0
          interfaces:
            - bootOrder: 2
              masquerade: {}
              model: virtio
              name: nic0
          networkInterfaceMultiqueue: true
          rng: {}
        resources:
          requests:
            memory: 2G
      evictionStrategy: LiveMigrate
      hostname: vm1
      networks:
        - name: nic0
          pod: {}
      terminationGracePeriodSeconds: 0
      volumes:
        - name: disk0
          persistentVolumeClaim:
            claimName: upload-pvc

Comment 6 Tomas Jelinek 2019-12-04 08:45:07 UTC
After playing with it for a while, this is what I found:

If I have the annotation:
    vm.kubevirt.io/template-namespace: openshift
present, it fails with:
admission webhook "virt-template-admission.kubevirt.io" denied the request: invalid character ']' looking for beginning of value

If I have the annotation:
    vm.kubevirt.io/template.namespace: openshift
present, it fails with:


If I remove the annotation, the validation passes.
Attaching the template to where this annotation points to

Comment 7 Tomas Jelinek 2019-12-04 08:47:04 UTC
Created attachment 1641934 [details]
template which the VM points to

Comment 8 Tomas Jelinek 2019-12-04 08:51:56 UTC
moving to ssp for further investigation

Comment 9 Tomas Jelinek 2019-12-04 08:53:34 UTC
(In reply to Tomas Jelinek from comment #6)
> After playing with it for a while, this is what I found:
> 
> If I have the annotation:
>     vm.kubevirt.io/template-namespace: openshift
> present, it fails with:
> admission webhook "virt-template-admission.kubevirt.io" denied the request:
> invalid character ']' looking for beginning of value
> 
> If I have the annotation:
>     vm.kubevirt.io/template.namespace: openshift
> present, it fails with:

sorry, seems I did not finish the sentence. I wanted to say that it fails with the same error:
admission webhook "virt-template-admission.kubevirt.io" denied the request: invalid character ']' looking for beginning of value

> 
> 
> If I remove the annotation, the validation passes.
> Attaching the template to where this annotation points to

Comment 10 Francesco Romani 2019-12-04 09:00:14 UTC
Looks like this:

    validations: |
      [
        {
          "name": "minimal-required-memory",
          "path": "jsonpath::.spec.domain.resources.requests.memory",
          "rule": "integer",
          "message": "This VM requires more memory.",
          "min": 1073741824
        },
      ]

is not valid JSON - trailing comma on the last item of an array

Comment 11 Francesco Romani 2019-12-04 09:03:42 UTC
version changed by mistake

Comment 12 Karel Šimon 2019-12-04 10:21:07 UTC
This PR should fix it: https://github.com/kubevirt/common-templates/pull/114

Comment 17 Guohua Ouyang 2019-12-04 12:10:12 UTC
@tomas, we need to correct the label "vm.kubevirt.io/template-namespace" on console side, does it need a new bug?

Comment 20 Tomas Jelinek 2019-12-04 12:29:22 UTC
(In reply to Guohua Ouyang from comment #17)
> @tomas, we need to correct the label "vm.kubevirt.io/template-namespace" on
> console side, does it need a new bug?

we should, eventually. So far it is only a warning but the API accepts both versions (template-namespace and template.namespace), so not a 4.3 material. But in 4.4 timeframe we should adapt. So please open a low priority bug on the console kubevirt plugin.

Comment 21 Francesco Romani 2019-12-04 13:11:51 UTC
all patches merged -> MODIFIED

Comment 22 Fabian Deutsch 2019-12-04 22:11:35 UTC
Great, thanks, Francesco!

Comment 23 Marek Libra 2019-12-06 08:38:38 UTC
The root-cause is identified correctly but other templates needs to be fixed as well.
So moving back to Assigned.

Please note, simple workaround on a deployed environment consisting of sort of:
   $ oc edit template centos6-server-tiny-v0.7.0 -n openshift -o yaml
   # and removing the trailing comma

is not sufficient since something (operator??) keeps replacing this manually updated object with the original (broken) one.

Comment 24 Karel Šimon 2019-12-06 08:44:28 UTC
Marek this PR (https://github.com/kubevirt/common-templates/pull/114) will fix all templates, not only centos6. SSP operator is replacing them as you mentioned. But this can be stopped by https://github.com/MarSik/kubevirt-ssp-operator/blob/master/TROUBLESHOOTING.md. We are currently working on new release of ssp operator, which should fix this.

Comment 25 Ilanit Stein 2019-12-08 11:28:35 UTC
Karel,

I tried to stop the the SSP operator, as can be seen bellow, however, it seems that the wrong "," is keep put back in 
oc edit template centos6-server-tiny-v0.7.0 -n openshift -o yaml
Is there something missed here please?

oc -n openshift-cnv edit KubevirtTemplateValidator template-validator-hyperconverged-cluster

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: kubevirt.io/v1
kind: KubevirtTemplateValidator
metadata:
  annotations:
    kubevirt.io/operator.paused: "true"
  creationTimestamp: "2019-12-04T10:57:17Z"
  generation: 1
  labels:
    app: hyperconverged-cluster
  name: template-validator-hyperconverged-cluster
  namespace: openshift-cnv
  ownerReferences:
  - apiVersion: hco.kubevirt.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: HyperConverged
    name: hyperconverged-cluster
    uid: 3fa33cbd-0d68-43ae-809f-633974e592da
  resourceVersion: "3545462"
  selfLink: /apis/kubevirt.io/v1/namespaces/openshift-cnv/kubevirttemplatevalidators/template-validator-hyperconverged-cluster
  uid: 1384fec2-bbc0-4375-8aae-b1dfc1eb2dfb
spec: {}
status:
  conditions:
  - lastTransitionTime: "2019-12-04T10:58:20Z"
    message: Template-validator is progressing.
    reason: progressing
    status: "False"
    type: Progressing
  - lastTransitionTime: "2019-12-04T10:58:03Z"
    message: Template-validator is available.
    reason: available
    status: "True"

Comment 26 Karel Šimon 2019-12-09 07:58:45 UTC
You have to stop it in KubevirtCommonTemplatesBundle not in KubevirtTemplateValidator CR. This bug is fixed in kubevirt-ssp-operator-container-v2.2.0-14.

Comment 27 Ilanit Stein 2019-12-09 14:32:06 UTC
Verified on 
"kubeVirtVersion":"v2.2.0-10"
IMAGE_REFERENCE:    registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-kubevirt-ssp-operator:v2.2.0-14

The comma that used to be located after validation is not longer there for:

$ oc edit template rhel7-server-small-v0.7.0 -n openshift
 
This was blocking us from running import VM, and now we can import VM successfully.

Comment 28 Guohua Ouyang 2019-12-10 02:29:00 UTC
confirm the issue is fixed on kubevirt-ssp-operator:v2.2.0-14.

$ oc get template rhel7-server-small-v0.7.0 -n openshift -o yaml | grep validation -A 10
    validations: |
      [
        {
          "name": "minimal-required-memory",
          "path": "jsonpath::.spec.domain.resources.requests.memory",
          "rule": "integer",
          "message": "This VM requires more memory.",
          "min": 1073741824
        }
      ]
  creationTimestamp: "2019-12-10T01:13:01Z"

And create VM from UI is okay.

Comment 29 Dan Kenigsberg 2020-01-30 20:02:48 UTC
closing this bug manually; it should have been included in

https://access.redhat.com/errata/RHEA-2020:0307


Note You need to log in before you can comment on or make changes to this bug.