Bug 2029343 - High performance VM fail to start on libvirt error (kvm-hint-dedicated)
Summary: High performance VM fail to start on libvirt error (kvm-hint-dedicated)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Virtualization
Version: 4.10.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 4.10.0
Assignee: Igor Bezukh
QA Contact: Kedar Bidarkar
URL:
Whiteboard:
: 2035679 (view as bug list)
Depends On: 2034942
Blocks: 1945586
TreeView+ depends on / blocked
 
Reported: 2021-12-06 09:37 UTC by Ruth Netser
Modified: 2022-03-24 16:25 UTC (History)
8 users (show)

Fixed In Version: virt-launcher-container-v4.10.0-205 hco-bundle-registry-container-v4.10.0-645
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-03-16 15:57:01 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2022:0947 0 None None None 2022-03-16 15:57:17 UTC

Description Ruth Netser 2021-12-06 09:37:49 UTC
Description of problem:
High performance VM fails to start due to:

    message: 'server error. command SyncVMI failed: "LibvirtError(Code=67, Domain=10,
      Message=''unsupported configuration: kvm-hint-dedicated=on is only applicable
      for cpu host-passthrough'')"'
    reason: Synchronizing with the Domain failed.
    status: "False"
    type: Synchronized

This was fixed in libvirt bug 1857671 (libvirt-7.3.0-1.el8)
but 4.10 currently uses 7.0.0-14


Version-Release number of selected component (if applicable):
CNV 4.10.0 (virt v4.10.0-124)


How reproducible:
100%

Steps to Reproduce:
1. Create VM from rhel high performance template (rhel8-highperformance-tiny)
2. Start the VM
3.

Actual results:
VMI fails to run
VM errors on:

    Message:               server error. command SyncVMI failed: "LibvirtError(Code=67, Domain=10, Message='unsupported configuration: kvm-hint-dedicated=on is only applicable for cpu host-passthrough')"
    Reason:                Synchronizing with the Domain failed.


Expected results:
VM should successfully start.

Additional info:

[root@high-performance-rhel-vm-1638779426-1444833 /]# rpm -qa|grep libvirt
libvirt-daemon-7.0.0-14.5.module+el8.4.0+13026+f38c77ab.x86_64
libvirt-client-7.0.0-14.5.module+el8.4.0+13026+f38c77ab.x86_64
libvirt-bash-completion-7.0.0-14.5.module+el8.4.0+13026+f38c77ab.x86_64
libvirt-libs-7.0.0-14.5.module+el8.4.0+13026+f38c77ab.x86_64
libvirt-daemon-driver-qemu-7.0.0-14.5.module+el8.4.0+13026+f38c77ab.x86_64
[root@high-performance-rhel-vm-1638779426-1444833 /]# 



$ oc get vm -n ssp-high-performance-vm-test-high-performance-templates high-performance-rhel-vm-1638779426-1444833 -oyaml
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  annotations:
    kubemacpool.io/transaction-timestamp: "2021-12-06T08:30:27.780471608Z"
    kubevirt.io/latest-observed-api-version: v1
    kubevirt.io/storage-observed-api-version: v1alpha3
    vm.kubevirt.io/validations: |
      [
        {
          "name": "minimal-required-memory",
          "path": "jsonpath::.spec.domain.resources.requests.memory",
          "rule": "integer",
          "message": "This VM requires more memory.",
          "min": 1610612736
        }
      ]
  creationTimestamp: "2021-12-06T08:30:27Z"
  generation: 2
  labels:
    app: high-performance-rhel-vm-1638779426-1444833
    vm.kubevirt.io/template: rhel8-highperformance-small
    vm.kubevirt.io/template.namespace: openshift
    vm.kubevirt.io/template.revision: "1"
    vm.kubevirt.io/template.version: v0.17.0
  name: high-performance-rhel-vm-1638779426-1444833
  namespace: ssp-high-performance-vm-test-high-performance-templates
  resourceVersion: "1589628"
  uid: 86b55e09-6856-4648-8f4f-ebea4db8dff8
spec:
  dataVolumeTemplates:
  - apiVersion: cdi.kubevirt.io/v1beta1
    kind: DataVolume
    metadata:
      creationTimestamp: null
      name: high-performance-rhel-vm-1638779426-1444833
    spec:
      source:
        pvc:
          name: rhel8-hp-vm-dv
          namespace: openshift-virtualization-os-images
      storage:
        accessModes:
        - ReadWriteMany
        resources:
          requests:
            storage: 20Gi
        storageClassName: ocs-storagecluster-ceph-rbd
        volumeMode: Block
  running: true
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: small
        vm.kubevirt.io/os: rhel8
        vm.kubevirt.io/workload: highperformance
      creationTimestamp: null
      labels:
        kubevirt.io/domain: high-performance-rhel-vm-1638779426-1444833
        kubevirt.io/size: small
        kubevirt.io/vm: high-performance-rhel-vm-1638779426-1444833
    spec:
      domain:
        cpu:
          cores: 1
          dedicatedCpuPlacement: true
          isolateEmulatorThread: true
          sockets: 1
          threads: 1
        devices:
          disks:
          - dedicatedIOThread: true
            disk:
              bus: virtio
            name: high-performance-rhel-vm-1638779426-1444833
          - disk:
              bus: virtio
            name: cloudinitdisk
          interfaces:
          - macAddress: 02:22:18:00:00:79
            masquerade: {}
            name: default
          networkInterfaceMultiqueue: true
          rng: {}
        ioThreadsPolicy: shared
        machine:
          type: pc-q35-rhel8.4.0
        resources:
          requests:
            memory: 2Gi
      evictionStrategy: LiveMigrate
      networks:
      - name: default
        pod: {}
      terminationGracePeriodSeconds: 180
      volumes:
      - dataVolume:
          name: high-performance-rhel-vm-1638779426-1444833
        name: high-performance-rhel-vm-1638779426-1444833
      - cloudInitNoCloud:
          userData: |-
            #cloud-config
            user: cloud-user
            password: password
            chpasswd: { expire: False }
            ssh_authorized_keys:
             [ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj47ubVnxR16JU7ZfDli3N5QVBAwJBRh2xMryyjk5dtfugo5JIPGB2cyXTqEDdzuRmI+Vkb/A5duJyBRlA+9RndGGmhhMnj8and3wu5/cEb7DkF6ZJ25QV4LQx3K/i57LStUHXRTvruHOZ2nCuVXWqi7wSvz5YcvEv7O8pNF5uGmqHlShBdxQxcjurXACZ1YY0YDJDr3AJai1KF9zehVJODuSbrnOYpThVWGjFuFAnNxbtuZ8EOSougN2aYTf2qr/KFGDHtewIkzZmP6cjzKO5bN3pVbXxmb2Gces/BYHntY4MXBTUqwsmsCRC5SAz14bEP/vsLtrNhjq9vCS+BjMT root]
            runcmd: ["sudo sed -i '/^PubkeyAccepted/ s/$/,ssh-rsa/' /etc/crypto-policies/back-ends/opensshserver.config", "sudo sed -i 's/^#\\?PasswordAuthentication no/PasswordAuthentication yes/g' /etc/ssh/sshd_config", 'sudo systemctl enable sshd', 'sudo systemctl restart sshd']
        name: cloudinitdisk
status:
  conditions:
  - lastProbeTime: "2021-12-06T09:31:29Z"
    lastTransitionTime: "2021-12-06T09:31:29Z"
    message: Guest VM is not reported as running
    reason: GuestNotRunning
    status: "False"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: null
    status: "True"
    type: LiveMigratable
  - lastProbeTime: null
    lastTransitionTime: "2021-12-06T09:31:52Z"
    message: 'server error. command SyncVMI failed: "LibvirtError(Code=67, Domain=10,
      Message=''unsupported configuration: kvm-hint-dedicated=on is only applicable
      for cpu host-passthrough'')"'
    reason: Synchronizing with the Domain failed.
    status: "False"
    type: Synchronized
  created: true
  printableStatus: Starting
  startFailure:
    consecutiveFailCount: 7
    lastFailedVMIUID: 67e2fb42-82c5-4852-b006-1483c2a3f200
    retryAfterTimestamp: "2021-12-06T09:31:29Z"
  volumeSnapshotStatuses:
  - enabled: true
    name: high-performance-rhel-vm-1638779426-1444833
  - enabled: false
    name: cloudinitdisk
    reason: Snapshot is not supported for this volumeSource type [cloudinitdisk]

Comment 1 Kedar Bidarkar 2021-12-06 09:47:01 UTC
RHEL-AV Bug related to this issue, https://bugzilla.redhat.com/show_bug.cgi?id=1857671

Comment 2 sgott 2021-12-06 17:06:40 UTC
I'm trying to understand how it is that we didn't spot this before now. This worked in the past? Were we testing this configuration?

Understood that the libvirt error appears, but there's also a warning about the VM not having enough memory.

Comment 3 sgott 2021-12-06 17:09:59 UTC
Michal,

It appears that we're running into an error that was fixed in 8.5 (Per Comment #1). Is it this fix a candidate for backporting to 8.4? If so, what's the process we need to follow?

Comment 4 sgott 2021-12-06 17:21:26 UTC
We're using the identical libvirt versions for both 4.9 and 4.10: 7.0.0-14

So the mystery to me is how did the previous version pass where this one fails?

Comment 5 Ruth Netser 2021-12-06 18:22:20 UTC
@sgott this works in 4.9.1.
Attaching VM spec from both versions.

Comment 14 Kedar Bidarkar 2021-12-29 14:01:11 UTC
*** Bug 2035679 has been marked as a duplicate of this bug. ***

Comment 22 sgott 2022-01-31 21:08:30 UTC
libvirt has now shipped a fix for this. Moving BZ to MODIFIED as we should pick this up with the next images we build.

Comment 24 Kedar Bidarkar 2022-02-04 20:01:06 UTC
Both RHEL and Windows related High Performance Template Related tests have PASSED.

VERIFIED With v4.10.0-648

Comment 29 errata-xmlrpc 2022-03-16 15:57:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Virtualization 4.10.0 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0947


Note You need to log in before you can comment on or make changes to this bug.