Bug 1974289

Summary: Pending VMIs when creating concurrent bulk of VMs backed by WFFC DVs
Product: Container Native Virtualization (CNV) Reporter: Alex Kalenyuk <akalenyu>
Component: StorageAssignee: Alex Kalenyuk <akalenyu>
Status: CLOSED ERRATA QA Contact: dalia <dafrank>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.8.0CC: cnv-qe-bugs, yadu, ycui
Target Milestone: ---   
Target Release: 4.8.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: virt-controller v4.8.1-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-08-24 12:49:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Alex Kalenyuk 2021-06-21 09:57:11 UTC
Description of problem:
When concurrently creating a bulk of VMs that are backed by WFFC DVs,
it is possible that an actual virt-launcher won't get created (VMI will remain Pending)

Version-Release number of selected component (if applicable):
CNV 4.8.0

How reproducible:
100% with 10 VMs

Steps to Reproduce:
1. Create a bulk of VMs backed by WFFC DVs (script below)

Actual results:
At least VMI is 'Pending'

Expected results:
All VMIs 'Running'

Additional info:

#!/bin/bash

for (( c=0; c<10; c++ ))
do
  cat << __EOF__ | oc create -f -
---
apiVersion: cdi.kubevirt.io/v1beta1
kind: DataVolume
metadata: {name: import-public-registry-quay-dv-${c}, namespace: default}
spec:
  pvc:
    accessModes: [ReadWriteOnce]
    resources:
      requests: {storage: 1Gi}
    storageClassName: hostpath-provisioner
    volumeMode: Filesystem
  source:
    registry: {url: 'docker://quay.io/kubevirt/cirros-registry-disk-demo'}
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  labels: {}
  name: import-public-registry-quay-dv-${c}
  namespace: default
spec:
  running: true
  template:
    metadata:
      labels: {kubevirt.io/domain: import-public-registry-quay-dv-${c},
        kubevirt.io/vm: import-public-registry-quay-dv-${c}}
    spec:
      domain:
        devices:
          disks:
          - disk: {bus: virtio}
            name: dv-disk
          rng: {}
        resources:
          requests: {memory: 64M}
      volumes:
      - dataVolume: {name: import-public-registry-quay-dv-${c}}
        name: dv-disk
__EOF__
done

Comment 2 Ying Cui 2021-07-14 12:27:45 UTC
Could you create a cherrypick PR to 4.8 branch? We will hold the pr till 4.8.0 release.

Comment 3 Alex Kalenyuk 2021-07-14 14:22:25 UTC
Added

Comment 4 dalia 2021-08-04 13:50:37 UTC
verified.
CDI - v1.34.1-4
virtctl - v0.41.0-65
CNV - 4.8.1

Comment 9 errata-xmlrpc 2021-08-24 12:49:09 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Virtualization 4.8.1 Images security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:3259