Description of problem: If the VM was live migrated, there will be virt-launcher pods with the status "completed". ~~~ # oc get pod NAME READY STATUS RESTARTS AGE virt-launcher-rhel8-i181f1yxkfqqrv83-75lxb 1/1 Running 0 55s virt-launcher-rhel8-i181f1yxkfqqrv83-bn728 0/1 Completed 0 86s virt-launcher-rhel8-i181f1yxkfqqrv83-lmlqv 0/1 Completed 0 29h virt-launcher-rhel8-i181f1yxkfqqrv83-lzcwx 0/1 Completed 0 22h virt-launcher-rhel8-i181f1yxkfqqrv83-v5lvb 0/1 Completed 0 73s ~~~ must-gather is trying to do `oc exec` on all these completed pods which will fail. This will result in multiple empty files in must gather. ~~~ nijin-cnv]# ls -l vms/rhel8-i181f1yxkfqqrv83/ total 44 drwxr-xr-x. 2 root root 6 Jun 21 17:39 log -rw-------. 1 root root 5206 Jun 21 17:34 nijin-cnv_rhel8-i181f1yxkfqqrv83.log -rw-r--r--. 1 root root 1133 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-75lxb.bridge.txt -rw-r--r--. 1 root root 8330 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-75lxb.dumpxml.xml -rw-r--r--. 1 root root 1287 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-75lxb.ip.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-75lxb.ruletables.txt -rw-r--r--. 1 root root 263 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-bn728.bridge.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-bn728.dumpxml.xml -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-bn728.ip.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-bn728.ruletables.txt -rw-r--r--. 1 root root 263 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lmlqv.bridge.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lmlqv.dumpxml.xml -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lmlqv.ip.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lmlqv.ruletables.txt -rw-r--r--. 1 root root 263 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lzcwx.bridge.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lzcwx.dumpxml.xml -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lzcwx.ip.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-lzcwx.ruletables.txt -rw-r--r--. 1 root root 263 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-v5lvb.bridge.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-v5lvb.dumpxml.xml -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-v5lvb.ip.txt -rw-r--r--. 1 root root 0 Jun 21 17:39 virt-launcher-rhel8-i181f1yxkfqqrv83-v5lvb.ruletables.txt ~~~ Version-Release number of selected component (if applicable): OpenShift Virtualization 4.13.1 How reproducible: 100% Steps to Reproduce: 1. Migrate VMs multiple times so that we will have virt-launcher pods with "completed" status. 2. Collect a must-gather and list the files under namespace/<namespace-name>/vms/<vm-name> There will be multiple empty files under it. Actual results: must-gather: Multiple empty files under vms/<vm-name> if the VM was live migrated. Expected results: it should only do `oc exec` on running virt-launcher pods. Additional info:
QA: Tested fix version CNV-v4.14.0.rhel9-1279 created a VM and executed live migration multiple times: [cloud-user@ocp-psi-executor log]$ oc get virtualmachineinstancemigrations NAME PHASE VMI fedora-middle-jay-migration-7e1go Succeeded fedora-middle-jay fedora-middle-jay-migration-8unuc Succeeded fedora-middle-jay fedora-middle-jay-migration-duk2u Succeeded fedora-middle-jay fedora-middle-jay-migration-wcw52 Succeeded fedora-middle-jay rhel9-orthodox-dormouse-migration-kcyil Succeeded rhel9-orthodox-dormouse [cloud-user@ocp-psi-executor fedora-middle-jay]$ oc get pods NAME READY STATUS RESTARTS AGE virt-launcher-fedora-middle-jay-5z6hs 0/1 Completed 0 4h11m virt-launcher-fedora-middle-jay-f4485 0/1 Completed 0 4h47m virt-launcher-fedora-middle-jay-rnr67 0/1 Completed 0 4h20m virt-launcher-fedora-middle-jay-tm5h2 1/1 Running 0 163m virt-launcher-fedora-middle-jay-wj6xj 0/1 Completed 0 4h19m must_gather results under namespace/<namespace-name>/vms/<vm-name>: [cloud-user@ocp-psi-executor fedora-middle-jay]$ ls -l total 32 -rw-------. 1 cloud-user cloud-user 6812 Jul 20 07:58 default_fedora-middle-jay.log drwxr-sr-x. 2 cloud-user cloud-user 6 Jul 20 10:41 log -rw-r--r--. 1 cloud-user cloud-user 1202 Jul 20 10:41 virt-launcher-fedora-middle-jay-tm5h2.bridge.txt -rw-r--r--. 1 cloud-user cloud-user 10022 Jul 20 10:41 virt-launcher-fedora-middle-jay-tm5h2.dumpxml.xml -rw-r--r--. 1 cloud-user cloud-user 1286 Jul 20 10:41 virt-launcher-fedora-middle-jay-tm5h2.ip.txt -rw-r--r--. 1 cloud-user cloud-user 1655 Jul 20 10:41 virt-launcher-fedora-middle-jay-tm5h2.ruletables.txt no empty files and logs are only about running virt-launcher pods
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Important: OpenShift Virtualization 4.12.5 security and bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:4421