Bug 2011463

Summary:	cnv-must-gather-container is hanging on collection
Product:	Container Native Virtualization (CNV)	Reporter:	Jonathan Edwards <joedward>
Component:	Installation	Assignee:	Nahshon Unna-Tsameret <nunnatsa>
Status:	CLOSED ERRATA	QA Contact:	Satyajit Bulage <sbulage>
Severity:	urgent	Docs Contact:
Priority:	urgent
Version:	4.8.2	CC:	cnv-qe-bugs, dbasunag, fdeutsch, giridhar.ramaraju, nunnatsa, pelauter, sbulage, stirabos, ycui
Target Milestone:	---	Keywords:	TestCannotAutomate
Target Release:	4.8.4	Flags:	nunnatsa: needinfo+
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:	hco-bundle-registry-container-v4.8.4-20	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-01-20 17:21:21 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jonathan Edwards 2021-10-06 16:06:03 UTC

Disconnected Cluster with 800+ VMs pulling CNV must-gather container through a proxy registry

CNV must-gather hangs for a long time here and doesn't seem to complete with active VMs:
$ oc adm must-gather --image=<proxy>/container-native-virtualization/cnv-must-gather-rhel8:v4.8.2
[must-gather      ] OUT Using must-gather plug-in image: <proxy>/container-native-virtualization/cnv-must-gather-rhel8:v4.8.2
[must-gather      ] OUT namespace/openshift-must-gather-5ssk5 created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-x2kgd created
[must-gather      ] OUT pod for plug-in image <proxy>container-native-virtualization/cnv-must-gather-rhel8:v4.8.2 created
[must-gather-x8w9j] POD 2021-10-05T15:40:16.299949683Z Inspecting resource ns/openshift-cnv...
[must-gather-x8w9j] POD 2021-10-05T15:41:12.373479846Z Inspecting resource ns/kubevirt-hyperconverged...
[must-gather-x8w9j] POD 2021-10-05T15:41:12.470425609Z Inspecting resource ns/openshift-operator-lifecycle-manager...
[must-gather-x8w9j] POD 2021-10-05T15:41:20.454569154Z Inspecting resource ns/openshift-marketplace...
[must-gather-x8w9j] POD 2021-10-05T15:41:26.394835061Z Inspecting resource ns/cluster-network-addons...
[must-gather-x8w9j] POD 2021-10-05T15:41:26.498061429Z Inspecting resource ns/openshift-sdn...
[must-gather-x8w9j] POD 2021-10-05T15:41:39.799455454Z Inspecting resource ns/sriov-network-operator...
[must-gather-x8w9j] POD 2021-10-05T15:41:39.901276929Z Inspecting resource ns/kubevirt-web-ui...
[must-gather-x8w9j] POD 2021-10-05T15:41:39.999049443Z Inspecting resource catalogsource...
[must-gather-x8w9j] POD 2021-10-05T15:41:40.125760274Z Inspecting resource clusterserviceversion...
[must-gather-x8w9j] POD 2021-10-05T15:41:40.387470450Z Inspecting resource virtualmachineinstances...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.457467862Z Inspecting resource virtualmachineinstancereplicasets...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.582086997Z Inspecting resource virtualmachineinstancepresets...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.713515435Z Inspecting resource virtualmachineinstancemigrations...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.846747532Z Inspecting resource v2vvmwares...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.000414601Z Inspecting resource virtualmachineimports...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.133910674Z Inspecting resource Network-attachment-definitions...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.263851558Z Inspecting resource networkaddonsconfig...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.393192747Z Inspecting resource ns/cdi...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.490933854Z Inspecting resource datavolumes...
[must-gather-x8w9j] POD 2021-10-05T15:41:45.923284313Z Inspecting resource nodenetworkstates...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.213452916Z Inspecting resource nodenetworkconfigurationenactments...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.378493448Z Inspecting resource nodenetworkconfigurationpolicies...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.570024717Z Inspecting resource istag...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.743161333Z Inspecting resource nodes...
[must-gather-x8w9j] POD 2021-10-05T15:41:47.120223937Z Inspecting resource machines...

Comment 1 Nahshon Unna-Tsameret 2021-10-10 11:33:35 UTC

This PR https://github.com/kubevirt/must-gather/pull/96 stops collecting the VM information by default. Instead, it offers to directly collect VMs info using a dedicated script. The script support targeted gathering so it is possible to collect ifo for all the VMs, only in a specific namespace or a specific VM.

Comment 5 Debarati Basu-Nag 2021-12-23 18:58:23 UTC

@sbulage, can you please verify this bug?

Comment 6 Satyajit Bulage 2021-12-24 07:47:44 UTC

I will work on it.

Comment 19 errata-xmlrpc 2022-01-20 17:21:21 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Virtualization 4.8.4 Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0213