Bug 2011463 - cnv-must-gather-container is hanging on collection
Summary: cnv-must-gather-container is hanging on collection
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Installation
Version: 4.8.2
Hardware: x86_64
OS: Linux
urgent
urgent
Target Milestone: ---
: 4.8.4
Assignee: Nahshon Unna-Tsameret
QA Contact: Satyajit Bulage
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-10-06 16:06 UTC by Jonathan Edwards
Modified: 2025-04-04 13:17 UTC (History)
9 users (show)

Fixed In Version: hco-bundle-registry-container-v4.8.4-20
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-01-20 17:21:21 UTC
Target Upstream Version:
Embargoed:
nunnatsa: needinfo+


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github kubevirt must-gather pull 108 0 None Merged [Release 4.9] Adopt the bug fixes and logs from the main branch 2021-12-27 09:18:02 UTC
Github kubevirt must-gather pull 109 0 None Merged [Release 4.8] Adopt the bug fixes and logs from the main branch 2021-12-27 09:18:02 UTC
Github kubevirt must-gather pull 96 0 None Merged Add targeted gathering to collect VM information 2021-12-03 15:46:35 UTC
Red Hat Issue Tracker CNV-14392 0 None None None 2023-01-20 10:34:28 UTC
Red Hat Product Errata RHBA-2022:0213 0 None None None 2022-01-20 17:21:33 UTC

Internal Links: 2016429

Description Jonathan Edwards 2021-10-06 16:06:03 UTC
Disconnected Cluster with 800+ VMs pulling CNV must-gather container through a proxy registry

CNV must-gather hangs for a long time here and doesn't seem to complete with active VMs:
$ oc adm must-gather --image=<proxy>/container-native-virtualization/cnv-must-gather-rhel8:v4.8.2
[must-gather      ] OUT Using must-gather plug-in image: <proxy>/container-native-virtualization/cnv-must-gather-rhel8:v4.8.2
[must-gather      ] OUT namespace/openshift-must-gather-5ssk5 created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-x2kgd created
[must-gather      ] OUT pod for plug-in image <proxy>container-native-virtualization/cnv-must-gather-rhel8:v4.8.2 created
[must-gather-x8w9j] POD 2021-10-05T15:40:16.299949683Z Inspecting resource ns/openshift-cnv...
[must-gather-x8w9j] POD 2021-10-05T15:41:12.373479846Z Inspecting resource ns/kubevirt-hyperconverged...
[must-gather-x8w9j] POD 2021-10-05T15:41:12.470425609Z Inspecting resource ns/openshift-operator-lifecycle-manager...
[must-gather-x8w9j] POD 2021-10-05T15:41:20.454569154Z Inspecting resource ns/openshift-marketplace...
[must-gather-x8w9j] POD 2021-10-05T15:41:26.394835061Z Inspecting resource ns/cluster-network-addons...
[must-gather-x8w9j] POD 2021-10-05T15:41:26.498061429Z Inspecting resource ns/openshift-sdn...
[must-gather-x8w9j] POD 2021-10-05T15:41:39.799455454Z Inspecting resource ns/sriov-network-operator...
[must-gather-x8w9j] POD 2021-10-05T15:41:39.901276929Z Inspecting resource ns/kubevirt-web-ui...
[must-gather-x8w9j] POD 2021-10-05T15:41:39.999049443Z Inspecting resource catalogsource...
[must-gather-x8w9j] POD 2021-10-05T15:41:40.125760274Z Inspecting resource clusterserviceversion...
[must-gather-x8w9j] POD 2021-10-05T15:41:40.387470450Z Inspecting resource virtualmachineinstances...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.457467862Z Inspecting resource virtualmachineinstancereplicasets...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.582086997Z Inspecting resource virtualmachineinstancepresets...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.713515435Z Inspecting resource virtualmachineinstancemigrations...
[must-gather-x8w9j] POD 2021-10-05T15:41:43.846747532Z Inspecting resource v2vvmwares...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.000414601Z Inspecting resource virtualmachineimports...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.133910674Z Inspecting resource Network-attachment-definitions...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.263851558Z Inspecting resource networkaddonsconfig...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.393192747Z Inspecting resource ns/cdi...
[must-gather-x8w9j] POD 2021-10-05T15:41:44.490933854Z Inspecting resource datavolumes...
[must-gather-x8w9j] POD 2021-10-05T15:41:45.923284313Z Inspecting resource nodenetworkstates...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.213452916Z Inspecting resource nodenetworkconfigurationenactments...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.378493448Z Inspecting resource nodenetworkconfigurationpolicies...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.570024717Z Inspecting resource istag...
[must-gather-x8w9j] POD 2021-10-05T15:41:46.743161333Z Inspecting resource nodes...
[must-gather-x8w9j] POD 2021-10-05T15:41:47.120223937Z Inspecting resource machines...

Comment 1 Nahshon Unna-Tsameret 2021-10-10 11:33:35 UTC
This PR https://github.com/kubevirt/must-gather/pull/96 stops collecting the VM information by default. Instead, it offers to directly collect VMs info using a dedicated script. The script support targeted gathering so it is possible to collect ifo for all the VMs, only in a specific namespace or a specific VM.

Comment 5 Debarati Basu-Nag 2021-12-23 18:58:23 UTC
@sbulage, can you please verify this bug?

Comment 6 Satyajit Bulage 2021-12-24 07:47:44 UTC
I will work on it.

Comment 19 errata-xmlrpc 2022-01-20 17:21:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Virtualization 4.8.4 Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:0213


Note You need to log in before you can comment on or make changes to this bug.