Bug 1878153
Summary: | OCS 4.6 must-gather: collect node information under cluster_scoped_resources/oc_output directory | ||||||
---|---|---|---|---|---|---|---|
Product: | [Red Hat Storage] Red Hat OpenShift Container Storage | Reporter: | Neha Berry <nberry> | ||||
Component: | must-gather | Assignee: | Pulkit Kundra <pkundra> | ||||
Status: | CLOSED ERRATA | QA Contact: | Neha Berry <nberry> | ||||
Severity: | high | Docs Contact: | |||||
Priority: | unspecified | ||||||
Version: | 4.6 | CC: | assingh, madam, muagarwa, ocs-bugs, pkundra, sabose | ||||
Target Milestone: | --- | Keywords: | AutomationBackLog | ||||
Target Release: | OCS 4.6.0 | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | 4.6.0-142.ci | Doc Type: | No Doc Update | ||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2020-12-17 06:24:14 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
Neha Berry
2020-09-11 13:45:19 UTC
Hi Pulkit, Do you plan to fix this BZ in OCS 4.6 ? IIRC from our offline discussion, it was a small change in moving the file to another folder. Let me know. *** Bug 1890216 has been marked as a duplicate of this bug. *** Created attachment 1723268 [details] terminal output from must-gather Checked in the latest OCS 4.6.0-137.ci build and the command to collect "oc get nodes --show-labels" is failing. a) the oc get nodes --show-labels collection is failing and is neither collected under namespaces/openshift-storage/oc_output (original) , nor under cluster-scoped-resources/oc_output/ [must-gather-gn74s] POD collecting oc command sc [must-gather-gn74s] POD collecting oc command nodes -o wide --show-labels >> [must-gather-gn74s] POD error: the server doesn't have a resource type "nodes -o wide --show-labels" [must-gather-gn74s] POD collecting oc command clusterversion [must-gather-gn74s] POD collecting oc command infrastructures.config Sample must-gather from OCS 4.6 internal mode cluster http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/1890183/must-gather.local.7263335647167393572/quay-io-rhceph-dev-ocs-must-gather-sha256-3255bdcfc54ce04e8b0b948cc2d6e4ba5e7fbd2ca14dc8512d5c845e4a9ae157/ Version of all relevant components (if applicable): -------------------------------------------- OCS = ocs-operator.v4.6.0-137.ci the command actually works on the cluster, but seems to be failing during must-gather $ oc get nodes -o wide --show-labels NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME LABELS argo002.ceph.redhat.com Ready master 9h v1.19.0+d59ce34 10.8.128.202 <none> Red Hat Enterprise Linux CoreOS 46.82.202010201440-0 (Ootpa) 4.18.0-193.28.1.el8_2.x86_64 cri-o://1.19.0-22.rhaos4.6.gitc0306f1.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=argo002.ceph.redhat.com,kubernetes.io/os=linux,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos argo003.ceph.redhat.com Ready master 9h v1.19.0+d59ce34 10.8.128.203 <none> Red Hat Enterprise Linux CoreOS 46.82.202010201440-0 (Ootpa) 4.18.0-193.28.1.el8_2.x86_64 cri-o://1.19.0-22.rhaos4.6.gitc0306f1.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=argo003.ceph.redhat.com,kubernetes.io/os=linux,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos argo004.ceph.redhat.com Ready master 9h v1.19.0+d59ce34 10.8.128.204 <none> Red Hat Enterprise Linux CoreOS 46.82.202010201440-0 (Ootpa) 4.18.0-193.28.1.el8_2.x86_64 cri-o://1.19.0-22.rhaos4.6.gitc0306f1.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=argo004.ceph.redhat.com,kubernetes.io/os=linux,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos argo005.ceph.redhat.com Ready worker 9h v1.19.0+d59ce34 10.8.128.205 <none> Red Hat Enterprise Linux CoreOS 46.82.202010201440-0 (Ootpa) 4.18.0-193.28.1.el8_2.x86_64 cri-o://1.19.0-22.rhaos4.6.gitc0306f1.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cluster.ocs.openshift.io/openshift-storage=,kubernetes.io/arch=amd64,kubernetes.io/hostname=argo005.ceph.redhat.com,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,topology.rook.io/rack=rack2 argo006.ceph.redhat.com Ready worker 9h v1.19.0+d59ce34 10.8.128.206 <none> Red Hat Enterprise Linux CoreOS 46.82.202010201440-0 (Ootpa) 4.18.0-193.28.1.el8_2.x86_64 cri-o://1.19.0-22.rhaos4.6.gitc0306f1.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cluster.ocs.openshift.io/openshift-storage=,kubernetes.io/arch=amd64,kubernetes.io/hostname=argo006.ceph.redhat.com,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,topology.rook.io/rack=rack0 argo007.ceph.redhat.com Ready worker 9h v1.19.0+d59ce34 10.8.128.207 <none> Red Hat Enterprise Linux CoreOS 46.82.202010201440-0 (Ootpa) 4.18.0-193.28.1.el8_2.x86_64 cri-o://1.19.0-22.rhaos4.6.gitc0306f1.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cluster.ocs.openshift.io/openshift-storage=,kubernetes.io/arch=amd64,kubernetes.io/hostname=argo007.ceph.redhat.com,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,topology.rook.io/rack=rack1 ________________________________________________________________ BTW, Verified the other node related files are now moved under "./cluster-scoped-resources" directory and are present as expected. a) Nodes in yaml - [1] b) Describe of nodes -[2] [1] - http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/1890183/must-gather.local.7263335647167393572/quay-io-rhceph-dev-ocs-must-gather-sha256-3255bdcfc54ce04e8b0b948cc2d6e4ba5e7fbd2ca14dc8512d5c845e4a9ae157/cluster-scoped-resources/core/nodes/ [2] - http://rhsqe-repo.lab.eng.blr.redhat.com/OCS/ocs-qe-bugs/1890183/must-gather.local.7263335647167393572/quay-io-rhceph-dev-ocs-must-gather-sha256-3255bdcfc54ce04e8b0b948cc2d6e4ba5e7fbd2ca14dc8512d5c845e4a9ae157/cluster-scoped-resources/oc_output/desc_nodes Thanks Pulkit Verified the fix on OCS = ocs-operator.v4.6.0-147.ci and OCP = 4.6.0-0.nightly-2020-10-22-034051. Observation: The `oc get nodes -o wide --show-labels` is working now and "the server doesn't have a resource type "nodes -o wide --show-labels"" is no longer seen in terminal >>$ oc adm must-gather --image=quay.io/rhceph-dev/ocs-must-gather:latest-4.6 |tee terminal-must-gather Starting pod/control-plane-2-debug ... To use host binaries, run `chroot /host` quay.io/rhceph-dev/ocs-must-gather latest-4.6 a0c951853a5f 17 hours ago 402 MB ... ... [must-gather-qwghc] POD collecting oc command sc [must-gather-qwghc] POD collecting oc command nodes -o wide --show-labels [must-gather-qwghc] POD collecting oc command clusterversion [must-gather-qwghc] POD collecting oc command infrastructures.config ------------------------------------------------------- Logs ========= >> Describe and oc get nodes $ ls -ltrh must-gather.local.6594167730655910032/quay-io-rhceph-dev-ocs-must-gather-sha256-3ce7cfc0a70f533270e9918895844cda82bdfc7e0e1850f34daa6cd58d008083/cluster-scoped-resources/oc_output |grep nodes -rw-r--r--. 1 nberry nberry 3.0K Oct 28 23:21 get_nodes_-o_wide_--show-labels -rw-r--r--. 1 nberry nberry 57K Oct 28 23:21 desc_nodes >> nodes in yaml $ ls -ltrh must-gather.local.6594167730655910032/quay-io-rhceph-dev-ocs-must-gather-sha256-3ce7cfc0a70f533270e9918895844cda82bdfc7e0e1850f34daa6cd58d008083/cluster-scoped-resources/core/nodes/ total 120K -rwxr-xr-x. 1 nberry nberry 19K Oct 28 23:21 compute-0.yaml -rwxr-xr-x. 1 nberry nberry 17K Oct 28 23:21 compute-1.yaml -rwxr-xr-x. 1 nberry nberry 19K Oct 28 23:21 compute-2.yaml -rwxr-xr-x. 1 nberry nberry 18K Oct 28 23:21 control-plane-0.yaml -rwxr-xr-x. 1 nberry nberry 18K Oct 28 23:21 control-plane-1.yaml -rwxr-xr-x. 1 nberry nberry 18K Oct 28 23:21 control-plane-2.yaml $ cat must-gather.local.6594167730655910032/quay-io-rhceph-dev-ocs-must-gather-sha256-3ce7cfc0a70f533270e9918895844cda82bdfc7e0e1850f34daa6cd58d008083/cluster-scoped-resources/oc_output/get_nodes_-o_wide_--show-labels NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME LABELS compute-0 Ready worker 13d v1.19.0+d59ce34 10.1.160.165 10.1.160.165 Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) 4.18.0-193.24.1.el8_2.dt1.x86_64 cri-o://1.19.0-20.rhaos4.6.git97d715e.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cluster.ocs.openshift.io/openshift-storage=,kubernetes.io/arch=amd64,kubernetes.io/hostname=compute-0,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,topology.rook.io/rack=rack0 compute-1 Ready worker 13d v1.19.0+d59ce34 10.1.160.161 10.1.160.161 Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) 4.18.0-193.24.1.el8_2.dt1.x86_64 cri-o://1.19.0-20.rhaos4.6.git97d715e.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cluster.ocs.openshift.io/openshift-storage=,kubernetes.io/arch=amd64,kubernetes.io/hostname=compute-1,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,topology.rook.io/rack=rack1 compute-2 Ready worker 13d v1.19.0+d59ce34 10.1.160.180 10.1.160.180 Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) 4.18.0-193.24.1.el8_2.dt1.x86_64 cri-o://1.19.0-20.rhaos4.6.git97d715e.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,cluster.ocs.openshift.io/openshift-storage=,kubernetes.io/arch=amd64,kubernetes.io/hostname=compute-2,kubernetes.io/os=linux,node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhcos,topology.rook.io/rack=rack2 control-plane-0 Ready master 13d v1.19.0+d59ce34 10.1.160.163 10.1.160.163 Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) 4.18.0-193.24.1.el8_2.dt1.x86_64 cri-o://1.19.0-20.rhaos4.6.git97d715e.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=control-plane-0,kubernetes.io/os=linux,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos control-plane-1 Ready master 13d v1.19.0+d59ce34 10.1.160.166 10.1.160.166 Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) 4.18.0-193.24.1.el8_2.dt1.x86_64 cri-o://1.19.0-20.rhaos4.6.git97d715e.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=control-plane-1,kubernetes.io/os=linux,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos control-plane-2 Ready master 13d v1.19.0+d59ce34 10.1.160.162 10.1.160.162 Red Hat Enterprise Linux CoreOS 46.82.202010091720-0 (Ootpa) 4.18.0-193.24.1.el8_2.dt1.x86_64 cri-o://1.19.0-20.rhaos4.6.git97d715e.el8 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=control-plane-2,kubernetes.io/os=linux,node-role.kubernetes.io/master=,node.openshift.io/os_id=rhcos Moving the BZ to verified state as all node related information are now collected under cluster_scoped directory. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: Red Hat OpenShift Container Storage 4.6.0 security, bug fix, enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:5605 |