Bug 1829113 - Baremetal infra container logs missing from must-gather
Summary: Baremetal infra container logs missing from must-gather
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Machine Config Operator
Version: 4.5
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 4.5.0
Assignee: Ben Nemec
QA Contact: Eldar Weiss
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-28 22:13 UTC by Ben Nemec
Modified: 2020-07-13 17:32 UTC (History)
3 users (show)

Fixed In Version: 4.5.0-0.nightly-2020-05-03-051551
Doc Type: Bug Fix
Doc Text:
Cause: must-gather was not configured to collect details of some baremetal services Consequence: It was difficult to debug problems in baremetal deployments. Fix: Added the openshift-kni-infra namespace to the list of namespaces to be collected. Result: Baremetal service logs are now included in must-gather output.
Clone Of:
Environment:
titan37
Last Closed: 2020-07-13 17:32:29 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift must-gather pull 150 0 None closed Bug 1829113: Add openshift-kni-infra namespace to gather 2021-02-08 05:16:17 UTC
Red Hat Product Errata RHBA-2020:2409 0 None None None 2020-07-13 17:32:47 UTC

Description Ben Nemec 2020-04-28 22:13:14 UTC
Description of problem: Running must-gather against a baremetal deployment does not collect the logs from the KNI infra static pods. This makes debugging problematic.


Version-Release number of selected component (if applicable): 4.5


How reproducible: Always


Steps to Reproduce:
1. Run oc adm must-gather against baremetal cluster

Actual results: No logs for the containers in openshift-kni-infra


Expected results: Logs for containers in openshift-kni-infra (coredns, mdns-publisher, keepalived, haproxy)

Comment 1 Eldar Weiss 2020-04-30 07:16:43 UTC
]# oc adm must-gather
[must-gather      ] OUT unable to resolve the imagestream tag openshift/must-gather:latest
[must-gather      ] OUT 
[must-gather      ] OUT Using must-gather plugin-in image: quay.io/openshift/origin-must-gather:latest
[must-gather      ] OUT namespace/openshift-must-gather-f8tdw created
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-nzrfb created
[must-gather      ] OUT pod for plug-in image quay.io/openshift/origin-must-gather:latest created
[must-gather-wxqwm] OUT gather did not start: unable to pull image: ErrImagePull: rpc error: code = Unknown desc = error pinging docker registry quay.io: Get https://quay.io/v2/: dial tcp 52.1.239.254:443: connect: network is unreachable
[must-gather      ] OUT clusterrolebinding.rbac.authorization.k8s.io/must-gather-nzrfb deleted
[must-gather      ] OUT namespace/openshift-must-gather-f8tdw deleted
error: gather did not start for pod must-gather-wxqwm: unable to pull image: ErrImagePull: rpc error: code = Unknown desc = error pinging docker registry quay.io: Get https://quay.io/v2/: dial tcp 52.1.239.254:443: connect: network is unreachable

Comment 2 Ben Nemec 2020-04-30 14:48:37 UTC
I'm guessing you ran that on an ipv6 deployment? By default, must-gather pulls an image from quay.io, which doesn't work with ipv6. I believe this is a known issue. To make it work, you either need to deploy with ipv4 or pass a must-gather image to the command that exists in an ipv6-accessible registry.

Comment 6 Eldar Weiss 2020-05-03 17:57:54 UTC
Hi,

I've ran it on titan37
Ran this version:
version   4.5.0-0.nightly-2020-05-03-051551

all logs created as expected.

Comment 7 Eldar Weiss 2020-05-04 06:50:59 UTC
Expected results: Logs for containers in openshift-kni-infra (coredns, mdns-publisher, keepalived, haproxy)
actual results : all logs created, including the above, etc, too many to list here.

OUT namespaces/openshift-kni-infra/pods/coredns-worker-0-0/coredns-monitor/coredns-monitor/logs/current.log

OUT namespaces/openshift-kni-infra/pods/coredns-worker-0-0/coredns-monitor/coredns-monitor/logs/current.log
OUT namespaces/openshift-kni-infra/pods/keepalived-worker-0-0/keepalived/keepalived/logs/current.log
OUT namespaces/openshift-kni-infra/pods/haproxy-master-0-0/haproxy/haproxy/logs/current.log

Comment 8 errata-xmlrpc 2020-07-13 17:32:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409


Note You need to log in before you can comment on or make changes to this bug.