Bug 1953292

Summary: A lot of error msg "Unable to fetch pod etc hosts stats" found in kubelet log
Product: OpenShift Container Platform Reporter: Qin Ping <piqin>
Component: NetworkingAssignee: Miciah Dashiel Butler Masters <mmasters>
Networking sub component: DNS QA Contact: Hongan Li <hongli>
Status: CLOSED DUPLICATE Docs Contact:
Severity: medium    
Priority: high CC: akrzos, amcdermo, aos-bugs, sgrunert
Version: 4.8   
Target Milestone: ---   
Target Release: 4.8.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-05-04 21:09:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Qin Ping 2021-04-25 08:51:45 UTC
Description of problem:
Detailed error msg:
Apr 25 05:41:20 ip-10-0-214-218 hyperkube[1391]: E0425 05:41:20.937906    1391 cadvisor_stats_provider.go:151] "Unable to fetch pod etc hosts stats" err="failed to get stats failed command 'du' ($ nice -n 19 du -x -s -B 1) on path /var/lib/kubelet/pods/c90680d5-ce3f-4659-a6dc-039397952fb9/etc-hosts with error exit status 1" pod="openshift-dns/node-resolver-zm4g8"
Apr 25 05:41:20 ip-10-0-214-218 hyperkube[1391]: E0425 05:41:20.980666    1391 cadvisor_stats_provider.go:151] "Unable to fetch pod etc hosts stats" err="failed to get stats failed command 'du' ($ nice -n 19 du -x -s -B 1) on path /var/lib/kubelet/pods/c90680d5-ce3f-4659-a6dc-039397952fb9/etc-hosts with error exit status 1" pod="openshift-dns/node-resolver-zm4g8"

Found thousands of this kind of msg in one kubelet log.

Version-Release number of selected component (if applicable):
4.8.0-0.nightly-2021-04-24-234710

How reproducible:
Found it in OCP on AWS and OCP on Azure clusters, didn't check other cloud platforms.

For a cluster running about 2 hours, the error msg account:
sh-4.4# journalctl -u kubelet|grep "Unable to fetch pod etc hosts stats"|wc
   1593   65313  623218

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Sascha Grunert 2021-05-03 10:41:48 UTC
I think this issue is a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1949612

Comment 3 Miciah Dashiel Butler Masters 2021-05-04 21:09:55 UTC

*** This bug has been marked as a duplicate of bug 1949612 ***