Bug 1983127

Summary:	Exec probes fail clusterwide after upgrade to cri-o-1.19.2-4.rhaos4.6.git4f7cb5e.el7.x86_64
Product:	OpenShift Container Platform	Reporter:	OpenShift BugZilla Robot <openshift-bugzilla-robot>
Component:	Node	Assignee:	Peter Hunt <pehunt>
Node sub component:	CRI-O	QA Contact:	MinLi <minmli>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	high	CC:	aos-bugs, ddelcian, eminguez, jchaloup, ltitov, obockows, pehunt, wking
Version:	4.6
Target Milestone:	---
Target Release:	4.8.z
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-08-03 06:09:58 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1978268
Bug Blocks:	1983128, 2026214

Comment 2 MinLi 2021-07-22 08:53:57 UTC

I find error "probe failed: command timed out" happen frequently in a 4.8 job : https://prow.ci.openshift.org/job-history/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.8-e2e-metal-ipi-serial-ipv4, also can be searched by: https://search.ci.openshift.org/?search=probe+failed%3A+command+timed+out&maxAge=48h&context=1&type=junit&name=&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job

"Jul 21 03:52:28.000 W ns/openshift-kni-infra pod/keepalived-worker-1 node/worker-1 reason/Unhealthy Liveness probe failed: command timed out (3 times)"

@Peter Hunt, 
Can you help to check if the issue haven't been fixed or it just happen by accident.

Comment 3 Peter Hunt 2021-07-23 19:36:18 UTC

I can only find one match https://search.ci.openshift.org/?search=probe+failed%3A+command+timed+out&maxAge=336h&context=1&type=junit&name=%5Eperiodic-ci-openshift-release-master-nightly-4%5C.7-e2e-vsphere%24&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job and I don't think that's indicative of the issue we were seeing

Comment 4 MinLi 2021-07-26 07:09:40 UTC

run a 4.8 cluster for a long time, don't find "probe failed: command timed out" events. 
set it verified.

Comment 7 errata-xmlrpc 2021-08-03 06:09:58 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.8.3 bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2896