Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1949671

Summary:	System services such as openvswitch are stopped before pod containers on system shutdown or reboot
Product:	OpenShift Container Platform	Reporter:	Mrunal Patel <mpatel>
Component:	Node	Assignee:	Kir Kolyshkin <kir>
Node sub component:	CRI-O	QA Contact:	Weinan Liu <weinliu>
Status:	CLOSED ERRATA	Docs Contact:
Severity:	high
Priority:	high	CC:	aos-bugs, lmohanty, nagrawal, weinliu, wking
Version:	4.7	Keywords:	Upgrades
Target Milestone:	---
Target Release:	4.8.0
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2021-07-27 23:00:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Mrunal Patel 2021-04-14 18:44:33 UTC

Description of problem:
The containers started by CRI-O do not have dependencies that ensure correct ordering at shutdown as a result of which system services such openvswitch may shutdown before something like kube-apiserver which then can't reach openshift-apiserver over pod network preventing a clean shutdown.

Here are some example logs:
Apr 14 03:37:39.816073 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: openvswitch.service: Succeeded.
Apr 14 03:37:39.816804 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: Stopped Open vSwitch
...
Apr 14 03:39:49.289223 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: Stopped libcontainer container 6d13682416c7a6f9acbddeb1cac5d23042f1b1700b17cd2e2d7659c947e2979c.
Apr 14 03:39:49.300516 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: crio-6d13682416c7a6f9acbddeb1cac5d23042f1b1700b17cd2e2d7659c947e2979c.scope: Consumed 16min 32.974s CPU time
Apr 14 03:39:49.300976 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: Reached target Shutdown.
Apr 14 03:39:49.307468 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: Reached target Final Step.
Apr 14 03:39:49.316853 ci-op-81677qy8-db044-bb8sk-master-2 systemd[1]: Starting Reboot...




Version-Release number of selected component (if applicable):


How reproducible:
Seen in CI





Expected results:
The system services such as openvswitch should only be shut down after all the container scopes.


Additional info:

Comment 1 Mrunal Patel 2021-04-19 21:19:42 UTC

https://github.com/cri-o/cri-o/pull/4766

Comment 8 errata-xmlrpc 2021-07-27 23:00:48 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:2438