Bug 1575948 - [3.6] Pod stuck in terminating state [NEEDINFO]
Summary: [3.6] Pod stuck in terminating state
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.6.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: 3.6.z
Assignee: Joel Smith
QA Contact: Xiaoli Tian
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-08 11:27 UTC by Neeraj
Modified: 2019-06-18 20:36 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-06-18 20:36:53 UTC
Target Upstream Version:
joelsmith: needinfo? (agawand)


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 1449097 0 medium CLOSED Pod is terminated, but some volumes have not been cleaned up 2021-02-22 00:41:40 UTC

Internal Links: 1449097

Description Neeraj 2018-05-08 11:27:18 UTC
Created attachment 1433157 [details]
pod termination doc

Description of problem:
Fluentd pods stuck in terminating state. 

Version-Release number of selected component (if applicable):

3.6

Comment 10 Joel Smith 2018-06-15 15:41:48 UTC
No, sorry, I mean, please ask the customer to watch for this to happen again, and when it does, have them view the list of mounted volumes on the node. They'll be looking to see if there is a tmpfs volume mounted in the "busy" directory that the node can't remove. In the above example, they'd do something like "mount | grep 4631-11e8-8ed1-005056ab75a6" and they'd expect to see a volume mounted at /var/lib/origin/openshift.local.volumes/pods/89e072ff-4631-11e8-8ed1-005056ab75a6/volumes/kubernetes.io~secret/aggregated-logging-fluentd-token-n5ppt"

Comment 11 Joel Smith 2018-06-15 15:43:04 UTC
One other thing I forgot to mention is that we have a temporary work-around they can use to get the pod to terminate.  If they manually umount the mounted path, the pod should terminate. We'll work on figuring out why it is failing to unmount on its own.


Note You need to log in before you can comment on or make changes to this bug.