Bug 1546156 - [3.7]Fix bug on kubelet failure to umount mount points.
Summary: [3.7]Fix bug on kubelet failure to umount mount points.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.7.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.7.z
Assignee: Seth Jennings
QA Contact: Wenqi He
URL:
Whiteboard:
Depends On:
Blocks: 1546159
TreeView+ depends on / blocked
 
Reported: 2018-02-16 13:04 UTC by Jaspreet Kaur
Modified: 2018-04-05 09:39 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Fixes an issue where a "transport endpoint is not connected" error is encountered when deleted projects with pods using CNS backed PVs.
Clone Of:
: 1546159 (view as bug list)
Environment:
Last Closed: 2018-04-05 09:38:31 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:0636 None None None 2018-04-05 09:39:21 UTC

Description Jaspreet Kaur 2018-02-16 13:04:41 UTC
Description of problem: UnmountVolume.TearDown tries to stat the gluster mount point and errors with "transport endpoint is not connected" due to the endpoint being deleted.

https://github.com/openshift/origin/blob/master/vendor/k8s.io/kubernetes/pkg/volume/util/util.go#L84

This blocks the umount later in that function causing the overall volume detach to fail and hang the pod termination.


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results: Fails every time.


Expected results: should not have an issue. 


Additional info: Required fix in 3.7 for below bugzilla to be backported 

https://bugzilla.redhat.com/show_bug.cgi?id=1489082

Comment 5 Humble Chirammal 2018-02-21 11:25:09 UTC
I had filed a 3.7 PR and its merged https://github.com/openshift/ose/pull/1068

Comment 6 Seth Jennings 2018-02-21 15:15:15 UTC
Ah ok, I searched for a PR with the upstream description but looks like there were two upstream commits in this one PR and the PR description was that of the other commit.  Hence why I didn't find it.  And it looks like it merged around the same time I was opening my PR :-/

I'll close mine and move this to modified.

Comment 8 Wenqi He 2018-03-01 14:23:45 UTC
Tested on below version:
openshift v3.7.35
kubernetes v1.7.6+a08f5eeb62

[root@host-172-16-120-121 ~]# oc get pvc
NAME      STATUS    VOLUME                                     CAPACITY   ACCESSMODES   STORAGECLASS        AGE
mypv      Bound     pvc-44ace2ba-1d5b-11e8-8a10-fa163e91297f   1Gi        RWO           glusterfs-storage   22s
[root@host-172-16-120-121 ~]# oc create -f pod.yaml
pod "busybox-pod" created
[root@host-172-16-120-121 ~]# oc get pods
NAME          READY     STATUS              RESTARTS   AGE
busybox-pod   0/1       ContainerCreating   0          3s
[root@host-172-16-120-121 ~]# oc get pods
NAME          READY     STATUS    RESTARTS   AGE
busybox-pod   1/1       Running   0          1m
[root@host-172-16-120-121 ~]# oc delete project test
project "test" deleted
[root@host-172-16-120-121 ~]# oc get pods
NAME          READY     STATUS    RESTARTS   AGE
busybox-pod   1/1       Running   0          1m

[root@host-172-16-120-121 ~]# oc get project
test                                               Terminating

[root@host-172-16-120-121 ~]# oc get project

Projects has been deleted

Comment 12 errata-xmlrpc 2018-04-05 09:38:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0636


Note You need to log in before you can comment on or make changes to this bug.