Bug 1638119 - PV stuck in pending state due to readiness probe and liveliness probe failures.
Summary: PV stuck in pending state due to readiness probe and liveliness probe failures.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: kubernetes
Version: rhgs-3.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: ---
Assignee: Humble Chirammal
QA Contact: Prasanth
URL:
Whiteboard:
Depends On:
Blocks: 1633499
TreeView+ depends on / blocked
 
Reported: 2018-10-10 18:33 UTC by vinutha
Modified: 2019-01-23 14:47 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-01-23 14:47:25 UTC
Embargoed:

Attachments (Terms of Use)

Description vinutha 2018-10-10 18:33:44 UTC 
Description of problem:
pvc stuck in Pending state and not getting bound due to errors: 
1. 'Readiness probe failed: rpc error: code = 14 desc = grpc: the connection is 
    unavailable'
2. 'Liveness probe failed: rpc error: code = 14 desc = grpc: the connection is 
    unavailable'


Version-Release number of selected component (if applicable):
# rpm -qa| grep docker
docker-common-1.13.1-75.git8633870.el7_5.x86_64
python-docker-2.4.2-1.3.el7.noarch
docker-client-1.13.1-75.git8633870.el7_5.x86_64
docker-1.13.1-75.git8633870.el7_5.x86_64
cockpit-docker-176-2.el7.x86_64
docker-rhel-push-plugin-1.13.1-75.git8633870.el7_5.x86_64
python-docker-pycreds-1.10.6-4.el7.noarch
atomic-openshift-docker-excluder-3.11.21-1.git.0.3e1eda6.el7.noarch

# docker version
Client:
 Version:         1.13.1
 API version:     1.26
 Package version: docker-1.13.1-75.git8633870.el7_5.x86_64
 Go version:      go1.9.2
 Git commit:      8633870/1.13.1
 Built:           Wed Sep 12 10:56:54 2018
 OS/Arch:         linux/amd64

Server:
 Version:         1.13.1
 API version:     1.26 (minimum version 1.12)
 Package version: docker-1.13.1-75.git8633870.el7_5.x86_64
 Go version:      go1.9.2
 Git commit:      8633870/1.13.1
 Built:           Wed Sep 12 10:56:54 2018
 OS/Arch:         linux/amd64
 Experimental:    false


# rpm -qa| grep openshift
openshift-ansible-3.11.21-1.git.0.7dc17ca.el7.noarch
atomic-openshift-excluder-3.11.21-1.git.0.3e1eda6.el7.noarch
openshift-ansible-docs-3.11.21-1.git.0.7dc17ca.el7.noarch
openshift-ansible-playbooks-3.11.21-1.git.0.7dc17ca.el7.noarch
atomic-openshift-node-3.11.21-1.git.0.3e1eda6.el7.x86_64
atomic-openshift-clients-3.11.21-1.git.0.3e1eda6.el7.x86_64
atomic-openshift-3.11.21-1.git.0.3e1eda6.el7.x86_64
openshift-ansible-roles-3.11.21-1.git.0.7dc17ca.el7.noarch
atomic-openshift-docker-excluder-3.11.21-1.git.0.3e1eda6.el7.noarch
atomic-openshift-hyperkube-3.11.21-1.git.0.3e1eda6.el7.x86_64


How reproducible:
Always on this setup

Steps to Reproduce:
1. On a 4 node setup of OCP 3.11.21 + CNS 3.11 latest created 1GB pvc which is stuck in pending state for ~30min 


Actual results:
PVC not getting Bound 

Expected results:
Pvc should be successfully bound

Additional info:
will update heketi logs and some command outputs
Note You need to log in before you can comment on or make changes to this bug.