Bug 1789029

Summary: Gluster pod going into "0/1" state after some tests execution
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Vaibhav Mahajan <vamahaja>
Component: tcmu-runnerAssignee: Prasanna Kumar Kalever <prasanna.kalever>
Status: CLOSED ERRATA QA Contact: Vaibhav Mahajan <vamahaja>
Severity: high Docs Contact:
Priority: unspecified    
Version: ocs-3.11CC: gscrivan, jmulligan, madam, nchilaka, pkarampu, pprakash, prasanna.kalever, puebele, rhs-bugs, rtalur, sabose, susgupta, vbellur, xiubli
Target Milestone: ---Keywords: Regression, ZStream
Target Release: OCS 3.11.z Batch Update 5   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tcmu-runner-1.2.0-32.el7rhgs Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-02-26 20:05:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vaibhav Mahajan 2020-01-08 15:37:29 UTC 
Description of problem:
Gluster pods went into "0/1" state after executing some test cases related to gluster node reboot and after blocking network ports related to gluster nodes.

Version-Release number of selected component (if applicable):

# oc version
oc v3.11.161
kubernetes v1.11.0+d4cacc0
features: Basic-Auth GSSAPI Kerberos SPNEGO

Server https://ocs-z-stream-ocp-3-11-z-ocs-3-11-5-master-0:8443
openshift v3.11.161
kubernetes v1.11.0+d4cacc0


sh-4.2# rpm -qa | grep glusterfs
glusterfs-api-6.0-25.el7rhgs.x86_64
glusterfs-server-6.0-25.el7rhgs.x86_64
glusterfs-libs-6.0-25.el7rhgs.x86_64
glusterfs-6.0-25.el7rhgs.x86_64
glusterfs-client-xlators-6.0-25.el7rhgs.x86_64
glusterfs-cli-6.0-25.el7rhgs.x86_64
glusterfs-fuse-6.0-25.el7rhgs.x86_64
glusterfs-geo-replication-6.0-25.el7rhgs.x86_64


How reproducible:
Always

Steps to Reproduce:
1. Deploy gluster server with latest openshift ansible & OCS 3.11.5 builds.
2. Execute test cases related to gluster node reboot and blocking gluster ports on gluster node.
3. After around 10 mins, check gluster pod status

Actual results:
Gluster pods are going into "0/1" state

# oc get pods
NAME                                          READY     STATUS              RESTARTS   AGE
glusterblock-storage-provisioner-dc-1-zkswm   1/1       Running             0          1d
glusterfs-storage-8p6mn                       1/1       Running             1          1d
glusterfs-storage-9zfbx                       0/1       Running             0          1d
glusterfs-storage-cp5sb                       0/1       Running             0          1d
glusterfs-storage-gm4jd                       1/1       Running             0          1d
glusterfs-storage-vh8k9                       0/1       Running             0          1d
heketi-storage-1-mmhtg                        0/1       ContainerCreating   0          1d


Expected results:
Gluster pod status should be "1/1"
Comment 34 errata-xmlrpc 2020-02-26 20:05:00 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0621