Description of problem:
=======================

ls complained cannot open directory on Fuse client.

[f]# ls
ls: cannot open directory .: Transport endpoint is not connected

Version-Release number of selected component (if applicable):
=============================================================

# rpm -qa | grep gluster 
glusterfs-fuse-3.4.0.9rhs-1.el6.x86_64
glusterfs-3.4.0.9rhs-1.el6.x86_64
glusterfs-debuginfo-3.4.0.9rhs-1.el6.x86_64
glusterfs-devel-3.4.0.9rhs-1.el6.x86_64
glusterfs-rdma-3.4.0.9rhs-1.el6.x86_64
# 


Steps Carried:
==============
1. Created 6*2 volume from 4 servers
2. Mounted on Client (NFS and FUSE)
3. Created directory names f and n from fuse mount 
4. From fuse mount cd to f
5. From NFS mount cd to n
6. Metadata,data and entry self heal were set to off
"for i in {'metadata','data','entry'} ; do gluster volume set <volume_name> $i-self-heal off ; done"
7. From both the mounted directories executed the following command to create directories, files within directories and files:

for i in `seq 1 10` ; do mkdir dir.$i ; for j in `seq 1 5` ; do dd if=/dev/urandom of=dir.$i/file.$j bs=1K count=1 ; done ; dd if=/dev/urandom of=file.$i bs=1k count=1 ; done

8. Set the self-heal-daemon to off

gluster volume set <vol_name> self-heal-daemon off

9. Brought down server 2 and server 4 (poweroff)
10. Modified the content of files from both the mounted directories (f and n)

for i in `seq 1 10` ; do for j in `seq 1 5` ; do dd if=/dev/urandom of=dir.$i/file.$j bs=1M count=1 ; done ; dd if=/dev/urandom of=file.$i bs=1k count=1 ; done

10. Brought back the server 2 and server 4

11. Once the servers are brought up, killed the brick processes of server 1 and server 3 which were earlier UP.

12. Tried to modify the content of files from fuse and nfs mounted directories (f and n)

for i in `seq 1 10` ; do for j in `seq 1 5` ; do dd if=/dev/urandom of=dir.$i/file.$j bs=1M count=1 ; done ; dd if=/dev/urandom of=file.$i bs=1k count=1 ; done


Actual results:
===============

Successful from NFS mounted directory (n) but failed from fuse mount it complained as 

[f]#  for i in `seq 1 10` ; do for j in `seq 1 5` ; do dd if=/dev/urandom of=dir.$i/file.$j bs=1M count=1 ; done ; dd if=/dev/urandom of=file.$i bs=1k count=1 ; done
dd: opening `dir.1/file.1': Transport endpoint is not connected
dd: opening `dir.1/file.2': Transport endpoint is not connected
dd: opening `dir.1/file.3': Transport endpoint is not connected
dd: opening `dir.1/file.4': Transport endpoint is not connected
dd: opening `dir.1/file.5': Transport endpoint is not connected
dd: opening `file.1': Transport endpoint is not connected
dd: opening `dir.2/file.1': Transport endpoint is not connected
dd: opening `dir.2/file.2': Transport endpoint is not connected
dd: opening `dir.2/file.3': Transport endpoint is not connected
dd: opening `dir.2/file.4': Transport endpoint is not connected
dd: opening `dir.2/file.5': Transport endpoint is not connected
dd: opening `file.2': Transport endpoint is not connected
dd: opening `dir.3/file.1': Transport endpoint is not connected
dd: opening `dir.3/file.2': Transport endpoint is not connected
dd: opening `dir.3/file.3': Transport endpoint is not connected
dd: opening `dir.3/file.4': Transport endpoint is not connected
dd: opening `dir.3/file.5': Transport endpoint is not connected
dd: opening `file.3': Transport endpoint is not connected
dd: opening `dir.4/file.1': Transport endpoint is not connected
dd: opening `dir.4/file.2': Transport endpoint is not connected
dd: opening `dir.4/file.3': Transport endpoint is not connected
dd: opening `dir.4/file.4': Transport endpoint is not connected
dd: opening `dir.4/file.5': Transport endpoint is not connected
dd: opening `file.4': Transport endpoint is not connected
dd: opening `dir.5/file.1': Transport endpoint is not connected
dd: opening `dir.5/file.2': Transport endpoint is not connected
dd: opening `dir.5/file.3': Transport endpoint is not connected
dd: opening `dir.5/file.4': Transport endpoint is not connected
dd: opening `dir.5/file.5': Transport endpoint is not connected
dd: opening `file.5': Transport endpoint is not connected
dd: opening `dir.6/file.1': Transport endpoint is not connected
dd: opening `dir.6/file.2': Transport endpoint is not connected
dd: opening `dir.6/file.3': Transport endpoint is not connected
dd: opening `dir.6/file.4': Transport endpoint is not connected
dd: opening `dir.6/file.5': Transport endpoint is not connected
dd: opening `file.6': Transport endpoint is not connected
dd: opening `dir.7/file.1': Transport endpoint is not connected
dd: opening `dir.7/file.2': Transport endpoint is not connected
dd: opening `dir.7/file.3': Transport endpoint is not connected
dd: opening `dir.7/file.4': Transport endpoint is not connected
dd: opening `dir.7/file.5': Transport endpoint is not connected
dd: opening `file.7': Transport endpoint is not connected
dd: opening `dir.8/file.1': Transport endpoint is not connected
dd: opening `dir.8/file.2': Transport endpoint is not connected
dd: opening `dir.8/file.3': Transport endpoint is not connected
dd: opening `dir.8/file.4': Transport endpoint is not connected
dd: opening `dir.8/file.5': Transport endpoint is not connected
dd: opening `file.8': Transport endpoint is not connected
dd: opening `dir.9/file.1': Transport endpoint is not connected
dd: opening `dir.9/file.2': Transport endpoint is not connected
dd: opening `dir.9/file.3': Transport endpoint is not connected
dd: opening `dir.9/file.4': Transport endpoint is not connected
dd: opening `dir.9/file.5': Transport endpoint is not connected
dd: opening `file.9': Transport endpoint is not connected
dd: opening `dir.10/file.1': Transport endpoint is not connected
dd: opening `dir.10/file.2': Transport endpoint is not connected
dd: opening `dir.10/file.3': Transport endpoint is not connected
dd: opening `dir.10/file.4': Transport endpoint is not connected
dd: opening `dir.10/file.5': Transport endpoint is not connected
dd: opening `file.10': Transport endpoint is not connected
[f]#

ls from the mount point also failed.

[f]# ls
ls: cannot open directory .: Transport endpoint is not connected
[f]# 


Expected results:
=================

ls and modification of files from fuse mount should also successful.


Additional info:
================

1. All the above operation from NFS mounts are successful.
2. Once came out of the mounted fuse directory (f) and cd again to it worked successfully and than the files were listed successfully.


[f]# ls
ls: cannot open directory .: Transport endpoint is not connected
[f]# 
[f]# 
[f]# ls /
[f]# 
[f]# 
[f]# cd
[~]# cd -
/mnt/vol-dis-rep/f
[f]# ls
dir.1  dir.10  dir.2  dir.3  dir.4  dir.5  dir.6  dir.7  dir.8  dir.9  file.1  file.10  file.2  file.3  file.4  file.5  file.6  file.7  file.8  file.9
[f]# 
[f]# ls dir.1/
file.1  file.2  file.3  file.4  file.5
[f]#