Bug 1266863

Description of problem:
========================
On a tiered volume, After setting quotas I did some IOs . After quotas exceeded, I killed the quota deamon on local node. Now when I tried do new file creates, I am getting transport end point error on fuse mount.
I tried and  deleted some files and retried , new files got created till probably till disk quota got reached and again started to throw transport end point error


NOTE: This may not be a tiering problem alone. It has to be more to do with quotas.


[root@zod ~]# rpm -qa|grep gluster
glusterfs-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-fuse-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-debuginfo-3.7.4-0.33.git1d02d4b.el7.centos.x86_64
glusterfs-api-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-client-xlators-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-server-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-cli-3.7.4-0.43.gitf139283.el7.centos.x86_64
glusterfs-libs-3.7.4-0.43.gitf139283.el7.centos.x86_64
[root@zod ~]# gluster --version
glusterfs 3.7.4 built on Sep 19 2015 01:30:43
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.
[root@zod ~]# 


Steps to Reproduce:
==================
1.create a tier volume 
2.enable quota and set hardlimit of root volume / to say 10gb
3. Now enable ctr and set the demote freq to say 1500sec
3.Now mount from glustefs-fuse and now fill the hardlimit to full
4. Now kill the quota deamon on local node
5. Now do new file creates and you can see transport end point error.





Logs:
====
fuse mount log sample:
[root@localhost togo]# tail /var/log/glusterfs/mnt-togo.log 
[2015-09-28 09:10:12.329534] W [fuse-bridge.c:1978:fuse_create_cbk] 0-glusterfs-fuse: 168844: /file.205 => -1 (Transport endpoint is not connected)
[2015-09-28 09:10:12.371421] W [MSGID: 114031] [client-rpc-fops.c:2399:client3_3_create_cbk] 0-togo-client-7: remote operation failed. Path: /file.206 [Disk quota exceeded]




deleted some files and retried(file 131-151 got created and then started to error out)
[root@localhost togo]# for i in {131..230};do cp file.1 file.$i;done
	cp: error writing ‘file.152’: Disk quota exceeded
cp: failed to extend ‘file.152’: Disk quota exceeded
cp: failed to close ‘file.152’: Disk quota exceeded
cp: cannot create regular file ‘file.153’: Transport endpoint is not connected
cp: cannot create regular file ‘file.154’: Transport endpoint is not connected
cp: cannot create regular file ‘file.155’: Transport endpoint is not connected
cp: cannot create regular file ‘file.156’: Transport endpoint is not connected