Description of problem: ======================= rm on symbolic link errors with "Bad file descriptor" from client only on tiered volume. [root@dj r1]# touch a [root@dj r1]# ln -s a b [root@dj r1]# ls a b [root@dj r1]# rm -rf b rm: cannot remove ‘b’: Bad file descriptor [root@dj r1]# ls a [root@dj r1]# Brick Log Snippet: ================== [2015-12-20 19:36:21.565019] E [MSGID: 113005] [posix.c:1700:posix_unlink] 0-vol0-posix: post operation fstat failed on fd=-1 [Bad file descriptor] [2015-12-20 19:36:21.565125] E [MSGID: 115062] [server-rpc-fops.c:1097:server_unlink_cbk] 0-vol0-server: 229668: UNLINK /r1/b (1ad39c8b-623b-455a-9914-6139453f4b31/b) ==> (Bad file descriptor) [Bad file descriptor] Version-Release number of selected component (if applicable): ============================================================= glusterfs-3.7.5-12.el7rhgs.x86_64 How reproducible: ================= Always Steps to Reproduce: =================== 1. Create Tier volume 2. Mount the volume on client 3. Create a file 4. Create symlink to the file 5. Remove symlink from client Actual results: =============== rm: cannot remove ‘etc.3/xdg/systemd/user’: Bad file descriptor rm: cannot remove ‘etc.3/rc.d/rc0.d/K50netconsole’: Bad file descriptor rm: cannot remove ‘etc.3/rc.d/rc0.d/K03rhnsd’: Bad file descriptor rm: cannot remove ‘etc.3/rc.d/rc0.d/K90network’: Bad file descriptor rm: cannot remove ‘etc.3/rc.d/rc5.d/S10network’: Bad file descriptor rm: cannot remove ‘etc.3/rc.d/rc5.d/K50netconsole’: Bad file descriptor [root@mia ~]# file /etc/xdg/systemd/user /etc/xdg/systemd/user: symbolic link to `/etc/systemd/user' [root@mia ~]# file /etc/rc.d/rc0.d/K50netconsole /etc/rc.d/rc0.d/K50netconsole: symbolic link to `../init.d/netconsole' [root@mia ~]# file /etc/rc.d/rc0.d/K03rhnsd /etc/rc.d/rc0.d/K03rhnsd: symbolic link to `../init.d/rhnsd' [root@mia ~]# ls -l /etc/rc.d/rc0.d/K03rhnsd lrwxrwxrwx. 1 root root 15 Oct 14 07:17 /etc/rc.d/rc0.d/K03rhnsd -> ../init.d/rhnsd [root@mia ~]# file /etc/rc.d/rc0.d/K90network /etc/rc.d/rc0.d/K90network: symbolic link to `../init.d/network' [root@mia ~]# file /etc/rc.d/rc5.d/S10network /etc/rc.d/rc5.d/S10network: symbolic link to `../init.d/network' [root@mia ~]# file /etc/rc.d/rc5.d/K50netconsole /etc/rc.d/rc5.d/K50netconsole: symbolic link to `../init.d/netconsole' [root@mia ~]# Expected results: ================= Symbolic link should also be removed without errors Additional info: ================ Removal of symlink file on non-tiered volume is successful
upstream pach : http://review.gluster.org/13034
Note: the directory fails to get deleted due to this issue, if there is symlinks in the directory Workaround: retry delete of the directory once all files are deleted.
Verified with build: glusterfs-3.7.5-14.el7rhgs.x86_64 Removal of symbolic and hardlink from tiervolume is successful without any errors. Moving this bug to verified state [root@dj ~]# mkdir /mnt/fuse [root@dj ~]# mount -t glusterfs 10.70.37.165:/tiervolume /mnt/fuse/ [root@dj ~]# cd /mnt/fuse/ [root@dj fuse]# [root@dj fuse]# touch a [root@dj fuse]# ls a [root@dj fuse]# ll total 0 -rw-r--r--. 1 root root 0 Jan 6 04:36 a [root@dj fuse]# ln -s a b [root@dj fuse]# ll total 0 -rw-r--r--. 1 root root 0 Jan 6 04:36 a lrwxrwxrwx. 1 root root 1 Jan 6 2016 b -> a [root@dj fuse]# rm -rf b [root@dj fuse]# ll total 0 -rw-r--r--. 1 root root 0 Jan 6 04:36 a [root@dj fuse]# ln -s a c [root@dj fuse]# ll total 0 -rw-r--r--. 1 root root 0 Jan 6 04:36 a lrwxrwxrwx. 1 root root 1 Jan 6 2016 c -> a [root@dj fuse]# rm c rm: remove symbolic link ‘c’? y [root@dj fuse]# ll total 0 -rw-r--r--. 1 root root 0 Jan 6 04:36 a [root@dj fuse]# [root@dj fuse]# [root@dj fuse]# for i in {1..3}; do cp -rf /etc etc.$i ; done [root@dj fuse]# [root@dj fuse]# ll total 336 -rw-r--r--. 1 root root 0 Jan 6 04:36 a drwxr-xr-x. 80 root root 65536 Jan 6 2016 etc.1 drwxr-xr-x. 80 root root 65536 Jan 6 2016 etc.2 drwxr-xr-x. 80 root root 65536 Jan 6 2016 etc.3 [root@dj fuse]# ln -s a d [root@dj fuse]# ln a e [root@dj fuse]# ll total 336 -rw-r--r--. 2 root root 0 Jan 6 04:36 a lrwxrwxrwx. 1 root root 1 Jan 6 2016 d -> a -rw-r--r--. 2 root root 0 Jan 6 04:36 e drwxr-xr-x. 80 root root 65536 Jan 6 2016 etc.1 drwxr-xr-x. 80 root root 65536 Jan 6 2016 etc.2 drwxr-xr-x. 80 root root 65536 Jan 6 2016 etc.3 [root@dj fuse]# [root@dj fuse]# rm -rf * [root@dj fuse]# ll total 0 [root@dj fuse]# ls [root@dj fuse]# [root@dj fuse]# touch {1..9} [root@dj fuse]# for i in {1..9}; do ln -s $i sym.$i ; done [root@dj fuse]# for i in {1..9}; do ln $i hard.$i ; done [root@dj fuse]# ls 1 2 3 4 5 6 7 8 9 hard.1 hard.2 hard.3 hard.4 hard.5 hard.6 hard.7 hard.8 hard.9 sym.1 sym.2 sym.3 sym.4 sym.5 sym.6 sym.7 sym.8 sym.9 [root@dj fuse]# ll total 0 -rw-r--r--. 2 root root 0 Jan 6 04:44 1 -rw-r--r--. 2 root root 0 Jan 6 04:44 2 -rw-r--r--. 2 root root 0 Jan 6 04:44 3 -rw-r--r--. 2 root root 0 Jan 6 04:44 4 -rw-r--r--. 2 root root 0 Jan 6 04:44 5 -rw-r--r--. 2 root root 0 Jan 6 04:44 6 -rw-r--r--. 2 root root 0 Jan 6 04:44 7 -rw-r--r--. 2 root root 0 Jan 6 04:44 8 -rw-r--r--. 2 root root 0 Jan 6 04:44 9 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.1 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.2 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.3 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.4 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.5 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.6 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.7 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.8 -rw-r--r--. 2 root root 0 Jan 6 04:44 hard.9 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.1 -> 1 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.2 -> 2 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.3 -> 3 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.4 -> 4 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.5 -> 5 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.6 -> 6 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.7 -> 7 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.8 -> 8 lrwxrwxrwx. 1 root root 1 Jan 6 2016 sym.9 -> 9 [root@dj fuse]# rm -rf * [root@dj fuse]# ll total 0 [root@dj fuse]# ls [root@dj fuse]# mount | grep tiervolume 10.70.37.165:/tiervolume on /mnt/fuse type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072) [root@dj fuse]#
*** Bug 1293798 has been marked as a duplicate of this bug. ***
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0193.html