Bug 1293256 - [Tier]: "Bad file descriptor" on removal of symlink only on tiered volume
[Tier]: "Bad file descriptor" on removal of symlink only on tiered volume
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: tiering (Show other bugs)
mainline
x86_64 Linux
unspecified Severity urgent
: ---
: ---
Assigned To: Mohammed Rafi KC
bugs@gluster.org
: ZStream
Depends On:
Blocks: 1295347
  Show dependency treegraph
 
Reported: 2015-12-21 04:09 EST by Mohammed Rafi KC
Modified: 2016-06-16 09:51 EDT (History)
4 users (show)

See Also:
Fixed In Version: glusterfs-3.8rc2
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1293237
: 1295347 (view as bug list)
Environment:
Last Closed: 2016-06-16 09:51:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Comment 1 Mohammed Rafi KC 2015-12-21 04:10:35 EST
Description of problem:
=======================

rm on symbolic link errors with "Bad file descriptor" from client only on tiered volume. 

[root@dj r1]# touch a
[root@dj r1]# ln -s a b
[root@dj r1]# ls
a  b
[root@dj r1]# rm -rf b
rm: cannot remove ‘b’: Bad file descriptor
[root@dj r1]# ls
a
[root@dj r1]# 


Brick Log Snippet:
==================

[2015-12-20 19:36:21.565019] E [MSGID: 113005] [posix.c:1700:posix_unlink] 0-vol0-posix: post operation fstat failed on fd=-1 [Bad file descriptor]
[2015-12-20 19:36:21.565125] E [MSGID: 115062] [server-rpc-fops.c:1097:server_unlink_cbk] 0-vol0-server: 229668: UNLINK /r1/b (1ad39c8b-623b-455a-9914-6139453f4b31/b) ==> (Bad file descriptor) [Bad file descriptor]


Version-Release number of selected component (if applicable):
=============================================================

glusterfs-3.7.5-12.el7rhgs.x86_64


How reproducible:
=================

Always

Steps to Reproduce:
===================
1. Create Tier volume
2. Mount the volume on client
3. Create a file
4. Create symlink to the file
5. Remove symlink from client

Actual results:
===============

rm: cannot remove ‘etc.3/xdg/systemd/user’: Bad file descriptor
rm: cannot remove ‘etc.3/rc.d/rc0.d/K50netconsole’: Bad file descriptor
rm: cannot remove ‘etc.3/rc.d/rc0.d/K03rhnsd’: Bad file descriptor
rm: cannot remove ‘etc.3/rc.d/rc0.d/K90network’: Bad file descriptor
rm: cannot remove ‘etc.3/rc.d/rc5.d/S10network’: Bad file descriptor
rm: cannot remove ‘etc.3/rc.d/rc5.d/K50netconsole’: Bad file descriptor

[root@mia ~]# file /etc/xdg/systemd/user
/etc/xdg/systemd/user: symbolic link to `/etc/systemd/user'
[root@mia ~]# file /etc/rc.d/rc0.d/K50netconsole
/etc/rc.d/rc0.d/K50netconsole: symbolic link to `../init.d/netconsole'
[root@mia ~]# file /etc/rc.d/rc0.d/K03rhnsd
/etc/rc.d/rc0.d/K03rhnsd: symbolic link to `../init.d/rhnsd'
[root@mia ~]# ls -l /etc/rc.d/rc0.d/K03rhnsd
lrwxrwxrwx. 1 root root 15 Oct 14 07:17 /etc/rc.d/rc0.d/K03rhnsd -> ../init.d/rhnsd
[root@mia ~]# file /etc/rc.d/rc0.d/K90network
/etc/rc.d/rc0.d/K90network: symbolic link to `../init.d/network'
[root@mia ~]# file /etc/rc.d/rc5.d/S10network
/etc/rc.d/rc5.d/S10network: symbolic link to `../init.d/network'
[root@mia ~]# file /etc/rc.d/rc5.d/K50netconsole
/etc/rc.d/rc5.d/K50netconsole: symbolic link to `../init.d/netconsole'
[root@mia ~]#


Expected results:
=================

Symbolic link should also be removed without errors


Additional info:
================

Removal of symlink file on non-tiered volume is successful
Comment 2 Vijay Bellur 2015-12-21 04:11:33 EST
REVIEW: http://review.gluster.org/13034 (tier/unlink: open fd for special file for fdstat) posted (#1) for review on master by mohammed rafi  kc (rkavunga@redhat.com)
Comment 3 Vijay Bellur 2015-12-21 12:16:39 EST
COMMIT: http://review.gluster.org/13034 committed in master by Dan Lambright (dlambrig@redhat.com) 
------
commit 7237f8f0d72743d2522b7db1be928c5016713c59
Author: Mohammed Rafi KC <rkavunga@redhat.com>
Date:   Mon Dec 21 14:18:26 2015 +0530

    tier/unlink: open fd for special file for fdstat
    
    DUring unlink of a file, dht request stat to see whether
    the file is under migration or not. But in posix_unlink
    currently we are opening for regular files. so the fdstat
    for special files are failing with EBAD
    
    Change-Id: Ic0678e42e7701c3dffb91d98272e664b0fc646b5
    BUG: 1293256
    Signed-off-by: Mohammed Rafi KC <rkavunga@redhat.com>
    Reviewed-on: http://review.gluster.org/13034
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
    Reviewed-by: Susant Palai <spalai@redhat.com>
    Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Dan Lambright <dlambrig@redhat.com>
    Tested-by: Dan Lambright <dlambrig@redhat.com>
Comment 4 Vijay Bellur 2016-01-04 04:10:13 EST
REVIEW: http://review.gluster.org/13160 (tier/unlink: open fd for special file for fdstat) posted (#1) for review on release-3.7 by mohammed rafi  kc (rkavunga@redhat.com)
Comment 5 Niels de Vos 2016-06-16 09:51:54 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.