Bug 1001631 - Quota: glusterfsd process crashed
Quota: glusterfsd process crashed
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: glusterd (Show other bugs)
2.1
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Raghavendra Bhat
Saurabh
: ZStream
: 1001919 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-08-27 08:33 EDT by Raghavendra Bhat
Modified: 2016-01-19 01:12 EST (History)
10 users (show)

See Also:
Fixed In Version: glusterfs-3.4.0.30rhs
Doc Type: Bug Fix
Doc Text:
Cause: When nameless lookup on a gfid came, posix was trying to resolve it using the gfid handle by doing readlink on it. The return value of readlink was received in a wrong variable. Consequence: The above cause made posix think readlink was successfu, despite the failure and continue leading to glusterfsd process being crashed. Fix: Now the return value of readlink call is received in a proper variable and checked for failure before continuing. Result:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-11-27 10:34:21 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Raghavendra Bhat 2013-08-27 08:33:28 EDT
Description of problem:
if the quota is set on a non existing directory, then glusterfsd process crashes with the below backtrace.

Simply trying to set quota limit on a non existing directory does not crash the brick process. To reproduce it,  one has to add a new brick, perform rebalance, remove a directory upon which quota limit was set. Now trying to set limit on the removed directory will crash the glusterfsd process.


this is the backtrace

155	        strcat (dir_name, "/");
Missing separate debuginfos, use: debuginfo-install glibc-2.15-59.fc17.x86_64 keyutils-libs-1.5.5-2.fc17.x86_64 krb5-libs-1.10.2-12.fc17.x86_64 libaio-0.3.109-5.fc17.x86_64 libcom_err-1.42.3-3.fc17.x86_64 libgcc-4.7.2-2.fc17.x86_64 libselinux-2.1.10-3.fc17.x86_64 openssl-1.0.0k-1.fc17.x86_64 zlib-1.2.5-7.fc17.x86_64
(gdb) bt
#0  0x00007fc7319ebff0 in posix_make_ancestryfromgfid (this=0x13bdef0, path=0x7fc7292117c0 "", pathsize=4097, head=0x7fc6fc002970, type=1, 
    gfid=0x7fc72a26d0d0 "\334\303\062\017\347\227G\205\253;\257\204\262\063$4\001", handle_size=67, priv_base_path=0x13eb720 "/export1/vol", itable=0x13f6970, 
    parent=0x7fc7292117b8, xdata=0x7fc73423624c) at ../../../../../xlators/storage/posix/src/posix-handle.c:155
#1  0x00007fc7319ddb4f in posix_get_ancestry_directory (this=0x13bdef0, real_path=0x7fc7292129e0 "/export1/vol/dir/", loc=0x7fc7342f24c8, dict=0x7fc734236cb0, type=1, 
    op_errno=0x7fc729212998, xdata=0x7fc73423624c) at ../../../../../xlators/storage/posix/src/posix.c:2764
#2  0x00007fc7319dea54 in posix_get_ancestry (this=0x13bdef0, real_path=0x7fc7292129e0 "/export1/vol/dir/", loc=0x7fc7342f24c8, dict=0x7fc734236cb0, type=1, 
    op_errno=0x7fc729212998, xdata=0x7fc73423624c) at ../../../../../xlators/storage/posix/src/posix.c:3084
#3  0x00007fc7319e8f89 in _posix_xattr_get_set (xattr_req=0x7fc73423624c, key=0x146da20 "glusterfs.ancestry.path", data=0x7fc734066b1c, xattrargs=0x7fc729212960)
    at ../../../../../xlators/storage/posix/src/posix-helpers.c:319
#4  0x00007fc735a2fa87 in dict_foreach (dict=0x7fc73423624c, fn=0x7fc7319e8b5b <_posix_xattr_get_set>, data=0x7fc729212960) at ../../../libglusterfs/src/dict.c:1109
#5  0x00007fc7319e9a72 in posix_lookup_xattr_fill (this=0x13bdef0, real_path=0x7fc7292129e0 "/export1/vol/dir/", loc=0x7fc7342f24c8, xattr_req=0x7fc73423624c, 
    buf=0x7fc729212ac0) at ../../../../../xlators/storage/posix/src/posix-helpers.c:558
#6  0x00007fc7319cee8c in posix_lookup (frame=0x7fc734879824, this=0x13bdef0, loc=0x7fc7342f24c8, xdata=0x7fc73423624c)
    at ../../../../../xlators/storage/posix/src/posix.c:156
#7  0x00007fc735a43190 in default_lookup (frame=0x7fc734879824, this=0x13bf720, loc=0x7fc7342f24c8, xdata=0x7fc73423624c) at ../../../libglusterfs/src/defaults.c:1253
#8  0x00007fc7315a8d79 in posix_acl_lookup (frame=0x7fc734883200, this=0x13c0910, loc=0x7fc7342f24c8, xattr=0x7fc73423624c)
    at ../../../../../xlators/system/posix-acl/src/posix-acl.c:793
#9  0x00007fc731395445 in pl_lookup (frame=0x7fc734876dd0, this=0x13c19c0, loc=0x7fc7342f24c8, xdata=0x7fc73423624c)
    at ../../../../../xlators/features/locks/src/posix.c:2081
#10 0x00007fc73116f5e1 in iot_lookup_wrapper (frame=0x7fc734882134, this=0x13c2a80, loc=0x7fc7342f24c8, xdata=0x7fc73423624c)
    at ../../../../../xlators/performance/io-threads/src/io-threads.c:346
#11 0x00007fc735a58251 in call_resume_wind (stub=0x7fc7342f2488) at ../../../libglusterfs/src/call-stub.c:2312
#12 0x00007fc735a5f0a8 in call_resume (stub=0x7fc7342f2488) at ../../../libglusterfs/src/call-stub.c:2645
#13 0x00007fc73116efbe in iot_worker (data=0x13e7190) at ../../../../../xlators/performance/io-threads/src/io-threads.c:191
#14 0x0000003fc5807d14 in start_thread () from /lib64/libpthread.so.0
#15 0x0000003fc54f168d in clone () from /lib64/libc.so.6
(gdb)  f 0
#0  0x00007fc7319ebff0 in posix_make_ancestryfromgfid (this=0x13bdef0, path=0x7fc7292117c0 "", pathsize=4097, head=0x7fc6fc002970, type=1, 
    gfid=0x7fc72a26d0d0 "\334\303\062\017\347\227G\205\253;\257\204\262\063$4\001", handle_size=67, priv_base_path=0x13eb720 "/export1/vol", itable=0x13f6970, 
    parent=0x7fc7292117b8, xdata=0x7fc73423624c) at ../../../../../xlators/storage/posix/src/posix-handle.c:155
155	        strcat (dir_name, "/");
(gdb) p dir_name
$1 = 0x0
(gdb) l
150	        len = readlink (dir_handle, linkname, 512);
151	        linkname[len] = '\0';
152	
153	        pgfidstr = strtok_r (linkname + SLEN("../../00/00/"), "/", &saveptr);
154	        dir_name = strtok_r (NULL, "/", &saveptr);
155	        strcat (dir_name, "/");
156	        uuid_parse (pgfidstr, tmp_gfid);
157	
158	        ret = posix_make_ancestryfromgfid (this, path, pathsize, head, type,
159	                                           tmp_gfid, handle_size,
(gdb) p len
$2 = 18446744073709551615
(gdb) p dir_handle
$3 = 0x7fc7292115e0 "/export1/vol/.glusterfs/dc/c3/dcc3320f-e797-4785-ab3b-af84b2332434"
(gdb) p pgfidstr
$4 = 0x0
(gdb) l posix_make_ancestryfromgfid
105	posix_make_ancestryfromgfid (xlator_t *this, char *path, int pathsize,
106	                             gf_dirent_t *head, int type, uuid_t gfid,
107	                             const size_t handle_size,
108	                             const char *priv_base_path, inode_table_t *itable,
109	                             inode_t **parent, dict_t *xdata)
110	{
111	        char        *linkname   = NULL; /* "../../<gfid[0]>/<gfid[1]/"
112	                                         "<gfidstr>/<NAME_MAX>" */
113	        char        *dir_handle = NULL;
114	        char        *dir_name   = NULL;
(gdb) 
115	        char        *pgfidstr   = NULL;
116	        char        *saveptr    = NULL;
117	        size_t       len        = 0;
118	        inode_t     *inode      = NULL;
119	        struct iatt  iabuf      = {0, };
120	        int          ret        = -1;
121	        uuid_t       tmp_gfid   = {0};
122	

Above, len is declared as an unsigned entity. But readlink returns a -ve value upon error and len will become a huge +ve value if readlink fails. Also the return value of readlink is never checked.



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 Raghavendra G 2013-09-12 03:37:04 EDT
*** Bug 1001919 has been marked as a duplicate of this bug. ***
Comment 2 Raghavendra G 2013-09-12 03:40:25 EDT
Patch was reviewed at
https://code.engineering.redhat.com/gerrit/#/c/12036/

Tag v3.4.0.30rhs has this fix.
Comment 3 shylesh 2013-10-07 03:19:20 EDT
Verified on 3.4.0.33rhs-1.el6rhs.x86_64
Comment 4 errata-xmlrpc 2013-11-27 10:34:21 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1769.html

Note You need to log in before you can comment on or make changes to this bug.