Bug 822827
| Summary: | [glusterfs-3.3.0beta4]: quota remove command makes glusterfs client process segfault | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Raghavendra Bhat <rabhat> |
| Component: | quota | Assignee: | Raghavendra Bhat <rabhat> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | mainline | CC: | gluster-bugs, vbellur |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | glusterfs-3.4.0 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2013-07-24 17:09:20 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | glusterfs-3.3.0 | Category: | --- |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 817967 | ||
CHANGE: http://review.gluster.com/3364 (features/quota: get the top xlator from the active graph outside the list) merged in master by Anand Avati (avati) Can we backport this to release-3.3 please? http://review.gluster.com/3379 fixes the issue in release-3.3 as well. Checked with glusterfs-3.3.0. http://review.gluster.com/3379 fixes the issue in release-3.3 as well. Checked with glusterfs-3.3.0. |
Description of problem: after setting a quota limit on a certain directory, if that limit is removed via cli, then the glusterfs client segfaults while reconfiguring according to the new changes. This is the backtrace of the core generated. Core was generated by `/usr/local/sbin/glusterfs --volfile-id=mirror --volfile-server=hyperspace /mnt/'. Program terminated with signal 11, Segmentation fault. #0 0x00007f9445fac8b7 in reconfigure (this=0x7f9438068e10, options=0x7f9445573aa0) at ../../../../../xlators/features/quota/src/quota.c:3254 3254 __quota_reconfigure (this, top->itable, limit); (gdb) bt #0 0x00007f9445fac8b7 in reconfigure (this=0x7f9438068e10, options=0x7f9445573aa0) at ../../../../../xlators/features/quota/src/quota.c:3254 #1 0x00007f94476b0e5c in xlator_reconfigure_rec (old_xl=0x7f9438068e10, new_xl=0x7f943846c7b0) at ../../../libglusterfs/src/options.c:942 #2 0x00007f94476b0d87 in xlator_reconfigure_rec (old_xl=0x7f94380697d0, new_xl=0x7f943846d170) at ../../../libglusterfs/src/options.c:925 #3 0x00007f94476b0d87 in xlator_reconfigure_rec (old_xl=0x7f943805ea00, new_xl=0x7f943846db30) at ../../../libglusterfs/src/options.c:925 #4 0x00007f94476b0d87 in xlator_reconfigure_rec (old_xl=0x7f943805f3c0, new_xl=0x7f943846e4f0) at ../../../libglusterfs/src/options.c:925 #5 0x00007f94476b0d87 in xlator_reconfigure_rec (old_xl=0x7f943805fd80, new_xl=0x7f943846eeb0) at ../../../libglusterfs/src/options.c:925 #6 0x00007f94476b0d87 in xlator_reconfigure_rec (old_xl=0x7f9438061070, new_xl=0x7f943846f870) at ../../../libglusterfs/src/options.c:925 #7 0x00007f94476b0d87 in xlator_reconfigure_rec (old_xl=0x7f9438061a30, new_xl=0x7f9438470230) at ../../../libglusterfs/src/options.c:925 #8 0x00007f94476b0f66 in xlator_option_info_list (list=0x7f9438061a30, key=0x7f9438470230 "p\264\001\070\224\177", def_val=0x7f94476b0f66, descr=0x7fff71037890) at ../../../libglusterfs/src/options.c:979 #9 0x00007f94476a0eec in uuid_copy ( dst=0x7f94476a0eec "\213E\330H\211E\360H\213E\320H\211E\350\353\033H\213E\350\017\266\020H\213E\360\210\020H\203E\360\001H\203E\350\001\203E\374\001\203}\374\017~\337\311\303UH\211\345SH\203\354(\213\005\265\026#", src=0x7fff710378c0 "\020y\003q\377\177") at ../../../contrib/uuid/copy.c:43 #10 0x000000000040d1ec in glusterfs_volfile_reconfigure (newvolfile_fp=0x7f943800c920) at ../../../glusterfsd/src/glusterfsd-mgmt.c:1514 #11 0x000000000040d4d1 in mgmt_getspec_cbk (req=0x1510df8, iov=0x1510e38, count=1, myframe=0x7f944572ad18) at ../../../glusterfsd/src/glusterfsd-mgmt.c:1586 #12 0x00007f944743ac91 in rpc_clnt_handle_reply (clnt=0x15102d0, pollin=0x7f943800fd30) at ../../../../rpc/rpc-lib/src/rpc-clnt.c:788 #13 0x00007f944743b009 in rpc_clnt_notify (trans=0x1515040, mydata=0x1510300, event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f943800fd30) at ../../../../rpc/rpc-lib/src/rpc-clnt.c:907 #14 0x00007f9447437021 in rpc_transport_notify (this=0x1515040, event=RPC_TRANSPORT_MSG_RECEIVED, data=0x7f943800fd30) at ../../../../rpc/rpc-lib/src/rpc-transport.c:489 #15 0x00007f94437553a5 in socket_event_poll_in (this=0x1515040) at ../../../../../rpc/rpc-transport/socket/src/socket.c:1677 #16 0x00007f9443755919 in socket_event_handler (fd=6, idx=0, data=0x1515040, poll_in=1, poll_out=0, poll_err=0) at ../../../../../rpc/rpc-transport/socket/src/socket.c:1792 #17 0x00007f9447691cf1 in event_dispatch_epoll (event_pool=0x1508cd0) at ../../../libglusterfs/src/event.c:801 #18 0x00007f9447691f0b in event_pool_new (count=0) at ../../../libglusterfs/src/event.c:876 #19 0x00007f944769227d in list_add (new=0xffffffff47af82e8, head=0x1255) at ../../../libglusterfs/src/list.h:33 #20 0x0000000000408858 in main (argc=4, argv=0x7fff71037e78) at ../../../glusterfsd/src/glusterfsd.c:1674 (gdb) f 0 #0 0x00007f9445fac8b7 in reconfigure (this=0x7f9438068e10, options=0x7f9445573aa0) at ../../../../../xlators/features/quota/src/quota.c:3254 3254 __quota_reconfigure (this, top->itable, limit); (gdb) p this $1 = (xlator_t *) 0x7f9438068e10 (gdb) p *this $2 = {name = 0x7f943801d260 "mirror-quota", type = 0x7f943801d5e0 "features/quota", next = 0x7f9438068450, prev = 0x7f94380697d0, parents = 0x7f94380503f0, children = 0x7f943802f0c0, options = 0x7f9445572ae0, dlhandle = 0x7f9438056070, fops = 0x7f94461b09c0, cbks = 0x7f94461b0c60, dumpops = 0x0, volume_options = {next = 0x7f9438076160, prev = 0x7f9438076160}, fini = 0x7f9445fac968 <fini>, init = 0x7f9445fac0f1 <init>, reconfigure = 0x7f9445fac6a0 <reconfigure>, mem_acct_init = 0x7f9445fabba0 <mem_acct_init>, notify = 0x7f9447670f6f <default_notify>, loglevel = GF_LOG_NONE, latencies = {{min = 0, max = 0, total = 0, std = 0, mean = 0, count = 0} <repeats 46 times>}, history = 0x0, ctx = 0x14ec010, graph = 0x7f9438011ab0, itable = 0x0, init_succeeded = 1 '\001', private = 0x7f943804bb70, mem_acct = {num_types = 99, rec = 0x7f9438089ce0}, winds = 0, switched = 0 '\000', local_pool = 0x7f9438022e90} (gdb) p top $3 = (xlator_t *) 0x0 (gdb) l quota.c:3205 3200 3201 out: 3202 return; 3203 } 3204 3205 3206 int 3207 reconfigure (xlator_t *this, dict_t *options) 3208 { 3209 int32_t ret = -1; (gdb) 3210 quota_priv_t *priv = NULL; 3211 limits_t *limit = NULL, *next = NULL, *new = NULL; 3212 struct list_head head = {0, }; 3213 xlator_t *top = NULL; 3214 char found = 0; 3215 3216 priv = this->private; 3217 3218 INIT_LIST_HEAD (&head); 3219 (gdb) 3220 LOCK (&priv->lock); 3221 { 3222 list_splice_init (&priv->limit_head, &head); 3223 } 3224 UNLOCK (&priv->lock); 3225 3226 ret = quota_parse_limits (priv, this, options, &head); 3227 if (ret == -1) { 3228 gf_log ("quota", GF_LOG_WARNING, 3229 "quota reconfigure failed, " (gdb) 3230 "new changes will not take effect"); 3231 goto out; 3232 } 3233 3234 LOCK (&priv->lock); 3235 { 3236 list_for_each_entry (limit, &priv->limit_head, limit_list) { 3237 top = ((glusterfs_ctx_t *)this->ctx)->active->top; 3238 3239 __quota_reconfigure (this, top->itable, limit); (gdb) info thr 7 Thread 4697 0x00007f9446fff4bd in nanosleep () at ../sysdeps/unix/syscall-template.S:82 6 Thread 4701 0x00007f9446ffecbd in read () at ../sysdeps/unix/syscall-template.S:82 5 Thread 4696 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162 4 Thread 4700 0x00007f94469aa4f9 in __libc_readv (fd=5, vector=0x7f9440c0fe50, count=2) at ../sysdeps/unix/sysv/linux/readv.c:56 3 Thread 4695 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162 2 Thread 4694 do_sigwait (set=<value optimized out>, sig=0x7f9445162eb8) at ../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:65 * 1 Thread 4693 0x00007f9445fac8b7 in reconfigure (this=0x7f9438068e10, options=0x7f9445573aa0) at ../../../../../xlators/features/quota/src/quota.c:3254 (gdb) Version-Release number of selected component (if applicable): How reproducible: always Steps to Reproduce: 1. create a volume, start volume and mount it 2. enable quota on the volume and set a limit on the volume 3. remove the limit set on the volume Actual results: glusterfs process crashed Expected results: glusterfs process should not crash Additional info: +------------------------------------------------------------------------------+ [2012-05-18 14:50:20.330683] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-mirror-client-0: changing port to 24053 (from 0) [2012-05-18 14:50:20.330844] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-mirror-client-1: changing port to 24043 (from 0) [2012-05-18 14:50:24.322354] I [client-handshake.c:1628:select_server_supported_programs] 0-mirror-client-0: Using Program GlusterFS 3.3.0beta 4, Num (1298437), Version (330) [2012-05-18 14:50:24.323297] I [client-handshake.c:1425:client_setvolume_cbk] 0-mirror-client-0: Connected to 127.0.1.1:24053, attached to rem ote volume '/tmp/dir3'. [2012-05-18 14:50:24.323377] I [client-handshake.c:1437:client_setvolume_cbk] 0-mirror-client-0: Server and Client lk-version numbers are not same, reopening the fds [2012-05-18 14:50:24.323580] I [afr-common.c:3620:afr_notify] 0-mirror-replicate-0: Subvolume 'mirror-client-0' came back up; going online. [2012-05-18 14:50:24.323910] I [client-handshake.c:1628:select_server_supported_programs] 0-mirror-client-1: Using Program GlusterFS 3.3.0beta 4, Num (1298437), Version (330) [2012-05-18 14:50:24.324227] I [client-handshake.c:453:client_set_lk_version_cbk] 0-mirror-client-0: Server lk version = 1 [2012-05-18 14:50:24.324562] I [client-handshake.c:1425:client_setvolume_cbk] 0-mirror-client-1: Connected to 127.0.1.1:24043, attached to rem ote volume '/tmp/dir6'. [2012-05-18 14:50:24.324620] I [client-handshake.c:1437:client_setvolume_cbk] 0-mirror-client-1: Server and Client lk-version numbers are not same, reopening the fds [2012-05-18 14:50:24.336644] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 0 [2012-05-18 14:50:24.336782] I [client-handshake.c:453:client_set_lk_version_cbk] 0-mirror-client-1: Server lk version = 1 [2012-05-18 14:50:24.336845] I [fuse-bridge.c:3376:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.16 [2012-05-18 14:50:24.337640] I [afr-common.c:1962:afr_set_root_inode_on_first_lookup] 0-mirror-replicate-0: added root inode [2012-05-18 14:50:33.670473] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed [2012-05-18 14:50:34.697266] I [quota.c:3060:quota_parse_limits] 0-mirror-quota: /:23622320128 [2012-05-18 14:50:34.697288] I [quota.c:3092:quota_parse_limits] 0-mirror-quota: /:23622320128 [2012-05-18 14:50:34.697360] I [io-cache.c:1549:check_cache_size_ok] 0-mirror-io-cache: Max cache size is 4066508800 [2012-05-18 14:50:34.697403] I [io-cache.c:1549:check_cache_size_ok] 0-mirror-quick-read: Max cache size is 4066508800 [2012-05-18 14:50:40.647930] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed [2012-05-18 14:50:41.676985] I [quota.c:3085:quota_parse_limits] 0-mirror-quota: no "limit-set" option provided pending frames: patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2012-05-18 14:50:41 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 : gluster volume info Volume Name: mirror Type: Replicate Volume ID: d4d776ee-2695-4470-8e12-21526b6ca72d Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: hyperspace:/tmp/dir3 Brick2: hyperspace:/tmp/dir6 Options Reconfigured: features.quota: on root@hyperspace:~#