| Summary: | gnfs crashes on start-up in mem_get0 | ||
|---|---|---|---|
| Product: | [Community] GlusterFS | Reporter: | Shehjar Tikoo <shehjart> |
| Component: | core | Assignee: | shishir gowda <sgowda> |
| Status: | CLOSED WONTFIX | QA Contact: | |
| Severity: | high | Docs Contact: | |
| Priority: | low | ||
| Version: | 3.1-alpha | CC: | gluster-bugs, nsathyan, rabhat |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | Type: | --- | |
| Regression: | RTP | Mount Type: | nfs |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
|
Description
Shehjar Tikoo
2010-09-03 09:29:05 UTC
Works for me. Please reproduce it, and save the core file. [2010-09-02 17:06:43.535277] D [glusterfsd.c:336:get_volfp] glusterfsd: loading volume file /tmp/shehjart-volfiles/nfs-buildbot-posix.vol [2010-09-02 17:06:43.536373] T [graph.y:192:new_volume] parser: New node for 'posix-brick' [2010-09-02 17:06:43.536493] T [xlator.c:698:xlator_set_type] xlator: attempt to load file /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/storage/posix.so [2010-09-02 17:06:43.536793] T [graph.y:223:volume_type] parser: Type:posix-brick:storage/posix [2010-09-02 17:06:43.536892] T [graph.y:258:volume_option] parser: Option:posix-brick:directory:/tmp/glusterfsd-master-posix [2010-09-02 17:06:43.536965] T [graph.y:365:volume_end] parser: end:posix-brick [2010-09-02 17:06:43.537042] T [graph.y:192:new_volume] parser: New node for 'posix-ac' [2010-09-02 17:06:43.537106] T [xlator.c:698:xlator_set_type] xlator: attempt to load file /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/access-control.so [2010-09-02 17:06:43.537298] D [xlator.c:738:xlator_set_type] xlator: dlsym(notify) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/access-control.so: undefined symbol: notify -- neglecting [2010-09-02 17:06:43.537379] D [xlator.c:743:xlator_set_type] xlator: dlsym(dumpops) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/access-control.so: undefined symbol: dumpops -- neglecting [2010-09-02 17:06:43.537444] D [xlator.c:749:xlator_set_type] posix-ac: dlsym(mem_acct_init) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/access-control.so: undefined symbol: mem_acct_init -- neglecting [2010-09-02 17:06:43.537520] D [xlator.c:765:xlator_set_type] posix-ac: Strict option validation not enforced -- neglecting [2010-09-02 17:06:43.537580] T [graph.y:223:volume_type] parser: Type:posix-ac:features/access-control [2010-09-02 17:06:43.537641] T [graph.y:348:volume_sub] parser: child:posix-ac->posix-brick [2010-09-02 17:06:43.537700] T [graph.y:365:volume_end] parser: end:posix-ac [2010-09-02 17:06:43.537761] T [graph.y:192:new_volume] parser: New node for 'posix-locked' [2010-09-02 17:06:43.537822] T [xlator.c:698:xlator_set_type] xlator: attempt to load file /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/locks.so [2010-09-02 17:06:43.538018] D [xlator.c:738:xlator_set_type] xlator: dlsym(notify) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/locks.so: undefined symbol: notify -- neglecting [2010-09-02 17:06:43.538108] T [graph.y:223:volume_type] parser: Type:posix-locked:features/locks [2010-09-02 17:06:43.538178] T [graph.y:348:volume_sub] parser: child:posix-locked->posix-ac [2010-09-02 17:06:43.538238] T [graph.y:365:volume_end] parser: end:posix-locked [2010-09-02 17:06:43.538298] T [graph.y:192:new_volume] parser: New node for 'posix' [2010-09-02 17:06:43.538365] T [xlator.c:698:xlator_set_type] xlator: attempt to load file /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/performance/io-threads.so [2010-09-02 17:06:43.538569] D [xlator.c:738:xlator_set_type] xlator: dlsym(notify) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/performance/io-threads.so: undefined symbol: notify -- neglecting [2010-09-02 17:06:43.538650] D [xlator.c:743:xlator_set_type] xlator: dlsym(dumpops) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/performance/io-threads.so: undefined symbol: dumpops -- neglecting [2010-09-02 17:06:43.538714] T [graph.y:223:volume_type] parser: Type:posix:performance/io-threads [2010-09-02 17:06:43.538776] T [graph.y:348:volume_sub] parser: child:posix->posix-locked [2010-09-02 17:06:43.538835] T [graph.y:365:volume_end] parser: end:posix [2010-09-02 17:06:43.538901] T [graph.y:192:new_volume] parser: New node for 'nfs' [2010-09-02 17:06:43.538991] T [xlator.c:698:xlator_set_type] xlator: attempt to load file /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/nfs/server.so [2010-09-02 17:06:43.539513] D [xlator.c:743:xlator_set_type] xlator: dlsym(dumpops) on /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/nfs/server.so: undefined symbol: dumpops -- neglecting [2010-09-02 17:06:43.539606] T [graph.y:223:volume_type] parser: Type:nfs:nfs/server [2010-09-02 17:06:43.539670] T [graph.y:348:volume_sub] parser: child:nfs->posix [2010-09-02 17:06:43.539731] T [graph.y:258:volume_option] parser: Option:nfs:rpc-auth.addr.allow:* [2010-09-02 17:06:43.539790] T [graph.y:365:volume_end] parser: end:nfs [2010-09-02 17:06:43.539938] D [xlator.c:960:xlator_mem_acct_init] nfs: Allocated mem_acct_rec for 95 types [2010-09-02 17:06:43.540029] T [rpcsvc-auth.c:105:nfs_rpcsvc_auth_init_auth] nfsrpc: Authentication enabled: AUTH_UNIX [2010-09-02 17:06:43.540094] T [rpcsvc-auth.c:105:nfs_rpcsvc_auth_init_auth] nfsrpc: Authentication enabled: AUTH_NULL [2010-09-02 17:06:43.540158] T [rpcsvc.c:86:nfs_rpcsvc_stage_init] nfsrpc: event pool size: 15360 [2010-09-02 17:06:43.540292] D [rpcsvc.c:167:nfs_rpcsvc_init] nfsrpc: RPC service inited. [2010-09-02 17:06:43.540478] T [nfs.c:345:nfs_init_subvolumes] nfs: inode table lru: 90000 [2010-09-02 17:06:43.540555] D [nfs.c:349:nfs_init_subvolumes] nfs: Initing subvolume: posix [2010-09-02 17:06:43.581268] T [nfs.c:369:nfs_init_subvolumes] nfs: Inited volumes: 1 [2010-09-02 17:06:43.581523] D [nfs.c:520:init] nfs: NFS service started [2010-09-02 17:06:43.581609] D [xlator.c:960:xlator_mem_acct_init] posix: Allocated mem_acct_rec for 77 types [2010-09-02 17:06:43.581739] D [io-threads.c:2017:__iot_workers_scale] posix: scaled threads to 1 (queue_size=0/1) [2010-09-02 17:06:43.581830] D [xlator.c:960:xlator_mem_acct_init] posix-locked: Allocated mem_acct_rec for 85 types [2010-09-02 17:06:43.581898] D [xlator.c:960:xlator_mem_acct_init] posix-ac: Allocated mem_acct_rec for 74 types [2010-09-02 17:06:43.581966] D [xlator.c:960:xlator_mem_acct_init] posix-brick: Allocated mem_acct_rec for 83 types [2010-09-02 17:06:43.582147] T [posix.c:1267:posix_janitor_thread_proc] posix-brick: janitor cleaning out /.landfill [2010-09-02 17:06:43.582290] T [nfs.c:536:notify] nfs: Notification received: 1 [2010-09-02 17:06:43.582358] T [nfs.c:536:notify] nfs: Notification received: 5 [2010-09-02 17:06:43.582421] D [nfs.c:212:nfs_subvolume_set_started] nfs: Starting up: posix , vols started till now: 1 [2010-09-02 17:06:43.582490] T [nfs-fops.c:280:nfs_fop_lookup] nfs: Lookup: / [2010-09-02 17:06:43.582620] D [dict.c:326:dict_get] dict: @this=(nil) @key=0x7fa673459df9 [2010-09-02 17:06:43.582634] D [dict.c:326:dict_get] dict: @this=(nil) @key=0x7fa673459e11 [2010-09-02 17:06:43.582641] D [dict.c:326:dict_get] dict: @this=(nil) @key=0x7fa673459e29 [2010-09-02 17:06:43.619016] T [nfs.c:546:notify] nfs: All children up, starting RPC [2010-09-02 17:06:43.619146] D [nfs.c:107:nfs_init_versions] nfs: Initing protocol versions [2010-09-02 17:06:43.619221] D [mount3.c:1484:mnt3svc_init] nfs-mount: Initing Mount v3 state [2010-09-02 17:06:43.619289] T [mount3.c:1383:__mnt3_init_volume_export] nfs-mount: Volume exports enabled [2010-09-02 17:06:43.619350] T [mount3.c:1404:mnt3_init_options] nfs-mount: Initing options for: posix [2010-09-02 17:06:43.619413] T [mount3.c:1245:mnt3_init_export_ent] nfs-mount: Initing volume export: posix [2010-09-02 17:06:43.619474] D [nfs.c:124:nfs_init_versions] nfs: Starting program: MOUNT3 [2010-09-02 17:06:43.619653] T [rpcsvc.c:642:nfs_rpcsvc_conn_init] nfsrpc: tx pool: 525 [2010-09-02 17:06:43.619792] T [rpcsvc.c:649:nfs_rpcsvc_conn_init] nfsrpc: rx pool: 525 [2010-09-02 17:06:43.620259] D [rpcsvc.c:665:nfs_rpcsvc_conn_init] nfsrpc: New connection inited: sockfd: 7 [2010-09-02 17:06:43.620619] D [rpcsvc.c:2720:nfs_rpcsvc_program_register] nfsrpc: New program registered: MOUNT3, Num: 100005, Ver: 3, Port: 38465 [2010-09-02 17:06:43.620710] D [mount3.c:1528:mnt1svc_init] nfs-mount: Initing Mount v1 state [2010-09-02 17:06:43.620774] T [mount3.c:1383:__mnt3_init_volume_export] nfs-mount: Volume exports enabled [2010-09-02 17:06:43.620846] T [mount3.c:1404:mnt3_init_options] nfs-mount: Initing options for: posix [2010-09-02 17:06:43.620907] T [mount3.c:1245:mnt3_init_export_ent] nfs-mount: Initing volume export: posix [2010-09-02 17:06:43.620977] D [nfs.c:124:nfs_init_versions] nfs: Starting program: MOUNT1 [2010-09-02 17:06:43.621076] T [rpcsvc.c:642:nfs_rpcsvc_conn_init] nfsrpc: tx pool: 525 [2010-09-02 17:06:43.621183] T [rpcsvc.c:649:nfs_rpcsvc_conn_init] nfsrpc: rx pool: 525 [2010-09-02 17:06:43.621597] D [rpcsvc.c:665:nfs_rpcsvc_conn_init] nfsrpc: New connection inited: sockfd: 8 [2010-09-02 17:06:43.621810] D [rpcsvc.c:2720:nfs_rpcsvc_program_register] nfsrpc: New program registered: MOUNT1, Num: 100005, Ver: 1, Port: 38466 [2010-09-02 17:06:43.621899] T [nfs3.c:4976:nfs3_init_state] nfs-nfsv3: local pool: 225 [2010-09-02 17:06:43.622153] T [nfs3.c:4896:nfs3_init_subvolume] nfs-nfsv3: Initing state: posix [2010-09-02 17:06:43.622242] T [nfs3.c:4876:nfs3_init_subvolume_options] nfs-nfsv3: posix: read-write, no trusted_sync, no trusted_write [2010-09-02 17:06:43.622305] D [nfs.c:124:nfs_init_versions] nfs: Starting program: NFS3 [2010-09-02 17:06:43.622401] T [rpcsvc.c:642:nfs_rpcsvc_conn_init] nfsrpc: tx pool: 525 [2010-09-02 17:06:43.622499] T [rpcsvc.c:649:nfs_rpcsvc_conn_init] nfsrpc: rx pool: 525 [2010-09-02 17:06:43.622933] D [rpcsvc.c:665:nfs_rpcsvc_conn_init] nfsrpc: New connection inited: sockfd: 9 [2010-09-02 17:06:43.623146] D [rpcsvc.c:2720:nfs_rpcsvc_program_register] nfsrpc: New program registered: NFS3, Num: 100003, Ver: 3, Port: 38467 Given volfile: +------------------------------------------------------------------------------+ 1: volume posix-brick 2: type storage/posix 3: option directory /tmp/glusterfsd-master-posix 4: end-volume 5: 6: volume posix-ac 7: type features/access-control 8: subvolumes posix-brick 9: end-volume 10: 11: volume posix-locked 12: type features/locks 13: subvolumes posix-ac 14: end-volume 15: 16: volume posix 17: type performance/io-threads 18: subvolumes posix-locked 19: end-volume 20: 21: volume nfs 22: type nfs/server 23: subvolumes posix 24: option rpc-auth.addr.allow * 25: end-volume 26: 27: +------------------------------------------------------------------------------+ pending frames: patchset: git://git.sv.gnu.org/gluster.git signal received: 11 time of crash: 2010-09-02 17:06:58 configuration details: argp 1 backtrace 1 dlfcn 1 fdatasync 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.1.0git /lib/libc.so.6[0x7fa6746d8100] /tmp/glusterfsd-master/lib/libglusterfs.so.0(mem_get0+0x48)[0x7fa6752864c8] /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/features/locks.so(pl_lookup+0xcf)[0x7fa673451d4f] /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/performance/io-threads.so(iot_lookup_wrapper+0xef)[0x7fa6732436df] /tmp/glusterfsd-master/lib/libglusterfs.so.0(call_resume+0x744)[0x7fa675277f94] /tmp/glusterfsd-master/lib/glusterfs/3.1.0git/xlator/performance/io-threads.so(iot_worker+0x124)[0x7fa673247694] /lib/libpthread.so.0[0x7fa674a0e3f7] /lib/libc.so.6(clone+0x6d)[0x7fa67477dbbd] --------- *** Bug 1523 has been marked as a duplicate of this bug. *** Fixed as a part of bug 763255. patch @ http://patches.gluster.com/patch/4550 Re-opening. 1523 happens on SIGTERM to the process. This is at process start-up without CTRL-C PATCH: http://patches.gluster.com/patch/4759 in master (Add checks in mem_pool to catch possible double frees) PATCH: http://patches.gluster.com/patch/4786 in master (Cleaning up mem_get0.) After the validation/checks were put in, and the issue with io-threads xlator init failure handling was fixed, this issue is not seen anymore. Please re-open it if its seen again |