Description of problem: When bricks are started with address sanitizer it fails with the following trace: 20:56:46 :( ⚡ /usr/local/sbin/glusterfsd -s localhost.localdomain --volfile-id r2.localhost.localdomain.home-gfs-r2_0 -p /var/lib/glusterd/vols/r2/run/localhost.localdomain-home-gfs-r2_0.pid -S /var/run/gluster/b2841ad84aaa62a337bfe471e2543598.socket --brick-name /home/gfs/r2_0 -l /var/log/glusterfs/bricks/home-gfs-r2_0.log --xlator-option *-posix.glusterd-uuid=78d15c7a-c1a2-4b5a-b0f5-df6edf282d69 --brick-port 49152 --xlator-option r2-server.listen-port=49152 -N ================================================================= ==22849==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7f1132bc2c40 at pc 0x7f114693c3e0 bp 0x7f113665d4e0 sp 0x7f113665d4d0 READ of size 8 at 0x7f1132bc2c40 thread T6 #0 0x7f114693c3df in xlator_volume_option_get_list /home/pk/workspace/gerrit-repo/libglusterfs/src/options.c:887 #1 0x7f114693c613 in xl_opt_validate /home/pk/workspace/gerrit-repo/libglusterfs/src/options.c:938 #2 0x7f1146845d5e in dict_foreach_match /home/pk/workspace/gerrit-repo/libglusterfs/src/dict.c:1254 #3 0x7f1146845b6e in dict_foreach /home/pk/workspace/gerrit-repo/libglusterfs/src/dict.c:1212 #4 0x7f114693c9df in xlator_options_validate_list /home/pk/workspace/gerrit-repo/libglusterfs/src/options.c:977 #5 0x7f114693cbb0 in xlator_options_validate /home/pk/workspace/gerrit-repo/libglusterfs/src/options.c:1005 #6 0x7f11468f85ab in glusterfs_graph_validate_options /home/pk/workspace/gerrit-repo/libglusterfs/src/graph.c:297 #7 0x7f11468f9c86 in glusterfs_graph_activate /home/pk/workspace/gerrit-repo/libglusterfs/src/graph.c:669 #8 0x40f3ac in glusterfs_process_volfp /home/pk/workspace/gerrit-repo/glusterfsd/src/glusterfsd.c:2330 #9 0x41890b in mgmt_getspec_cbk /home/pk/workspace/gerrit-repo/glusterfsd/src/glusterfsd-mgmt.c:1814 #10 0x7f11465b9229 in rpc_clnt_handle_reply /home/pk/workspace/gerrit-repo/rpc/rpc-lib/src/rpc-clnt.c:790 #11 0x7f11465b9c81 in rpc_clnt_notify /home/pk/workspace/gerrit-repo/rpc/rpc-lib/src/rpc-clnt.c:970 #12 0x7f11465b0601 in rpc_transport_notify /home/pk/workspace/gerrit-repo/rpc/rpc-lib/src/rpc-transport.c:538 #13 0x7f113813af37 in socket_event_poll_in /home/pk/workspace/gerrit-repo/rpc/rpc-transport/socket/src/socket.c:2268 #14 0x7f113813bc2f in socket_event_handler /home/pk/workspace/gerrit-repo/rpc/rpc-transport/socket/src/socket.c:2398 #15 0x7f1146957b23 in event_dispatch_epoll_handler /home/pk/workspace/gerrit-repo/libglusterfs/src/event-epoll.c:572 #16 0x7f1146958516 in event_dispatch_epoll_worker /home/pk/workspace/gerrit-repo/libglusterfs/src/event-epoll.c:675 #17 0x7f114561c6c9 in start_thread (/lib64/libpthread.so.0+0x76c9) #18 0x7f1144ef6f7e in clone (/lib64/libc.so.6+0x107f7e) 0x7f1132bc2c40 is located 0 bytes to the right of global variable 'options' defined in 'decompounder.c:895:23' (0x7f1132bc2c40) of size 0 'options' is ascii string '' SUMMARY: AddressSanitizer: global-buffer-overflow /home/pk/workspace/gerrit-repo/libglusterfs/src/options.c:887 in xlator_volume_option_get_list Shadow bytes around the buggy address: 0x0fe2a6570530: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a6570540: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a6570550: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a6570560: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a6570570: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 =>0x0fe2a6570580: 00 f9 f9 f9 f9 f9 f9 f9[f9]f9 f9 f9 00 00 00 00 0x0fe2a6570590: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a65705a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a65705b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a65705c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0fe2a65705d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb Thread T6 created by T0 here: #0 0x7f1146cc6488 in __interceptor_pthread_create (/lib64/libasan.so.3+0x31488) #1 0x7f114695882e in event_dispatch_epoll /home/pk/workspace/gerrit-repo/libglusterfs/src/event-epoll.c:727 #2 0x7f11468d20c4 in event_dispatch /home/pk/workspace/gerrit-repo/libglusterfs/src/event.c:124 #3 0x40fd2d in main /home/pk/workspace/gerrit-repo/glusterfsd/src/glusterfsd.c:2505 #4 0x7f1144e0f400 in __libc_start_main (/lib64/libc.so.6+0x20400) ==22849==ABORTING Version-Release number of selected component (if applicable): How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
REVIEW: https://review.gluster.org/16615 (performance/decompounder: Have terminal value in options[]) posted (#1) for review on master by Pranith Kumar Karampuri (pkarampu)
REVIEW: https://review.gluster.org/16615 (performance/decompounder: Have terminal value in options[]) posted (#2) for review on master by Pranith Kumar Karampuri (pkarampu)
COMMIT: https://review.gluster.org/16615 committed in master by Jeff Darcy (jdarcy) ------ commit fca31ae5faa15a02e64143b9efb7e19c3b907c2d Author: Pranith Kumar K <pkarampu> Date: Tue Feb 14 20:47:48 2017 +0530 performance/decompounder: Have terminal value in options[] Absence of terminal values is leading to buffer-over-flow errors in address sanitizer. BUG: 1422152 Change-Id: I769c0e4b5bbb3ef2849b8d1097b9def522ae08d9 Signed-off-by: Pranith Kumar K <pkarampu> Reviewed-on: https://review.gluster.org/16615 Smoke: Gluster Build System <jenkins.org> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: Jeff Darcy <jdarcy>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.11.0, please open a new bug report. glusterfs-3.11.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/announce/2017-May/000073.html [2] https://www.gluster.org/pipermail/gluster-users/