Bug 1745965

$ sudo gdb /usr/sbin/glusterd core.12898 
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-114.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/sbin/glusterfsd...Reading symbols from /usr/lib/debug/usr/sbin/glusterfsd.debug...done.
done.

warning: core file may not match specified executable file.
[New LWP 12898]
[New LWP 12899]
[New LWP 12900]
[New LWP 12901]
[New LWP 12902]
[New LWP 12903]
[New LWP 12904]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Core was generated by `glusterd --debug'.
Program terminated with signal 6, Aborted.
#0  0x00007fbd7ad852c7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
55	  return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
(gdb) bt
#0  0x00007fbd7ad852c7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
#1  0x00007fbd7ad869b8 in __GI_abort () at abort.c:90
#2  0x00007fbd7adc7e17 in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7fbd7aed8492 "*** %s ***: %s terminated\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:196
#3  0x00007fbd7ae66b67 in __GI___fortify_fail (msg=msg@entry=0x7fbd7aed8438 "buffer overflow detected") at fortify_fail.c:30
#4  0x00007fbd7ae64ce2 in __GI___chk_fail () at chk_fail.c:28
#5  0x00007fbd7ae643fb in ___vsnprintf_chk (s=<optimized out>, maxlen=<optimized out>, flags=<optimized out>, slen=<optimized out>, format=<optimized out>, args=args@entry=0x7ffefca6cdf8)
    at vsnprintf_chk.c:37
#6  0x00007fbd7ae64318 in ___snprintf_chk (s=s@entry=0x7ffefca6d130 "", maxlen=maxlen@entry=4096, flags=flags@entry=1, slen=slen@entry=3776, format=format@entry=0x7fbd709a8eab "%s")
    at snprintf_chk.c:35
#7  0x00007fbd70866029 in snprintf (__fmt=0x7fbd709a8eab "%s", __n=4096, __s=0x7ffefca6d130 "") at /usr/include/bits/stdio2.h:64
#8  init (this=0x557ef9f3b510) at glusterd.c:1450
#9  0x00007fbd7c740ed1 in __xlator_init (xl=0x557ef9f3b510) at xlator.c:597
#10 xlator_init (xl=xl@entry=0x557ef9f3b510) at xlator.c:623
#11 0x00007fbd7c77dbd9 in glusterfs_graph_init (graph=graph@entry=0x557ef9f37140) at graph.c:422
#12 0x00007fbd7c77e245 in glusterfs_graph_activate (graph=graph@entry=0x557ef9f37140, ctx=ctx@entry=0x557ef9ef2010) at graph.c:776
#13 0x0000557ef8287182 in glusterfs_process_volfp (ctx=ctx@entry=0x557ef9ef2010, fp=fp@entry=0x557ef9f36bb0) at glusterfsd.c:2728
#14 0x0000557ef828733d in glusterfs_volumes_init (ctx=ctx@entry=0x557ef9ef2010) at glusterfsd.c:2800
#15 0x0000557ef8282a3a in main (argc=2, argv=<optimized out>) at glusterfsd.c:2962
(gdb) f 8
#8  init (this=0x557ef9f3b510) at glusterd.c:1450
1450	        len = snprintf(logdir, PATH_MAX, "%s", DEFAULT_LOG_FILE_DIRECTORY);
(gdb) l 1396
1391	        0,
1392	    };
1393	    char rundir[PATH_MAX] = {
1394	        0,
1395	    };
1396	    char logdir[VALID_GLUSTERD_PATHMAX] = {
1397	        0,
1398	    };
1399	    char cmd_log_filename[PATH_MAX] = {
1400	        0,
(gdb) f 3
#3  0x00007fbd7ae66b67 in __GI___fortify_fail (msg=msg@entry=0x7fbd7aed8438 "buffer overflow detected") at fortify_fail.c:30
30	    __libc_message (2, "*** %s ***: %s terminated\n",

REVIEW: https://review.gluster.org/23309 (glusterd: Fixed incorrect size argument) posted (#1) for review on master by N Balachandran

REVIEW: https://review.gluster.org/23309 (glusterd: Fixed incorrect size argument) merged (#2) on master by Atin Mukherjee

RCA: rpm builds use the following flags:

$ rpm --showrc | grep stack-protector
-13: __global_compiler_flags	-O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
-fstack-protector-strong -grecord-gcc-switches %{_hardened_cflags}
%{_annotated_cflags}Thanks to Nithya for mentioning the presence of -fstack-protector
flag as a probable cause resulting in the crash which lead me to
check default rpm build macros.
-D_FORTIFY_SOURCE=2 checks for buffer overruns and aborts the process if it finds any.

From the coredump:


#8  init (this=0x557ef9f3b510) at glusterd.c:1450
1450	        len = snprintf(logdir, PATH_MAX, "%s", DEFAULT_LOG_FILE_DIRECTORY);


But

char logdir[VALID_GLUSTERD_PATHMAX] = {0,};

#define VALID_GLUSTERD_PATHMAX (PATH_MAX - (256 + 64))

so this can cause a buffer overrun.


More info at:
https://stackoverflow.com/questions/13517526/difference-between-gcc-d-fortify-source-1-and-d-fortify-source-2