+++ This bug was initially created as a clone of Bug #1359131 +++ Description of problem: I try to generate glusterdump file with SIGUSR1 to find out maximum op-version for the glusterd on this host. Glusterd crashes upon receiving SIGUSR1 Version-Release number of selected component (if applicable): # cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=14.04 DISTRIB_CODENAME=trusty DISTRIB_DESCRIPTION="Ubuntu 14.04.4 LTS" # dpkg -l | grep gluster ii glusterfs-client 3.8.1-ubuntu1~trusty1 amd64 clustered file-system (client package) ii glusterfs-common 3.8.1-ubuntu1~trusty1 amd64 GlusterFS common libraries and translator modules ii glusterfs-server 3.8.1-ubuntu1~trusty1 amd64 clustered file-system (server package) How reproducible: Glusterd crashes each time I send SIGUSR1 Steps to Reproduce: 1. rm -f /var/run/gluster/glusterdump.* 2. service glusterfs-server start 3. kill -SIGUSR1 $(pidof glusterd) 4. ps -ef | grep glusterd ; ls /var/run/gluster | grep glusterdump Actual results: No glusterd in the ps output. No glusterdump file created. glusterd crashes with the following lines in the log: [2016-07-22 11:22:50.923915] I [MSGID: 106493] [glusterd-rpc-ops.c:691:__glusterd_friend_update_cbk] 0-management: Received ACC from uuid: c6654b8f-3ca8-424f-8d0f-7c0b7d8108cc pending frames: patchset: git://git.gluster.com/glusterfs.git signal received: 11 time of crash: 2016-07-22 11:23:09 configuration details: argp 1 backtrace 1 dlfcn 1 libpthread 1 llistxattr 1 setfsid 1 spinlock 1 epoll.h 1 xattr.h 1 st_atim.tv_nsec 1 package-string: glusterfs 3.8.1 /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(_gf_msg_backtrace_nomem+0x92)[0x7f21f4b87b02] /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_print_trace+0x31d)[0x7f21f4b9104d] /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0)[0x7f21f3f81cb0] /lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x1d03)[0x7f21f3f96943] /lib/x86_64-linux-gnu/libc.so.6(__vsnprintf_chk+0x95)[0x7f21f4054165] /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_proc_dump_add_section_fd+0x69)[0x7f21f4bb18a9] /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_proc_dump_add_section+0xa5)[0x7f21f4bb1ab5] /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(+0x4ce3e)[0x7f21f4bb1e3e] /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_proc_dump_xlator_info+0x58)[0x7f21f4bb2658] /usr/lib/x86_64-linux-gnu/libglusterfs.so.0(gf_proc_dump_info+0x500)[0x7f21f4bb2d00] /usr/sbin/glusterd(glusterfs_sigwaiter+0xce)[0x55ffe99d2bae] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8184)[0x7f21f4318184] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f21f404537d] --------- Expected results: Glusterd creates glusterdump file in /var/run/gluster Additional info: --- Additional comment from Atin Mukherjee on 2016-07-22 08:47:08 EDT --- Does this happen everytime? I've seen it earlier as well but couldn't reproduce. --- Additional comment from Atin Mukherjee on 2016-07-22 08:48:06 EDT --- coredump file?
REVIEW: http://review.gluster.org/14987 (libglusterfs: fix glusterd statedump crash) posted (#1) for review on master by Atin Mukherjee (amukherj)
REVIEW: http://review.gluster.org/14987 (libglusterfs: fix glusterd statedump crash) posted (#2) for review on master by Atin Mukherjee (amukherj)
REVIEW: http://review.gluster.org/14987 (libglusterfs: fix glusterd statedump crash) posted (#3) for review on master by Atin Mukherjee (amukherj)
REVIEW: http://review.gluster.org/14987 (libglusterfs: fix glusterd statedump crash) posted (#4) for review on master by Atin Mukherjee (amukherj)
REVIEW: http://review.gluster.org/14987 (libglusterfs: fix glusterd statedump crash) posted (#5) for review on master by Atin Mukherjee (amukherj)
REVIEW: http://review.gluster.org/14987 (libglusterfs: fix glusterd statedump crash) posted (#6) for review on master by Atin Mukherjee (amukherj)
COMMIT: http://review.gluster.org/14987 committed in master by Jeff Darcy (jdarcy) ------ commit 049c91565dddb622b8902ccfeb36c0d414c609e1 Author: Atin Mukherjee <amukherj> Date: Tue May 31 16:44:48 2016 +0530 libglusterfs: fix glusterd statedump crash commit 3c04a91 removed setting typeStr to NULL if num_allocs is set to 0, this has caused this regression. Code has been put back like earlier and to avoid statedump printing all the NULL values check is modified to see skip the records if num_allocs is 0 instead of total_allocs Change-Id: Ib8bcc2fba908e88cf52b641c3f6bcba74f5e667c BUG: 1359190 Signed-off-by: Atin Mukherjee <amukherj> Reviewed-on: http://review.gluster.org/14987 NetBSD-regression: NetBSD Build System <jenkins.org> Smoke: Gluster Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.org> Reviewed-by: N Balachandran <nbalacha> Reviewed-by: Prashanth Pai <ppai> Reviewed-by: Jeff Darcy <jdarcy>
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.9.0, please open a new bug report. glusterfs-3.9.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution. [1] http://lists.gluster.org/pipermail/gluster-users/2016-November/029281.html [2] https://www.gluster.org/pipermail/gluster-users/