Hide Forgot
Description of problem: ======================== Ganesha got crashed on one of the node while exporting volume --------------------------- Reading symbols from /usr/bin/ganesha.nfsd...Reading symbols from /usr/lib/debug/usr/bin/ganesha.nfsd.debug...done. done. Missing separate debuginfo for /lib64/libntirpc.so.1.7 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/ba/f482e25a6a8dbfb3665ccc5c81f3bef51b5b30 Missing separate debuginfo for /lib64/libwbclient.so.0 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/b4/1852bdd635e26adba49a0e2f4e2f6e0165e27b Missing separate debuginfo for Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/ba/3372036854a6399a5260547cce7841b54ad536 Missing separate debuginfo for /lib64/libntirpc.so.1.7 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/ba/f482e25a6a8dbfb3665ccc5c81f3bef51b5b30.debug Missing separate debuginfo for /lib64/libwbclient.so.0 Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/b4/1852bdd635e26adba49a0e2f4e2f6e0165e27b.debug [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `/usr/bin/ganesha.nfsd -L /var/log/ganesha/ganesha.log -f /etc/ganesha/ganesha.c'. Program terminated with signal 11, Segmentation fault. #0 0x00007f3a901ec435 in syncop_stat (subvol=subvol@entry=0x7f3a700c2100, loc=loc@entry=0x7f3a74f2cca0, stbuf=stbuf@entry=0x7f3a74f2cce0, xdata_in=xdata_in@entry=0x0, xdata_out=xdata_out@entry=0x0) at syncop.c:2423 2423 SYNCOP(subvol, (&args), syncop_fstat_cbk, subvol->fops->stat, loc, Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 dbus-libs-1.10.24-13.el7_6.x86_64 elfutils-libelf-0.176-2.el7.x86_64 elfutils-libs-0.176-2.el7.x86_64 glibc-2.17-292.el7.x86_64 gssproxy-0.7.0-26.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-37.el7_6.x86_64 libacl-2.2.51-14.el7.x86_64 libattr-2.4.46-13.el7.x86_64 libblkid-2.23.2-61.el7.x86_64 libcap-2.22-10.el7.x86_64 libcom_err-1.42.9-16.el7.x86_64 libgcc-4.8.5-39.el7.x86_64 libgcrypt-1.5.3-14.el7.x86_64 libgpg-error-1.12-3.el7.x86_64 libnfsidmap-0.25-19.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libuuid-2.23.2-61.el7.x86_64 libwbclient-4.9.1-6.el7.x86_64 lz4-1.7.5-3.el7.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 pcre-8.32-17.el7.x86_64 samba-client-libs-4.9.1-6.el7.x86_64 systemd-libs-219-67.el7_7.1.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64 (gdb) bt #0 0x00007f3a901ec435 in syncop_stat (subvol=subvol@entry=0x7f3a700c2100, loc=loc@entry=0x7f3a74f2cca0, stbuf=stbuf@entry=0x7f3a74f2cce0, xdata_in=xdata_in@entry=0x0, xdata_out=xdata_out@entry=0x0) at syncop.c:2423 #1 0x00007f3a904aedab in pub_glfs_h_create_from_handle (fs=0x7f3a340058c0, handle=handle@entry=0x7f3a74f2ce70 "", len=len@entry=16, stat=stat@entry=0x7f3a74f2cde0) at glfs-handleops.c:1339 #2 0x00007f3a906bf30e in create_handle (export_pub=0x7f3a34005660, fh_desc=<optimized out>, pub_handle=0x7f3a74f2cf48, attrs_out=0x7f3a74f2cf70) at /usr/src/debug/nfs-ganesha-2.7.3/src/FSAL/FSAL_GLUSTER/export.c:239 #3 0x000055864ab7ea1f in mdcache_locate_host (fh_desc=0x7f3a74f2d160, export=export@entry=0x7f3a34005340, entry=entry@entry=0x7f3a74f2d0f0, attrs_out=attrs_out@entry=0x0) at /usr/src/debug/nfs-ganesha-2.7.3/src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_helpers.c:1055 #4 0x000055864ab77caa in mdcache_create_handle (exp_hdl=0x7f3a34005340, fh_desc=<optimized out>, handle=0x7f3a74f2d158, attrs_out=0x0) at /usr/src/debug/nfs-ganesha-2.7.3/src/FSAL/Stackable_FSALs/FSAL_MDCACHE/mdcache_handle.c:1578 #5 0x000055864aac6e92 in nfs4_mds_putfh (data=data@entry=0x7f3a74f2d720) at /usr/src/debug/nfs-ganesha-2.7.3/src/Protocols/NFS/nfs4_op_putfh.c:211 #6 0x000055864aac73c8 in nfs4_op_putfh (op=0x7f3a500019b0, data=0x7f3a74f2d720, resp=0x7f3a50002d60) at /usr/src/debug/nfs-ganesha-2.7.3/src/Protocols/NFS/nfs4_op_putfh.c:281 #7 0x000055864aab5703 in nfs4_Compound (arg=<optimized out>, req=<optimized out>, res=0x7f3a50001c90) at /usr/src/debug/nfs-ganesha-2.7.3/src/Protocols/NFS/nfs4_Compound.c:942 #8 0x000055864aaa8b1f in nfs_rpc_process_request (reqdata=0x7f3a50001170) at /usr/src/debug/nfs-ganesha-2.7.3/src/MainNFSD/nfs_worker_thread.c:1328 #9 0x000055864aaa7fca in nfs_rpc_decode_request (xprt=0x7f3a6c000c20, xdrs=0x7f3a50000f90) at /usr/src/debug/nfs-ganesha-2.7.3/src/MainNFSD/nfs_rpc_dispatcher_thread.c:1345 #10 0x00007f3a974e562d in svc_rqst_xprt_task () from /lib64/libntirpc.so.1.7 #11 0x00007f3a974e5b6a in svc_rqst_run_task () from /lib64/libntirpc.so.1.7 #12 0x00007f3a974edc0b in work_pool_thread () from /lib64/libntirpc.so.1.7 #13 0x00007f3a95883ea5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f3a9518e8cd in clone () from /lib64/libc.so.6 ------------------------ Version-Release number of selected component (if applicable): ======================================================= # rpm -qa | grep ganesha nfs-ganesha-gluster-2.7.3-7.el7rhgs.x86_64 nfs-ganesha-debuginfo-2.7.3-7.el7rhgs.x86_64 glusterfs-ganesha-6.0-11.el7rhgs.x86_64 nfs-ganesha-2.7.3-7.el7rhgs.x86_64 How reproducible: ================= 1/1 Steps to Reproduce: =================== 1. Created 4 node ganesha cluster 2. Created distributed-replicated volume (mani1). Exported the volume via ganesha 3. Mounted the volume on 3 clients via v4.1 4. Ran some IO's for couple of days 5. Unexported the volume.Volume was successfully unexported from all the nodes 6. Created new volume Distributed-Disperse (mani). 7. Exported the volume via ganesha ( gluster v set mani ganesha.enable on) Actual results: =============== Ganesha got crashed on one of the node Expected results: ================ Ganesha should not crash Additional info:
Tried reproducing this issue with # rpm -qa | grep ganesha nfs-ganesha-gluster-2.7.3-8.el7rhgs.x86_64 nfs-ganesha-debuginfo-2.7.3-7.el7rhgs.x86_64 glusterfs-ganesha-6.0-15.el7rhgs.x86_64 nfs-ganesha-2.7.3-8.el7rhgs.x86_64 I was unable to reproduce this issue with steps mentioned in comment #0 in 2 attempts. Will update this bug if I could hit this issue in later regression cycle
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (glusterfs bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1462