Description of problem: ======================= Brick process crashed immediate after volume start with force option. Version-Release number of selected component (if applicable): 3.12.2-11.el7rhgs.x86_64 How reproducible: hit twice Steps to Reproduce: =================== 1) Create a distributed-replicated volume and start it. 2) Set below volume options to the volume. Options Reconfigured: features.scrub: Active features.bitrot: on features.uss: on performance.client-io-threads: off nfs.disable: on transport.address-family: inet features.quota: on features.inode-quota: on features.quota-deem-statfs: on 3) FUSE mount it on a client. 4) Create few dirs on the mount point. 5) Bring down one entire dht sub-volume using gf_attach command. 6) Make metadata changes to the existing directories (like perms, chgrp, chown) 7) Now brick back the down bricks using gluster v start <vol> force. Actual results: =============== Brick process crashed immediate after volume start with force option. Expected results: ================= No crashes.
(gdb) t a a bt Thread 95 (Thread 0x7f748f7fe700 (LWP 5339)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d4082680, head=head@entry=0x7f748f7fde80) at posix-helpers.c:2151 #2 0x00007f7505aca525 in posix_fsyncer (d=0x7f74d4082680) at posix-helpers.c:2247 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 94 (Thread 0x7f751388c780 (LWP 5281)): #0 0x00007f7512208f47 in pthread_join () from /lib64/libpthread.so.0 #1 0x00007f7513407b38 in event_dispatch_epoll (event_pool=0x55d815ac2210) at event-epoll.c:746 #2 0x000055d813d9e2a7 in main (argc=19, argv=<optimized out>) at glusterfsd.c:2550 Thread 93 (Thread 0x7f74d3fff700 (LWP 5341)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 92 (Thread 0x7f74d0df2700 (LWP 5322)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 91 (Thread 0x7f74c9efc700 (LWP 5326)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d4000d40, head=head@entry=0x7f74c9efbe80) at posix-helpers.c:2151 #2 0x00007f7505aca525 in posix_fsyncer (d=0x7f74d4000d40) at posix-helpers.c:2247 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 90 (Thread 0x7f748defc700 (LWP 5343)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 89 (Thread 0x7f74d03ef700 (LWP 5344)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 88 (Thread 0x7f748d6fb700 (LWP 5345)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75049816e3 in changelog_ev_connector (data=0x7f74cc058698) at changelog-ev-handle.c:205 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 87 (Thread 0x7f748effd700 (LWP 5340)): #0 0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- #1 0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74d4639d00) at event-epoll.c:649 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 86 (Thread 0x7f74c96fb700 (LWP 5327)): #0 0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74d4082050) at event-epoll.c:649 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 85 (Thread 0x7f75091a5700 (LWP 5285)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acab30) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acab30) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 84 (Thread 0x7f748cefa700 (LWP 5346)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74cc058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 83 (Thread 0x7f7471ffb700 (LWP 5351)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74cc000d40) at posix-helpers.c:1419 #2 posix_janitor_thread_proc (data=0x7f74cc000d40) at posix-helpers.c:1467 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 82 (Thread 0x7f746aefe700 (LWP 5358)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d8058698) at changelog-ev-handle.c:205 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 81 (Thread 0x7f74717fa700 (LWP 5352)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74cc000d40, head=head@entry=0x7f74717f9e80) at posix-helpers.c:2151 #2 0x00007f7505aca525 in posix_fsyncer (d=0x7f74cc000d40) at posix-helpers.c:2247 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 80 (Thread 0x7f7468efa700 (LWP 5362)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d8000d40) at posix-helpers.c:2083 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 79 (Thread 0x7f74676f7700 (LWP 5365)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 ---Type <return> to continue, or q <return> to quit--- #1 0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d8000d40, head=head@entry=0x7f74676f6e80) at posix-helpers.c:2151 #2 0x00007f7505aca525 in posix_fsyncer (d=0x7f74d8000d40) at posix-helpers.c:2247 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 78 (Thread 0x7f74ec1f4700 (LWP 5357)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 77 (Thread 0x7f7463df1700 (LWP 5374)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4690f38) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 76 (Thread 0x7f7462def700 (LWP 5376)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d463a330) at posix-helpers.c:1897 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 75 (Thread 0x7f74625ee700 (LWP 5377)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d463a330) at posix-helpers.c:1419 #2 posix_janitor_thread_proc (data=0x7f74d463a330) at posix-helpers.c:1467 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 74 (Thread 0x7f746b6ff700 (LWP 5356)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 73 (Thread 0x7f74d026c700 (LWP 32212)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74d403b1c0) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 72 (Thread 0x7f7469efc700 (LWP 5360)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d8058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 71 (Thread 0x7f74ec4f5700 (LWP 5368)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74d4673a60) at io-threads.c:193 ---Type <return> to continue, or q <return> to quit--- #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 70 (Thread 0x7f74727fc700 (LWP 5350)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74cc000d40) at posix-helpers.c:1897 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 69 (Thread 0x7f74655f4700 (LWP 5371)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d4690f38) at changelog-ev-handle.c:205 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 68 (Thread 0x7f7472ffd700 (LWP 5349)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74cc000d40) at posix-helpers.c:2083 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 67 (Thread 0x7f7461ded700 (LWP 5378)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d463a330, head=head@entry=0x7f7461dece80) at posix-helpers.c:2151 #2 0x00007f7505aca525 in posix_fsyncer (d=0x7f74d463a330) at posix-helpers.c:2247 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 66 (Thread 0x7f74737fe700 (LWP 5348)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74cc058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 65 (Thread 0x7f74d3cb5700 (LWP 32186)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74cc03b1c0) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 64 (Thread 0x7f7473fff700 (LWP 5347)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74cc058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 63 (Thread 0x7f74fd7a6700 (LWP 32588)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74d803b1c0) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 ---Type <return> to continue, or q <return> to quit--- #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 62 (Thread 0x7f74fcea4700 (LWP 32589)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74d4673a60) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 61 (Thread 0x7f74edffb700 (LWP 32659)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74bc1bedd0) at posix-helpers.c:1419 #2 posix_janitor_thread_proc (data=0x7f74bc1bedd0) at posix-helpers.c:1467 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 60 (Thread 0x7f74d2afa700 (LWP 5318)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 59 (Thread 0x7f74d33fc700 (LWP 5315)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 58 (Thread 0x7f74d32fb700 (LWP 5317)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 57 (Thread 0x7f74d25f5700 (LWP 5319)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d4058698) at changelog-ev-handle.c:205 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 56 (Thread 0x7f74a17fa700 (LWP 5336)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d4082680) at posix-helpers.c:2083 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 55 (Thread 0x7f74d3bfd700 (LWP 5314)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acb2b0) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acb2b0) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 54 (Thread 0x7f745dde5700 (LWP 32248)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acc930) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acc930) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 53 (Thread 0x7f745d5e4700 (LWP 32249)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acccf0) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acccf0) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 52 (Thread 0x7f74a1ffb700 (LWP 5335)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d40d92e8) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 51 (Thread 0x7f745cde3700 (LWP 32250)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acd0b0) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acd0b0) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 50 (Thread 0x7f745e5e6700 (LWP 32247)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acc570) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acc570) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 49 (Thread 0x7f74fe0a6700 (LWP 32649)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 48 (Thread 0x7f75136ea700 (LWP 32650)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74bc1bc4d0) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 47 (Thread 0x7f745f5e8700 (LWP 32245)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acbdf0) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acbdf0) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 ---Type <return> to continue, or q <return> to quit--- #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 46 (Thread 0x7f74a27fc700 (LWP 5334)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d40d92e8) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 45 (Thread 0x7f745fde9700 (LWP 32244)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acba30) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acba30) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 44 (Thread 0x7f74615ec700 (LWP 32651)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 43 (Thread 0x7f74605ea700 (LWP 32210)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acb670) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acb670) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 42 (Thread 0x7f74fdfa5700 (LWP 32652)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 41 (Thread 0x7f7460deb700 (LWP 32209)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acaef0) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acaef0) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 40 (Thread 0x7f74a2ffd700 (LWP 5333)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d40d92e8) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 39 (Thread 0x7f74fd6a5700 (LWP 32653)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75049816e3 in changelog_ev_connector (data=0x7f74bc1923e8) at changelog-ev-handle.c:205 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 38 (Thread 0x7f74effff700 (LWP 32654)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74bc1923e8) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 37 (Thread 0x7f74635f0700 (LWP 5375)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d463a330) at posix-helpers.c:2083 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 36 (Thread 0x7f74ef7fe700 (LWP 32655)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74bc1923e8) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 35 (Thread 0x7f74ec4b4700 (LWP 5370)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 34 (Thread 0x7f74caefe700 (LWP 5324)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d4000d40) at posix-helpers.c:1897 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 33 (Thread 0x7f74fcda3700 (LWP 32656)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74bc1923e8) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 32 (Thread 0x7f74eeffd700 (LWP 32657)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74bc1bedd0) at posix-helpers.c:2083 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 31 (Thread 0x7f74cb6ff700 (LWP 5323)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d4000d40) at posix-helpers.c:2083 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 30 (Thread 0x7f74ee7fc700 (LWP 32658)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74bc1bedd0) at posix-helpers.c:1897 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 29 (Thread 0x7f7465df5700 (LWP 5369)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 28 (Thread 0x7f74ed7fa700 (LWP 32660)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74bc1bedd0, head=head@entry=0x7f74ed7f9e80) at posix-helpers.c:2151 #2 0x00007f7505aca525 in posix_fsyncer (d=0x7f74bc1bedd0) at posix-helpers.c:2247 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 27 (Thread 0x7f75099a6700 (LWP 5284)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f75133d222d in pool_sweeper (arg=<optimized out>) at mem-pool.c:481 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 26 (Thread 0x7f7467ef8700 (LWP 5364)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d8000d40) at posix-helpers.c:1419 #2 posix_janitor_thread_proc (data=0x7f74d8000d40) at posix-helpers.c:1467 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 25 (Thread 0x7f750a1a7700 (LWP 5283)): #0 0x00007f751220f411 in sigwait () from /lib64/libpthread.so.0 #1 0x000055d813da158b in glusterfs_sigwaiter (arg=<optimized out>) at glusterfsd.c:2137 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 24 (Thread 0x7f74696fb700 (LWP 5361)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d8058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 23 (Thread 0x7f746a6fd700 (LWP 5359)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d8058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 ---Type <return> to continue, or q <return> to quit--- Thread 22 (Thread 0x7f74a0ff9700 (LWP 5337)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d4082680) at posix-helpers.c:1897 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 21 (Thread 0x7f7466ef6700 (LWP 5366)): #0 0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74d85b9140) at event-epoll.c:649 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 20 (Thread 0x7f750a9a8700 (LWP 5282)): #0 0x00007f751220eeed in nanosleep () from /lib64/libpthread.so.0 #1 0x00007f75133b79f6 in gf_timer_proc (data=0x55d815aca310) at timer.c:174 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 19 (Thread 0x7f75064dd700 (LWP 5287)): #0 0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x55d815b0ba80) at event-epoll.c:649 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 18 (Thread 0x7f7470ff9700 (LWP 5353)): #0 0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6 #1 0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74cc5b9140) at event-epoll.c:649 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 17 (Thread 0x7f74d3cf6700 (LWP 32185)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74d803b1c0) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 16 (Thread 0x7f74ec2f5700 (LWP 5354)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 15 (Thread 0x7f748ffff700 (LWP 5338)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d4082680) at posix-helpers.c:1419 #2 posix_janitor_thread_proc (data=0x7f74d4082680) at posix-helpers.c:1467 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 14 (Thread 0x7f74ec5f6700 (LWP 5367)): ---Type <return> to continue, or q <return> to quit--- #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 13 (Thread 0x7f745ede7700 (LWP 32246)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acc1b0) at syncop.c:603 #2 0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acc1b0) at syncop.c:695 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 12 (Thread 0x7f74686f9700 (LWP 5363)): #0 0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6 #1 0x00007f7511a97394 in sleep () from /lib64/libc.so.6 #2 0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d8000d40) at posix-helpers.c:1897 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 11 (Thread 0x7f74645f2700 (LWP 5373)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4690f38) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 10 (Thread 0x7f74a37fe700 (LWP 5332)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d40d92e8) at changelog-ev-handle.c:205 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 9 (Thread 0x7f7464df3700 (LWP 5372)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4690f38) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 8 (Thread 0x7f74d1df4700 (LWP 5320)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 7 (Thread 0x7f74d15f3700 (LWP 5321)): #0 0x00007f7511ac7c03 in select () from /lib64/libc.so.6 #1 0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4058698) at changelog-ev-handle.c:350 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 6 (Thread 0x7f74ca6fd700 (LWP 5325)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d4000d40) at posix-helpers.c:1419 ---Type <return> to continue, or q <return> to quit--- #2 posix_janitor_thread_proc (data=0x7f74d4000d40) at posix-helpers.c:1467 #3 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #4 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 5 (Thread 0x7f74d04f0700 (LWP 5331)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 4 (Thread 0x7f74ec6f7700 (LWP 5328)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 3 (Thread 0x7f7504044700 (LWP 5329)): #0 0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f74ff5b1d3c in iot_worker (data=0x7f74d40bbe10) at io-threads.c:193 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 2 (Thread 0x7f74a3fff700 (LWP 5330)): #0 0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 #1 0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867 #2 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #3 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 Thread 1 (Thread 0x7f74ecff9700 (LWP 32661)): #0 rpcsvc_get_listener (svc=0x7f750007ee90, port=port@entry=65535, trans=0x7f750007fc70) at rpcsvc.c:1502 #1 0x00007f751316b39e in rpcsvc_notify (trans=0x7f749c008be0, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:823 #2 0x00007f751316d2ec in rpc_transport_unref (this=this@entry=0x7f749c008be0) at rpc-transport.c:517 #3 0x00007f7507f94be7 in socket_event_handler (fd=46, idx=<optimized out>, gen=<optimized out>, data=0x7f749c008be0, poll_in=<optimized out>, poll_out=0, poll_err=16) at socket.c:2479 #4 0x00007f7513407504 in event_dispatch_epoll_handler (event=0x7f74ecff8e80, event_pool=0x55d815ac2210) at event-epoll.c:583 #5 event_dispatch_epoll_worker (data=0x7f74bc185bc0) at event-epoll.c:659 #6 0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007f7511ad0b3d in clone () from /lib64/libc.so.6 (gdb)
Verified this BZ on glusterfs version 3.12.2-13.el7rhgs.x86_64. Followed the same steps as in the description and I didn't see any bricks getting crashed after volume start with force option. Moving this BZ to Verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2018:2607