Bug 1581647 - Brick process crashed immediate after volume start with force option
Summary: Brick process crashed immediate after volume start with force option
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: rpc
Version: rhgs-3.4
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: RHGS 3.4.0
Assignee: Mohit Agrawal
QA Contact: Prasad Desala
URL:
Whiteboard:
Depends On:
Blocks: 1503137
TreeView+ depends on / blocked
 
Reported: 2018-05-23 09:54 UTC by Prasad Desala
Modified: 2018-09-18 06:32 UTC (History)
5 users (show)

Fixed In Version: glusterfs-3.12.2-13
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-09-04 06:48:11 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2607 None None None 2018-09-04 06:49:57 UTC

Description Prasad Desala 2018-05-23 09:54:39 UTC
Description of problem:
=======================
Brick process crashed immediate after volume start with force option.

Version-Release number of selected component (if applicable):
3.12.2-11.el7rhgs.x86_64

How reproducible:
hit twice

Steps to Reproduce:
===================
1) Create a distributed-replicated volume and start it.
2) Set below volume options to the volume.
Options Reconfigured:
features.scrub: Active
features.bitrot: on
features.uss: on
performance.client-io-threads: off
nfs.disable: on
transport.address-family: inet
features.quota: on
features.inode-quota: on
features.quota-deem-statfs: on
3) FUSE mount it on a client.
4) Create few dirs on the mount point.
5) Bring down one entire dht sub-volume using gf_attach command.
6) Make metadata changes to the existing directories (like perms, chgrp, chown)
7) Now brick back the down bricks using gluster v start <vol> force.

Actual results:
===============
Brick process crashed immediate after volume start with force option.

Expected results:
=================
No crashes.

Comment 3 Prasad Desala 2018-05-23 10:00:30 UTC
(gdb) t a a bt

Thread 95 (Thread 0x7f748f7fe700 (LWP 5339)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d4082680, head=head@entry=0x7f748f7fde80) at posix-helpers.c:2151
#2  0x00007f7505aca525 in posix_fsyncer (d=0x7f74d4082680) at posix-helpers.c:2247
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 94 (Thread 0x7f751388c780 (LWP 5281)):
#0  0x00007f7512208f47 in pthread_join () from /lib64/libpthread.so.0
#1  0x00007f7513407b38 in event_dispatch_epoll (event_pool=0x55d815ac2210) at event-epoll.c:746
#2  0x000055d813d9e2a7 in main (argc=19, argv=<optimized out>) at glusterfsd.c:2550

Thread 93 (Thread 0x7f74d3fff700 (LWP 5341)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 92 (Thread 0x7f74d0df2700 (LWP 5322)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 91 (Thread 0x7f74c9efc700 (LWP 5326)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d4000d40, head=head@entry=0x7f74c9efbe80) at posix-helpers.c:2151
#2  0x00007f7505aca525 in posix_fsyncer (d=0x7f74d4000d40) at posix-helpers.c:2247
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 90 (Thread 0x7f748defc700 (LWP 5343)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 89 (Thread 0x7f74d03ef700 (LWP 5344)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 88 (Thread 0x7f748d6fb700 (LWP 5345)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75049816e3 in changelog_ev_connector (data=0x7f74cc058698) at changelog-ev-handle.c:205
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 87 (Thread 0x7f748effd700 (LWP 5340)):
#0  0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit--- 
#1  0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74d4639d00) at event-epoll.c:649
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 86 (Thread 0x7f74c96fb700 (LWP 5327)):
#0  0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74d4082050) at event-epoll.c:649
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 85 (Thread 0x7f75091a5700 (LWP 5285)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acab30) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acab30) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 84 (Thread 0x7f748cefa700 (LWP 5346)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74cc058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 83 (Thread 0x7f7471ffb700 (LWP 5351)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74cc000d40) at posix-helpers.c:1419
#2  posix_janitor_thread_proc (data=0x7f74cc000d40) at posix-helpers.c:1467
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 82 (Thread 0x7f746aefe700 (LWP 5358)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d8058698) at changelog-ev-handle.c:205
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 81 (Thread 0x7f74717fa700 (LWP 5352)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74cc000d40, head=head@entry=0x7f74717f9e80) at posix-helpers.c:2151
#2  0x00007f7505aca525 in posix_fsyncer (d=0x7f74cc000d40) at posix-helpers.c:2247
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 80 (Thread 0x7f7468efa700 (LWP 5362)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d8000d40) at posix-helpers.c:2083
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 79 (Thread 0x7f74676f7700 (LWP 5365)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#1  0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d8000d40, head=head@entry=0x7f74676f6e80) at posix-helpers.c:2151
#2  0x00007f7505aca525 in posix_fsyncer (d=0x7f74d8000d40) at posix-helpers.c:2247
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 78 (Thread 0x7f74ec1f4700 (LWP 5357)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 77 (Thread 0x7f7463df1700 (LWP 5374)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4690f38) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 76 (Thread 0x7f7462def700 (LWP 5376)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d463a330) at posix-helpers.c:1897
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 75 (Thread 0x7f74625ee700 (LWP 5377)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d463a330) at posix-helpers.c:1419
#2  posix_janitor_thread_proc (data=0x7f74d463a330) at posix-helpers.c:1467
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 74 (Thread 0x7f746b6ff700 (LWP 5356)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 73 (Thread 0x7f74d026c700 (LWP 32212)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74d403b1c0) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 72 (Thread 0x7f7469efc700 (LWP 5360)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d8058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 71 (Thread 0x7f74ec4f5700 (LWP 5368)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74d4673a60) at io-threads.c:193
---Type <return> to continue, or q <return> to quit---
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 70 (Thread 0x7f74727fc700 (LWP 5350)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74cc000d40) at posix-helpers.c:1897
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 69 (Thread 0x7f74655f4700 (LWP 5371)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d4690f38) at changelog-ev-handle.c:205
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 68 (Thread 0x7f7472ffd700 (LWP 5349)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74cc000d40) at posix-helpers.c:2083
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 67 (Thread 0x7f7461ded700 (LWP 5378)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74d463a330, head=head@entry=0x7f7461dece80) at posix-helpers.c:2151
#2  0x00007f7505aca525 in posix_fsyncer (d=0x7f74d463a330) at posix-helpers.c:2247
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 66 (Thread 0x7f74737fe700 (LWP 5348)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74cc058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 65 (Thread 0x7f74d3cb5700 (LWP 32186)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74cc03b1c0) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 64 (Thread 0x7f7473fff700 (LWP 5347)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74cc058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 63 (Thread 0x7f74fd7a6700 (LWP 32588)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74d803b1c0) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 62 (Thread 0x7f74fcea4700 (LWP 32589)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74d4673a60) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 61 (Thread 0x7f74edffb700 (LWP 32659)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74bc1bedd0) at posix-helpers.c:1419
#2  posix_janitor_thread_proc (data=0x7f74bc1bedd0) at posix-helpers.c:1467
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 60 (Thread 0x7f74d2afa700 (LWP 5318)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 59 (Thread 0x7f74d33fc700 (LWP 5315)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 58 (Thread 0x7f74d32fb700 (LWP 5317)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 57 (Thread 0x7f74d25f5700 (LWP 5319)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d4058698) at changelog-ev-handle.c:205
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 56 (Thread 0x7f74a17fa700 (LWP 5336)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d4082680) at posix-helpers.c:2083
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 55 (Thread 0x7f74d3bfd700 (LWP 5314)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acb2b0) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acb2b0) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---

Thread 54 (Thread 0x7f745dde5700 (LWP 32248)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acc930) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acc930) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 53 (Thread 0x7f745d5e4700 (LWP 32249)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acccf0) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acccf0) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 52 (Thread 0x7f74a1ffb700 (LWP 5335)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d40d92e8) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 51 (Thread 0x7f745cde3700 (LWP 32250)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acd0b0) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acd0b0) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 50 (Thread 0x7f745e5e6700 (LWP 32247)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acc570) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acc570) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 49 (Thread 0x7f74fe0a6700 (LWP 32649)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 48 (Thread 0x7f75136ea700 (LWP 32650)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74bc1bc4d0) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 47 (Thread 0x7f745f5e8700 (LWP 32245)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acbdf0) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acbdf0) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 46 (Thread 0x7f74a27fc700 (LWP 5334)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d40d92e8) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 45 (Thread 0x7f745fde9700 (LWP 32244)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acba30) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acba30) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 44 (Thread 0x7f74615ec700 (LWP 32651)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 43 (Thread 0x7f74605ea700 (LWP 32210)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acb670) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acb670) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 42 (Thread 0x7f74fdfa5700 (LWP 32652)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 41 (Thread 0x7f7460deb700 (LWP 32209)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acaef0) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acaef0) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 40 (Thread 0x7f74a2ffd700 (LWP 5333)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d40d92e8) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 39 (Thread 0x7f74fd6a5700 (LWP 32653)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75049816e3 in changelog_ev_connector (data=0x7f74bc1923e8) at changelog-ev-handle.c:205
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---

Thread 38 (Thread 0x7f74effff700 (LWP 32654)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74bc1923e8) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 37 (Thread 0x7f74635f0700 (LWP 5375)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d463a330) at posix-helpers.c:2083
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 36 (Thread 0x7f74ef7fe700 (LWP 32655)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74bc1923e8) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 35 (Thread 0x7f74ec4b4700 (LWP 5370)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 34 (Thread 0x7f74caefe700 (LWP 5324)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d4000d40) at posix-helpers.c:1897
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 33 (Thread 0x7f74fcda3700 (LWP 32656)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74bc1923e8) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 32 (Thread 0x7f74eeffd700 (LWP 32657)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74bc1bedd0) at posix-helpers.c:2083
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 31 (Thread 0x7f74cb6ff700 (LWP 5323)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505aca091 in posix_disk_space_check_thread_proc (data=0x7f74d4000d40) at posix-helpers.c:2083
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---

Thread 30 (Thread 0x7f74ee7fc700 (LWP 32658)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74bc1bedd0) at posix-helpers.c:1897
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 29 (Thread 0x7f7465df5700 (LWP 5369)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 28 (Thread 0x7f74ed7fa700 (LWP 32660)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505aca29b in posix_fsyncer_pick (this=this@entry=0x7f74bc1bedd0, head=head@entry=0x7f74ed7f9e80) at posix-helpers.c:2151
#2  0x00007f7505aca525 in posix_fsyncer (d=0x7f74bc1bedd0) at posix-helpers.c:2247
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 27 (Thread 0x7f75099a6700 (LWP 5284)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f75133d222d in pool_sweeper (arg=<optimized out>) at mem-pool.c:481
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 26 (Thread 0x7f7467ef8700 (LWP 5364)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d8000d40) at posix-helpers.c:1419
#2  posix_janitor_thread_proc (data=0x7f74d8000d40) at posix-helpers.c:1467
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 25 (Thread 0x7f750a1a7700 (LWP 5283)):
#0  0x00007f751220f411 in sigwait () from /lib64/libpthread.so.0
#1  0x000055d813da158b in glusterfs_sigwaiter (arg=<optimized out>) at glusterfsd.c:2137
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 24 (Thread 0x7f74696fb700 (LWP 5361)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d8058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 23 (Thread 0x7f746a6fd700 (LWP 5359)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d8058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6
---Type <return> to continue, or q <return> to quit---

Thread 22 (Thread 0x7f74a0ff9700 (LWP 5337)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d4082680) at posix-helpers.c:1897
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 21 (Thread 0x7f7466ef6700 (LWP 5366)):
#0  0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74d85b9140) at event-epoll.c:649
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 20 (Thread 0x7f750a9a8700 (LWP 5282)):
#0  0x00007f751220eeed in nanosleep () from /lib64/libpthread.so.0
#1  0x00007f75133b79f6 in gf_timer_proc (data=0x55d815aca310) at timer.c:174
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 19 (Thread 0x7f75064dd700 (LWP 5287)):
#0  0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x55d815b0ba80) at event-epoll.c:649
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 18 (Thread 0x7f7470ff9700 (LWP 5353)):
#0  0x00007f7511ad1113 in epoll_wait () from /lib64/libc.so.6
#1  0x00007f75134073d2 in event_dispatch_epoll_worker (data=0x7f74cc5b9140) at event-epoll.c:649
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 17 (Thread 0x7f74d3cf6700 (LWP 32185)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74d803b1c0) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 16 (Thread 0x7f74ec2f5700 (LWP 5354)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 15 (Thread 0x7f748ffff700 (LWP 5338)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d4082680) at posix-helpers.c:1419
#2  posix_janitor_thread_proc (data=0x7f74d4082680) at posix-helpers.c:1467
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 14 (Thread 0x7f74ec5f6700 (LWP 5367)):
---Type <return> to continue, or q <return> to quit---
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 13 (Thread 0x7f745ede7700 (LWP 32246)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75133e4e88 in syncenv_task (proc=proc@entry=0x55d815acc1b0) at syncop.c:603
#2  0x00007f75133e5d50 in syncenv_processor (thdata=0x55d815acc1b0) at syncop.c:695
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 12 (Thread 0x7f74686f9700 (LWP 5363)):
#0  0x00007f7511a974fd in nanosleep () from /lib64/libc.so.6
#1  0x00007f7511a97394 in sleep () from /lib64/libc.so.6
#2  0x00007f7505ac989d in posix_health_check_thread_proc (data=0x7f74d8000d40) at posix-helpers.c:1897
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 11 (Thread 0x7f74645f2700 (LWP 5373)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4690f38) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 10 (Thread 0x7f74a37fe700 (LWP 5332)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f75049816e3 in changelog_ev_connector (data=0x7f74d40d92e8) at changelog-ev-handle.c:205
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 9 (Thread 0x7f7464df3700 (LWP 5372)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4690f38) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 8 (Thread 0x7f74d1df4700 (LWP 5320)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 7 (Thread 0x7f74d15f3700 (LWP 5321)):
#0  0x00007f7511ac7c03 in select () from /lib64/libc.so.6
#1  0x00007f750498192a in changelog_ev_dispatch (data=0x7f74d4058698) at changelog-ev-handle.c:350
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7f74ca6fd700 (LWP 5325)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f7505ac5ca5 in janitor_get_next_fd (this=0x7f74d4000d40) at posix-helpers.c:1419
---Type <return> to continue, or q <return> to quit---
#2  posix_janitor_thread_proc (data=0x7f74d4000d40) at posix-helpers.c:1467
#3  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#4  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7f74d04f0700 (LWP 5331)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453dd2b in br_stub_worker (data=<optimized out>) at bit-rot-stub-helpers.c:375
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7f74ec6f7700 (LWP 5328)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74fed6e065 in index_worker (data=<optimized out>) at index.c:217
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7f7504044700 (LWP 5329)):
#0  0x00007f751220bcf2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f74ff5b1d3c in iot_worker (data=0x7f74d40bbe10) at io-threads.c:193
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7f74a3fff700 (LWP 5330)):
#0  0x00007f751220b945 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00007f750453f303 in br_stub_signth (arg=<optimized out>) at bit-rot-stub.c:867
#2  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#3  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f74ecff9700 (LWP 32661)):
#0  rpcsvc_get_listener (svc=0x7f750007ee90, port=port@entry=65535, trans=0x7f750007fc70) at rpcsvc.c:1502
#1  0x00007f751316b39e in rpcsvc_notify (trans=0x7f749c008be0, mydata=<optimized out>, event=<optimized out>, data=<optimized out>) at rpcsvc.c:823
#2  0x00007f751316d2ec in rpc_transport_unref (this=this@entry=0x7f749c008be0) at rpc-transport.c:517
#3  0x00007f7507f94be7 in socket_event_handler (fd=46, idx=<optimized out>, gen=<optimized out>, data=0x7f749c008be0, poll_in=<optimized out>, poll_out=0, poll_err=16) at socket.c:2479
#4  0x00007f7513407504 in event_dispatch_epoll_handler (event=0x7f74ecff8e80, event_pool=0x55d815ac2210) at event-epoll.c:583
#5  event_dispatch_epoll_worker (data=0x7f74bc185bc0) at event-epoll.c:659
#6  0x00007f7512207dd5 in start_thread () from /lib64/libpthread.so.0
#7  0x00007f7511ad0b3d in clone () from /lib64/libc.so.6
(gdb)

Comment 10 Prasad Desala 2018-07-16 10:15:38 UTC
Verified this BZ on glusterfs version 3.12.2-13.el7rhgs.x86_64.

Followed the same steps as in the description and I didn't see any bricks getting crashed after volume start with force option.

Moving this BZ to Verified state.

Comment 11 errata-xmlrpc 2018-09-04 06:48:11 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607


Note You need to log in before you can comment on or make changes to this bug.