Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 585640 Details for
Bug 823255
[1d939fe7adef651b90bb5c4cd5843768417f0138]: blocked entrylks causes hang
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
log file of the client process started by geo-replication
mount_gsync.log (text/x-log), 114.26 KB, created by
Raghavendra Bhat
on 2012-05-20 13:01:35 UTC
(
hide
)
Description:
log file of the client process started by geo-replication
Filename:
MIME Type:
Creator:
Raghavendra Bhat
Created:
2012-05-20 13:01:35 UTC
Size:
114.26 KB
patch
obsolete
>[2012-05-20 16:38:34.866484] I [glusterfsd.c:1651:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.3git >[2012-05-20 16:38:34.872392] I [io-cache.c:1549:check_cache_size_ok] 0-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:38:34.872519] I [io-cache.c:1549:check_cache_size_ok] 0-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:38:34.877352] I [client.c:2142:notify] 0-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:38:34.878425] I [client.c:2142:notify] 0-new-client-1: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-replicate-0 > 20: type cluster/replicate > 21: subvolumes new-client-0 new-client-1 > 22: end-volume > 23: > 24: volume new-write-behind > 25: type performance/write-behind > 26: subvolumes new-replicate-0 > 27: end-volume > 28: > 29: volume new-read-ahead > 30: type performance/read-ahead > 31: subvolumes new-write-behind > 32: end-volume > 33: > 34: volume new-io-cache > 35: type performance/io-cache > 36: subvolumes new-read-ahead > 37: end-volume > 38: > 39: volume new-quick-read > 40: type performance/quick-read > 41: subvolumes new-io-cache > 42: end-volume > 43: > 44: volume new-md-cache > 45: type performance/md-cache > 46: subvolumes new-quick-read > 47: end-volume > 48: > 49: volume new > 50: type debug/io-stats > 51: option latency-measurement off > 52: option count-fop-hits off > 53: subvolumes new-md-cache > 54: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:38:34.879856] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:38:34.880007] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:38:38.872311] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:38:38.873325] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:38:38.873408] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:38:38.873609] I [afr-common.c:3628:afr_notify] 0-new-replicate-0: Subvolume 'new-client-0' came back up; going online. >[2012-05-20 16:38:38.873745] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-0: Server lk version = 1 >[2012-05-20 16:38:38.874066] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:38:38.874744] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:38:38.874821] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:38:38.886583] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 0 >[2012-05-20 16:38:38.886734] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-1: Server lk version = 1 >[2012-05-20 16:38:38.886808] I [fuse-bridge.c:3376:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.16 >[2012-05-20 16:38:38.887726] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 0-new-replicate-0: added root inode >[2012-05-20 16:38:40.755642] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-1: remote operation failed: No data available. Path: / (00000000-0000-0000-0000-000000000001). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:38:40.756513] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-0: remote operation failed: No data available. Path: / (00000000-0000-0000-0000-000000000001). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:38:54.014208] I [fuse-bridge.c:4037:fuse_thread_proc] 0-fuse: unmounting /tmp/gsyncd-aux-mount-IBxbrg >[2012-05-20 16:38:54.014607] W [glusterfsd.c:816:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f693636b04d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7f69369aed8c] (-->/usr/local/sbin/glusterfs(glusterfs_sigwaiter+0xea) [0x407f1f]))) 0-: received signum (15), shutting down >[2012-05-20 16:38:54.653400] I [glusterfsd.c:1651:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.3git >[2012-05-20 16:38:54.663368] I [io-cache.c:1549:check_cache_size_ok] 0-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:38:54.663440] I [io-cache.c:1549:check_cache_size_ok] 0-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:38:54.666098] I [client.c:2142:notify] 0-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:38:54.666673] I [client.c:2142:notify] 0-new-client-1: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-replicate-0 > 20: type cluster/replicate > 21: subvolumes new-client-0 new-client-1 > 22: end-volume > 23: > 24: volume new-write-behind > 25: type performance/write-behind > 26: subvolumes new-replicate-0 > 27: end-volume > 28: > 29: volume new-read-ahead > 30: type performance/read-ahead > 31: subvolumes new-write-behind > 32: end-volume > 33: > 34: volume new-io-cache > 35: type performance/io-cache > 36: subvolumes new-read-ahead > 37: end-volume > 38: > 39: volume new-quick-read > 40: type performance/quick-read > 41: subvolumes new-io-cache > 42: end-volume > 43: > 44: volume new-md-cache > 45: type performance/md-cache > 46: subvolumes new-quick-read > 47: end-volume > 48: > 49: volume new > 50: type debug/io-stats > 51: option latency-measurement off > 52: option count-fop-hits off > 53: subvolumes new-md-cache > 54: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:38:54.667303] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:38:54.667376] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:38:58.661455] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:38:58.662332] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:38:58.662470] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:38:58.662642] I [afr-common.c:3628:afr_notify] 0-new-replicate-0: Subvolume 'new-client-0' came back up; going online. >[2012-05-20 16:38:58.662994] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:38:58.663222] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-0: Server lk version = 1 >[2012-05-20 16:38:58.663705] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:38:58.663784] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:38:58.676522] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 0 >[2012-05-20 16:38:58.676666] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-1: Server lk version = 1 >[2012-05-20 16:38:58.676731] I [fuse-bridge.c:3376:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.16 >[2012-05-20 16:38:58.677568] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 0-new-replicate-0: added root inode >[2012-05-20 16:39:03.714633] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2012-05-20 16:39:04.824810] I [io-cache.c:1549:check_cache_size_ok] 1-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:39:04.824907] I [io-cache.c:1549:check_cache_size_ok] 1-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:39:04.825034] I [quota.c:3085:quota_parse_limits] 1-new-quota: no "limit-set" option provided >[2012-05-20 16:39:04.829602] I [client.c:2142:notify] 1-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:39:04.830584] I [client.c:2142:notify] 1-new-client-1: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-replicate-0 > 20: type cluster/replicate > 21: subvolumes new-client-0 new-client-1 > 22: end-volume > 23: > 24: volume new-quota > 25: type features/quota > 26: option timeout 0 > 27: subvolumes new-replicate-0 > 28: end-volume > 29: > 30: volume new-write-behind > 31: type performance/write-behind > 32: subvolumes new-quota > 33: end-volume > 34: > 35: volume new-read-ahead > 36: type performance/read-ahead > 37: subvolumes new-write-behind > 38: end-volume > 39: > 40: volume new-io-cache > 41: type performance/io-cache > 42: subvolumes new-read-ahead > 43: end-volume > 44: > 45: volume new-quick-read > 46: type performance/quick-read > 47: subvolumes new-io-cache > 48: end-volume > 49: > 50: volume new-md-cache > 51: type performance/md-cache > 52: subvolumes new-quick-read > 53: end-volume > 54: > 55: volume new > 56: type debug/io-stats > 57: option latency-measurement off > 58: option count-fop-hits off > 59: subvolumes new-md-cache > 60: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:39:04.833581] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 1-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:39:04.833651] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 1-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:39:08.665025] I [client-handshake.c:1628:select_server_supported_programs] 1-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:39:08.665929] I [client-handshake.c:1425:client_setvolume_cbk] 1-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:39:08.666027] I [client-handshake.c:1437:client_setvolume_cbk] 1-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:39:08.666173] I [afr-common.c:3628:afr_notify] 1-new-replicate-0: Subvolume 'new-client-0' came back up; going online. >[2012-05-20 16:39:08.666266] I [client-handshake.c:1628:select_server_supported_programs] 1-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:39:08.666433] I [client-handshake.c:453:client_set_lk_version_cbk] 1-new-client-0: Server lk version = 1 >[2012-05-20 16:39:08.666844] I [client-handshake.c:1425:client_setvolume_cbk] 1-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:39:08.666899] I [client-handshake.c:1437:client_setvolume_cbk] 1-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:39:08.683953] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 1 >[2012-05-20 16:39:08.684043] I [client-handshake.c:453:client_set_lk_version_cbk] 1-new-client-1: Server lk version = 1 >[2012-05-20 16:39:08.983792] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 1-new-replicate-0: added root inode >[2012-05-20 16:39:08.984459] I [client.c:2151:notify] 0-new-client-0: current graph is no longer active, destroying rpc_client >[2012-05-20 16:39:08.984596] I [client.c:2151:notify] 0-new-client-1: current graph is no longer active, destroying rpc_client >[2012-05-20 16:39:08.984666] I [client.c:2090:client_rpc_notify] 0-new-client-0: disconnected >[2012-05-20 16:39:08.984785] I [client.c:2090:client_rpc_notify] 0-new-client-1: disconnected >[2012-05-20 16:39:08.984847] E [afr-common.c:3665:afr_notify] 0-new-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up. >[2012-05-20 16:40:04.675628] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (a72d64ac-cc23-413d-a65a-ea5a209c5a56) >[2012-05-20 16:40:04.675757] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (a72d64ac-cc23-413d-a65a-ea5a209c5a56) >[2012-05-20 16:40:07.169583] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (efa20558-68fa-4389-ac6f-88e62a50fb27) >[2012-05-20 16:40:07.169654] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (efa20558-68fa-4389-ac6f-88e62a50fb27) >[2012-05-20 16:40:08.390186] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (f0703dcd-df40-463e-9550-971b3c8328e5) >[2012-05-20 16:40:08.390251] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (f0703dcd-df40-463e-9550-971b3c8328e5) >[2012-05-20 16:40:08.391365] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.er1 (4e3e63ab-1260-488b-9be6-f39812417bf3) >[2012-05-20 16:40:08.391427] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.er1 (4e3e63ab-1260-488b-9be6-f39812417bf3) >[2012-05-20 16:40:09.761325] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (601226e0-447c-4835-8717-6fa1f625ecad) >[2012-05-20 16:40:09.762667] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (601226e0-447c-4835-8717-6fa1f625ecad) >[2012-05-20 16:40:13.511666] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.er1 (e66ca931-9f99-4225-86f5-1799540e4a27) >[2012-05-20 16:40:13.511730] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.er1 (e66ca931-9f99-4225-86f5-1799540e4a27) >[2012-05-20 16:40:13.519320] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (56e98a52-a7cc-44a7-9031-17f7e2a36b02) >[2012-05-20 16:40:13.519387] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (56e98a52-a7cc-44a7-9031-17f7e2a36b02) >[2012-05-20 16:40:17.353761] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (14ee6cab-a27f-4637-b3ea-a811b89e95d8) >[2012-05-20 16:40:17.359479] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/conftest.c (14ee6cab-a27f-4637-b3ea-a811b89e95d8) >[2012-05-20 16:40:22.289912] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/confpmiroZ/out (31becfdd-6592-4b78-800c-9fb59140500e) >[2012-05-20 16:40:22.289979] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/confpmiroZ/out (31becfdd-6592-4b78-800c-9fb59140500e) >[2012-05-20 16:40:27.115355] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/glusterfs.spec (c59f51c2-cae8-4d0b-b590-132e457d2cd8) >[2012-05-20 16:40:27.115572] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/glusterfs.spec (c59f51c2-cae8-4d0b-b590-132e457d2cd8) >[2012-05-20 16:40:27.254498] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfsd-Redhat (66f809e7-508f-4c1b-a95f-92833f4653eb) >[2012-05-20 16:40:27.255508] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfsd-Redhat (66f809e7-508f-4c1b-a95f-92833f4653eb) >[2012-05-20 16:40:27.258790] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfsd-Debian (d503bf8c-d764-4fc0-8aba-5244ceb005c9) >[2012-05-20 16:40:27.258914] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfsd-Debian (d503bf8c-d764-4fc0-8aba-5244ceb005c9) >[2012-05-20 16:40:27.264571] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfsd-SuSE (ab927109-c4c3-4506-bdac-80d094730bac) >[2012-05-20 16:40:27.264727] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfsd-SuSE (ab927109-c4c3-4506-bdac-80d094730bac) >[2012-05-20 16:40:27.270375] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-0: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfs-server.plist (3ec6ffbd-ee24-4cfe-9e13-bdf206166bcc) >[2012-05-20 16:40:27.270689] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 1-new-client-1: remote operation failed: Stale NFS file handle. Path: /glusterfs-3.0.0pre2/extras/init.d/glusterfs-server.plist (3ec6ffbd-ee24-4cfe-9e13-bdf206166bcc) >[2012-05-20 16:40:28.384550] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2012-05-20 16:40:32.228326] I [io-cache.c:1549:check_cache_size_ok] 2-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:40:32.228448] I [io-cache.c:1549:check_cache_size_ok] 2-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:40:32.228599] I [quota.c:3085:quota_parse_limits] 2-new-quota: no "limit-set" option provided >[2012-05-20 16:40:32.235530] I [client.c:2142:notify] 2-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:40:32.236335] I [client.c:2142:notify] 2-new-client-1: parent translators are ready, attempting connect on transport >[2012-05-20 16:40:32.237289] I [client.c:2142:notify] 2-new-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-client-2 > 20: type protocol/client > 21: option remote-host hyperspace > 22: option remote-subvolume /mnt/sda10/export3 > 23: option transport-type tcp > 24: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 25: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 26: end-volume > 27: > 28: volume new-replicate-0 > 29: type cluster/replicate > 30: subvolumes new-client-0 new-client-1 new-client-2 > 31: end-volume > 32: > 33: volume new-quota > 34: type features/quota > 35: option timeout 0 > 36: subvolumes new-replicate-0 > 37: end-volume > 38: > 39: volume new-write-behind > 40: type performance/write-behind > 41: subvolumes new-quota > 42: end-volume > 43: > 44: volume new-read-ahead > 45: type performance/read-ahead > 46: subvolumes new-write-behind > 47: end-volume > 48: > 49: volume new-io-cache > 50: type performance/io-cache > 51: subvolumes new-read-ahead > 52: end-volume > 53: > 54: volume new-quick-read > 55: type performance/quick-read > 56: subvolumes new-io-cache > 57: end-volume > 58: > 59: volume new-md-cache > 60: type performance/md-cache > 61: subvolumes new-quick-read > 62: end-volume > 63: > 64: volume new > 65: type debug/io-stats > 66: option latency-measurement off > 67: option count-fop-hits off > 68: subvolumes new-md-cache > 69: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:40:32.239143] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 2-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:40:32.239297] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 2-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:40:32.239385] E [client-handshake.c:1709:client_query_portmap_cbk] 2-new-client-2: failed to get the port number for remote subvolume >[2012-05-20 16:40:32.239494] I [client.c:2090:client_rpc_notify] 2-new-client-2: disconnected >[2012-05-20 16:40:35.679723] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:40:36.787575] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:40:36.794869] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:40:36.794896] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:40:36.794950] I [afr-common.c:3628:afr_notify] 2-new-replicate-0: Subvolume 'new-client-0' came back up; going online. >[2012-05-20 16:40:36.795096] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:40:36.795120] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:40:36.805557] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 2 >[2012-05-20 16:40:36.805623] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 2-new-client-2: changing port to 24071 (from 0) >[2012-05-20 16:40:36.805684] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-0: Server lk version = 1 >[2012-05-20 16:40:36.806565] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-1: Server lk version = 1 >[2012-05-20 16:40:36.807007] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 2-new-replicate-0: added root inode >[2012-05-20 16:40:36.808311] I [client.c:2151:notify] 1-new-client-0: current graph is no longer active, destroying rpc_client >[2012-05-20 16:40:36.808535] I [client.c:2151:notify] 1-new-client-1: current graph is no longer active, destroying rpc_client >[2012-05-20 16:40:36.808946] I [socket.c:2315:socket_submit_request] 1-new-client-0: not connected (priv->connected = 255) >[2012-05-20 16:40:36.808971] W [rpc-clnt.c:1498:rpc_clnt_submit] 1-new-client-0: failed to submit rpc-request (XID: 0x5675x Program: GlusterFS 3.1, ProgVers: 330, Proc: 30) to rpc-transport (new-client-0) >[2012-05-20 16:40:36.808996] W [client3_1-fops.c:1545:client3_1_finodelk_cbk] 1-new-client-0: remote operation failed: Transport endpoint is not connected >[2012-05-20 16:40:36.809074] I [socket.c:2315:socket_submit_request] 1-new-client-1: not connected (priv->connected = 255) >[2012-05-20 16:40:36.809095] W [rpc-clnt.c:1498:rpc_clnt_submit] 1-new-client-1: failed to submit rpc-request (XID: 0x5811x Program: GlusterFS 3.1, ProgVers: 330, Proc: 30) to rpc-transport (new-client-1) >[2012-05-20 16:40:36.809115] W [client3_1-fops.c:1545:client3_1_finodelk_cbk] 1-new-client-1: remote operation failed: Transport endpoint is not connected >[2012-05-20 16:40:36.809163] I [socket.c:1798:socket_event_handler] 1-transport: disconnecting now >[2012-05-20 16:40:36.809196] I [client.c:2090:client_rpc_notify] 1-new-client-0: disconnected >[2012-05-20 16:40:36.809230] I [client.c:2090:client_rpc_notify] 1-new-client-1: disconnected >[2012-05-20 16:40:36.809249] E [afr-common.c:3665:afr_notify] 1-new-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up. >[2012-05-20 16:40:40.726342] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 11588: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) / => -1 (Transport endpoint is not connected) >[2012-05-20 16:40:41.091005] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-2: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:40:41.091348] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-2: Connected to 192.168.1.88:24071, attached to remote volume '/mnt/sda10/export3'. >[2012-05-20 16:40:41.091378] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:40:41.091626] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-2: Server lk version = 1 >[2012-05-20 16:40:41.095597] I [fuse-bridge.c:4037:fuse_thread_proc] 0-fuse: unmounting /tmp/gsyncd-aux-mount-qXKmER >[2012-05-20 16:40:41.095986] W [glusterfsd.c:816:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f07e850004d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7f07e8b43d8c] (-->/usr/local/sbin/glusterfs(glusterfs_sigwaiter+0xea) [0x407f1f]))) 0-: received signum (15), shutting down >[2012-05-20 16:40:51.682045] I [glusterfsd.c:1651:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.3git >[2012-05-20 16:40:51.686527] I [io-cache.c:1549:check_cache_size_ok] 0-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:40:51.686632] I [io-cache.c:1549:check_cache_size_ok] 0-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:40:51.686732] I [quota.c:3085:quota_parse_limits] 0-new-quota: no "limit-set" option provided >[2012-05-20 16:40:51.690740] I [client.c:2142:notify] 0-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:40:51.691463] I [client.c:2142:notify] 0-new-client-1: parent translators are ready, attempting connect on transport >[2012-05-20 16:40:51.691841] I [client.c:2142:notify] 0-new-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-client-2 > 20: type protocol/client > 21: option remote-host hyperspace > 22: option remote-subvolume /mnt/sda10/export3 > 23: option transport-type tcp > 24: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 25: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 26: end-volume > 27: > 28: volume new-replicate-0 > 29: type cluster/replicate > 30: subvolumes new-client-0 new-client-1 new-client-2 > 31: end-volume > 32: > 33: volume new-quota > 34: type features/quota > 35: option timeout 0 > 36: subvolumes new-replicate-0 > 37: end-volume > 38: > 39: volume new-write-behind > 40: type performance/write-behind > 41: subvolumes new-quota > 42: end-volume > 43: > 44: volume new-read-ahead > 45: type performance/read-ahead > 46: subvolumes new-write-behind > 47: end-volume > 48: > 49: volume new-io-cache > 50: type performance/io-cache > 51: subvolumes new-read-ahead > 52: end-volume > 53: > 54: volume new-quick-read > 55: type performance/quick-read > 56: subvolumes new-io-cache > 57: end-volume > 58: > 59: volume new-md-cache > 60: type performance/md-cache > 61: subvolumes new-quick-read > 62: end-volume > 63: > 64: volume new > 65: type debug/io-stats > 66: option latency-measurement off > 67: option count-fop-hits off > 68: subvolumes new-md-cache > 69: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:40:51.692554] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:40:51.692611] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:40:51.692679] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-2: changing port to 24071 (from 0) >[2012-05-20 16:40:55.686508] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:40:55.687381] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:40:55.687459] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:40:55.687720] I [afr-common.c:3628:afr_notify] 0-new-replicate-0: Subvolume 'new-client-1' came back up; going online. >[2012-05-20 16:40:55.688133] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:40:55.688413] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-1: Server lk version = 1 >[2012-05-20 16:40:55.689053] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:40:55.689135] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:40:55.689351] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-2: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:40:55.689569] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-0: Server lk version = 1 >[2012-05-20 16:40:55.689980] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-2: Connected to 127.0.1.1:24071, attached to remote volume '/mnt/sda10/export3'. >[2012-05-20 16:40:55.690035] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:40:55.705236] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 0 >[2012-05-20 16:40:55.705350] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-2: Server lk version = 1 >[2012-05-20 16:40:55.705393] I [fuse-bridge.c:3376:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.16 >[2012-05-20 16:40:55.706220] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 0-new-replicate-0: added root inode >[2012-05-20 16:40:56.849863] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No data available. Path: / (00000000-0000-0000-0000-000000000001). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:56.852613] E [mem-pool.c:503:mem_put] (-->/usr/local/lib/glusterfs/3.3git/xlator/cluster/replicate.so(afr_setxattr_wind_cbk+0xe8) [0x7fa977077d66] (-->/usr/local/lib/glusterfs/3.3git/xlator/cluster/replicate.so(afr_setxattr_unwind+0x1fc) [0x7fa977077c40] (-->/usr/local/lib/glusterfs/3.3git/xlator/features/quota.so(quota_setxattr_cbk+0x1bf) [0x7fa976e44d5b]))) 0-mem-pool: invalid argument >[2012-05-20 16:40:56.973277] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: entry self-heal triggered. path: /, reason: checksums of directory differ >[2012-05-20 16:40:56.976558] E [afr-self-heal-common.c:1002:afr_sh_common_lookup_resp_handler] 0-new-replicate-0: path /glusterfs-3.0.0pre2 on subvolume new-client-2 => -1 (No such file or directory) >[2012-05-20 16:40:56.983505] I [afr-self-heal-common.c:2045:afr_self_heal_completion_cbk] 0-new-replicate-0: background entry self-heal completed on / >[2012-05-20 16:40:56.989043] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No data available. Path: /glusterfs-3.0.0pre2 (ce8e42d4-c908-41c1-962c-37705342d5ec). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:56.990321] E [mem-pool.c:503:mem_put] (-->/usr/local/lib/glusterfs/3.3git/xlator/cluster/replicate.so(afr_setxattr_wind_cbk+0xe8) [0x7fa977077d66] (-->/usr/local/lib/glusterfs/3.3git/xlator/cluster/replicate.so(afr_setxattr_unwind+0x1fc) [0x7fa977077c40] (-->/usr/local/lib/glusterfs/3.3git/xlator/features/quota.so(quota_setxattr_cbk+0x1bf) [0x7fa976e44d5b]))) 0-mem-pool: invalid argument >[2012-05-20 16:40:56.992292] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: entry self-heal triggered. path: /glusterfs-3.0.0pre2, reason: checksums of directory differ >[2012-05-20 16:40:56.992974] E [afr-self-heal-entry.c:2364:afr_sh_post_nonblocking_entry_cbk] 0-new-replicate-0: Non Blocking entrylks failed for /glusterfs-3.0.0pre2. >[2012-05-20 16:40:56.993094] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background entry self-heal failed on /glusterfs-3.0.0pre2 >[2012-05-20 16:40:57.006862] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/THANKS. >[2012-05-20 16:40:57.006963] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/THANKS, reason: lookup detected pending operations >[2012-05-20 16:40:57.008050] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.008120] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/THANKS >[2012-05-20 16:40:57.009337] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/THANKS (3739ed6d-c980-4008-be18-f053d8f91897). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.009436] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 29: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/THANKS => -1 (No such file or directory) >[2012-05-20 16:40:57.011498] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/libglusterfs. >[2012-05-20 16:40:57.011575] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/libglusterfs, reason: lookup detected pending operations >[2012-05-20 16:40:57.012326] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.012391] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/libglusterfs >[2012-05-20 16:40:57.013513] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/libglusterfs (918c8ec3-9b04-43b7-9369-37d7b9faeff7). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.013591] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 31: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/libglusterfs => -1 (No such file or directory) >[2012-05-20 16:40:57.015459] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/glusterfsd. >[2012-05-20 16:40:57.015534] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/glusterfsd, reason: lookup detected pending operations >[2012-05-20 16:40:57.016288] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.016363] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/glusterfsd >[2012-05-20 16:40:57.017478] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/glusterfsd (a4d2ee1c-436e-49be-97c9-64f6016c01a5). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.017551] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 33: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/glusterfsd => -1 (No such file or directory) >[2012-05-20 16:40:57.019588] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/depcomp. >[2012-05-20 16:40:57.019674] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/depcomp, reason: lookup detected pending operations >[2012-05-20 16:40:57.020418] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.020481] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/depcomp >[2012-05-20 16:40:57.021556] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/depcomp (becca04d-9ef1-4991-854f-0ef94dff5028). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.021660] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 35: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/depcomp => -1 (No such file or directory) >[2012-05-20 16:40:57.023666] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/confdefs.h. >[2012-05-20 16:40:57.023758] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/confdefs.h, reason: lookup detected pending operations >[2012-05-20 16:40:57.024639] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.024726] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/confdefs.h >[2012-05-20 16:40:57.026113] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/confdefs.h (51266156-613d-427f-a220-31ef9e739ad4). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.026194] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 37: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/confdefs.h => -1 (No such file or directory) >[2012-05-20 16:40:57.028022] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/ltmain.sh. >[2012-05-20 16:40:57.028104] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/ltmain.sh, reason: lookup detected pending operations >[2012-05-20 16:40:57.028938] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.029004] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/ltmain.sh >[2012-05-20 16:40:57.030057] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/ltmain.sh (a58bb72a-b7b1-4fcd-bbb2-aba093111057). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.030138] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 39: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/ltmain.sh => -1 (No such file or directory) >[2012-05-20 16:40:57.031858] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/argp-standalone. >[2012-05-20 16:40:57.031973] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/argp-standalone, reason: lookup detected pending operations >[2012-05-20 16:40:57.032695] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.032759] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/argp-standalone >[2012-05-20 16:40:57.033953] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/argp-standalone (983b7319-621a-42fb-87d5-0b7e7c721ab4). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.034061] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 41: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/argp-standalone => -1 (No such file or directory) >[2012-05-20 16:40:57.036082] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/transport. >[2012-05-20 16:40:57.036154] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/transport, reason: lookup detected pending operations >[2012-05-20 16:40:57.036957] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.037022] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/transport >[2012-05-20 16:40:57.038109] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/transport (d23a45c7-3da5-4d75-b52a-15518676c6dd). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.038181] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 43: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/transport => -1 (No such file or directory) >[2012-05-20 16:40:57.039950] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/extras. >[2012-05-20 16:40:57.040021] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/extras, reason: lookup detected pending operations >[2012-05-20 16:40:57.040765] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.040828] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/extras >[2012-05-20 16:40:57.041839] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/extras (44aaccf6-497c-43da-a386-c3e36c26dfbb). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.041909] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 45: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/extras => -1 (No such file or directory) >[2012-05-20 16:40:57.043800] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/Makefile.am. >[2012-05-20 16:40:57.043883] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/Makefile.am, reason: lookup detected pending operations >[2012-05-20 16:40:57.044642] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.044705] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/Makefile.am >[2012-05-20 16:40:57.046069] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/Makefile.am (ae066f41-87ce-4e59-b5dd-a4407c53f938). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.046208] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 47: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/Makefile.am => -1 (No such file or directory) >[2012-05-20 16:40:57.048240] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/contrib. >[2012-05-20 16:40:57.048314] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/contrib, reason: lookup detected pending operations >[2012-05-20 16:40:57.049093] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.049173] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/contrib >[2012-05-20 16:40:57.050390] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/contrib (557a46ba-0f8f-4425-bedf-3549946f6c95). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.050469] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 49: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/contrib => -1 (No such file or directory) >[2012-05-20 16:40:57.052313] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/compile. >[2012-05-20 16:40:57.052393] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/compile, reason: lookup detected pending operations >[2012-05-20 16:40:57.053313] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.053377] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/compile >[2012-05-20 16:40:57.054557] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/compile (ac4d960d-9cea-4224-b17d-362c1106787b). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.054628] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 51: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/compile => -1 (No such file or directory) >[2012-05-20 16:40:57.056885] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/xlators. >[2012-05-20 16:40:57.056956] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/xlators, reason: lookup detected pending operations >[2012-05-20 16:40:57.057723] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.057847] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/xlators >[2012-05-20 16:40:57.059027] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/xlators (5cb58bce-bd11-4c20-9c47-b19a4bbc2e29). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.059123] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 53: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/xlators => -1 (No such file or directory) >[2012-05-20 16:40:57.061257] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/libtoolT. >[2012-05-20 16:40:57.061339] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/libtoolT, reason: lookup detected pending operations >[2012-05-20 16:40:57.062079] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.062142] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/libtoolT >[2012-05-20 16:40:57.063406] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/libtoolT (8c4449e0-c141-4233-bfc8-c981bf990f2b). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.063482] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 55: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/libtoolT => -1 (No such file or directory) >[2012-05-20 16:40:57.065498] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/auth. >[2012-05-20 16:40:57.065582] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/auth, reason: lookup detected pending operations >[2012-05-20 16:40:57.066361] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.066442] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/auth >[2012-05-20 16:40:57.067788] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/auth (187fbbc8-fe31-4ddf-8d88-ffc35a2d6226). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.067862] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 57: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/auth => -1 (No such file or directory) >[2012-05-20 16:40:57.069740] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/config.h. >[2012-05-20 16:40:57.069849] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/config.h, reason: lookup detected pending operations >[2012-05-20 16:40:57.070618] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.070679] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/config.h >[2012-05-20 16:40:57.071779] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/config.h (e401bcf3-0b43-467e-a71b-a5155b57e9b8). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.071853] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 59: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/config.h => -1 (No such file or directory) >[2012-05-20 16:40:57.073991] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/stamp-h1. >[2012-05-20 16:40:57.074078] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/stamp-h1, reason: lookup detected pending operations >[2012-05-20 16:40:57.074899] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.074961] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/stamp-h1 >[2012-05-20 16:40:57.076212] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/stamp-h1 (87a7a0f7-e380-4a7f-8adc-a3f728f15c5d). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.076305] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 61: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/stamp-h1 => -1 (No such file or directory) >[2012-05-20 16:40:57.078326] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/missing. >[2012-05-20 16:40:57.078424] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/missing, reason: lookup detected pending operations >[2012-05-20 16:40:57.079295] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.079364] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/missing >[2012-05-20 16:40:57.080396] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/missing (6672c375-f22c-4678-8783-6a74baefccb8). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.080473] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 63: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/missing => -1 (No such file or directory) >[2012-05-20 16:40:57.082177] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/config.log. >[2012-05-20 16:40:57.082260] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/config.log, reason: lookup detected pending operations >[2012-05-20 16:40:57.083412] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.083505] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/config.log >[2012-05-20 16:40:57.084793] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/config.log (3757a823-c00e-42f4-a49b-80ff439e1f86). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.084899] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 65: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/config.log => -1 (No such file or directory) >[2012-05-20 16:40:57.087708] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/config.guess. >[2012-05-20 16:40:57.087801] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/config.guess, reason: lookup detected pending operations >[2012-05-20 16:40:57.088606] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.088667] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/config.guess >[2012-05-20 16:40:57.089890] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/config.guess (1841de0d-e028-47e4-b164-504884328b89). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.089978] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 67: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/config.guess => -1 (No such file or directory) >[2012-05-20 16:40:57.091887] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/libglusterfsclient. >[2012-05-20 16:40:57.091955] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/libglusterfsclient, reason: lookup detected pending operations >[2012-05-20 16:40:57.092687] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.092750] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/libglusterfsclient >[2012-05-20 16:40:57.093859] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/libglusterfsclient (e4a09bda-232c-4c33-9104-67ff8b843631). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.093930] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 69: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/libglusterfsclient => -1 (No such file or directory) >[2012-05-20 16:40:57.095994] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/libtool. >[2012-05-20 16:40:57.096083] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/libtool, reason: lookup detected pending operations >[2012-05-20 16:40:57.097014] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.097106] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/libtool >[2012-05-20 16:40:57.098192] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/libtool (462a613e-5d99-412b-8c3e-c856fcb8e04c). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.098267] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 71: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/libtool => -1 (No such file or directory) >[2012-05-20 16:40:57.100222] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/README. >[2012-05-20 16:40:57.100304] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/README, reason: lookup detected pending operations >[2012-05-20 16:40:57.101069] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.101130] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/README >[2012-05-20 16:40:57.102196] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/README (6dcc8aae-5244-4549-8e64-4b0e9388a66c). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.102268] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 73: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/README => -1 (No such file or directory) >[2012-05-20 16:40:57.104585] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/Makefile. >[2012-05-20 16:40:57.104670] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/Makefile, reason: lookup detected pending operations >[2012-05-20 16:40:57.105375] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.105435] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/Makefile >[2012-05-20 16:40:57.106722] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/Makefile (25d58663-febc-4604-93b9-705775b03a9c). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.106817] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 75: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/Makefile => -1 (No such file or directory) >[2012-05-20 16:40:57.109941] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/COPYING. >[2012-05-20 16:40:57.110038] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/COPYING, reason: lookup detected pending operations >[2012-05-20 16:40:57.110770] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.110831] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/COPYING >[2012-05-20 16:40:57.111957] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/COPYING (aeb224f3-bcc7-4ab1-b25c-423ee0d9e6c2). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.112132] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 77: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/COPYING => -1 (No such file or directory) >[2012-05-20 16:40:57.114105] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/ChangeLog. >[2012-05-20 16:40:57.114228] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/ChangeLog, reason: lookup detected pending operations >[2012-05-20 16:40:57.115015] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.115091] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/ChangeLog >[2012-05-20 16:40:57.116267] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/ChangeLog (25e62348-63b7-4066-b28d-e9152d98031b). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.116368] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 79: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/ChangeLog => -1 (No such file or directory) >[2012-05-20 16:40:57.118271] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/doc. >[2012-05-20 16:40:57.118341] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/doc, reason: lookup detected pending operations >[2012-05-20 16:40:57.119221] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.119302] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/doc >[2012-05-20 16:40:57.120550] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/doc (bd8a6e0d-0957-4e1a-8b5f-f1e434227b0a). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.120654] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 81: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/doc => -1 (No such file or directory) >[2012-05-20 16:40:57.123256] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/glusterfs.spec.in. >[2012-05-20 16:40:57.123338] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/glusterfs.spec.in, reason: lookup detected pending operations >[2012-05-20 16:40:57.124180] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.124263] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/glusterfs.spec.in >[2012-05-20 16:40:57.125382] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/glusterfs.spec.in (1597fbec-ec8c-4623-856e-5b8160762a98). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.125469] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 83: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/glusterfs.spec.in => -1 (No such file or directory) >[2012-05-20 16:40:57.127510] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/NEWS. >[2012-05-20 16:40:57.127596] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/NEWS, reason: lookup detected pending operations >[2012-05-20 16:40:57.128399] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.128468] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/NEWS >[2012-05-20 16:40:57.129483] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/NEWS (a249bfb0-8452-46c4-b55d-8fdbb1a7f269). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.129557] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 85: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/NEWS => -1 (No such file or directory) >[2012-05-20 16:40:57.131340] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/configure. >[2012-05-20 16:40:57.131420] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/configure, reason: lookup detected pending operations >[2012-05-20 16:40:57.132213] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.132278] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/configure >[2012-05-20 16:40:57.133508] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/configure (43a474c7-1ae4-4f2f-a0fc-0cae110c5294). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.133590] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 87: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/configure => -1 (No such file or directory) >[2012-05-20 16:40:57.135416] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/confpmiroZ. >[2012-05-20 16:40:57.135508] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/confpmiroZ, reason: lookup detected pending operations >[2012-05-20 16:40:57.136225] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.136289] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/confpmiroZ >[2012-05-20 16:40:57.137507] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/confpmiroZ (db10050a-424f-446d-abcd-879e8a926268). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.137629] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 89: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/confpmiroZ => -1 (No such file or directory) >[2012-05-20 16:40:57.139892] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/scheduler. >[2012-05-20 16:40:57.139971] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/scheduler, reason: lookup detected pending operations >[2012-05-20 16:40:57.140686] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.140791] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/scheduler >[2012-05-20 16:40:57.142125] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/scheduler (0cc9ce48-97f7-4d9c-abd0-657d3852e2e6). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.142204] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 91: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/scheduler => -1 (No such file or directory) >[2012-05-20 16:40:57.144414] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/AUTHORS. >[2012-05-20 16:40:57.144504] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/AUTHORS, reason: lookup detected pending operations >[2012-05-20 16:40:57.145534] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.145608] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/AUTHORS >[2012-05-20 16:40:57.146945] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/AUTHORS (f5ec7309-9f02-4873-9770-533a5897f0bd). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.147045] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 93: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/AUTHORS => -1 (No such file or directory) >[2012-05-20 16:40:57.149366] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/INSTALL. >[2012-05-20 16:40:57.149451] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/INSTALL, reason: lookup detected pending operations >[2012-05-20 16:40:57.150347] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.150414] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/INSTALL >[2012-05-20 16:40:57.151762] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/INSTALL (403a79b0-00e0-4d18-a998-b12172846254). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.151839] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 95: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/INSTALL => -1 (No such file or directory) >[2012-05-20 16:40:57.154177] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/glusterfs.spec. >[2012-05-20 16:40:57.154285] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/glusterfs.spec, reason: lookup detected pending operations >[2012-05-20 16:40:57.155222] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.155292] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/glusterfs.spec >[2012-05-20 16:40:57.156961] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/glusterfs.spec (d2c72eb8-0a08-4fa6-9aa7-3939d997e25a). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.157086] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 97: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/glusterfs.spec => -1 (No such file or directory) >[2012-05-20 16:40:57.159252] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/autogen.sh. >[2012-05-20 16:40:57.159335] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/autogen.sh, reason: lookup detected pending operations >[2012-05-20 16:40:57.160221] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.160289] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/autogen.sh >[2012-05-20 16:40:57.161493] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/autogen.sh (8da67bd2-929b-420f-acc5-a43cfcdacf09). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.161603] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 99: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/autogen.sh => -1 (No such file or directory) >[2012-05-20 16:40:57.163530] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/booster. >[2012-05-20 16:40:57.163605] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/booster, reason: lookup detected pending operations >[2012-05-20 16:40:57.164362] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.164424] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/booster >[2012-05-20 16:40:57.165753] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/booster (89c2678f-f9e6-4e91-acff-8354cc5bfcdf). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.165830] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 101: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/booster => -1 (No such file or directory) >[2012-05-20 16:40:57.168279] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/config.sub. >[2012-05-20 16:40:57.168375] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/config.sub, reason: lookup detected pending operations >[2012-05-20 16:40:57.169164] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.169228] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/config.sub >[2012-05-20 16:40:57.170434] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/config.sub (238ce087-c24b-4f30-8b48-7e0a2644e31f). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.170554] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 103: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/config.sub => -1 (No such file or directory) >[2012-05-20 16:40:57.172582] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/config.h.in. >[2012-05-20 16:40:57.172663] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/config.h.in, reason: lookup detected pending operations >[2012-05-20 16:40:57.173437] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.173499] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/config.h.in >[2012-05-20 16:40:57.174838] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/config.h.in (75908253-8eac-4c8d-a04f-b6f8bb432bf3). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.174918] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 105: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/config.h.in => -1 (No such file or directory) >[2012-05-20 16:40:57.177212] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/aclocal.m4. >[2012-05-20 16:40:57.177305] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/aclocal.m4, reason: lookup detected pending operations >[2012-05-20 16:40:57.178171] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.178241] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/aclocal.m4 >[2012-05-20 16:40:57.179518] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/aclocal.m4 (6354c1eb-e1ef-4529-a4bb-16d767815efe). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.179598] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 107: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/aclocal.m4 => -1 (No such file or directory) >[2012-05-20 16:40:57.181802] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/configure.ac. >[2012-05-20 16:40:57.181914] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/configure.ac, reason: lookup detected pending operations >[2012-05-20 16:40:57.183028] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.183102] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/configure.ac >[2012-05-20 16:40:57.184457] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/configure.ac (38a04c8e-7710-4903-970e-56d0e87717f2). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.184535] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 109: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/configure.ac => -1 (No such file or directory) >[2012-05-20 16:40:57.186832] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/Makefile.in. >[2012-05-20 16:40:57.186947] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/Makefile.in, reason: lookup detected pending operations >[2012-05-20 16:40:57.187958] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.188032] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/Makefile.in >[2012-05-20 16:40:57.189628] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/Makefile.in (9ae1691a-f035-4ed1-a9a6-9f5a078c5b86). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.189732] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 111: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/Makefile.in => -1 (No such file or directory) >[2012-05-20 16:40:57.191855] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/config.status. >[2012-05-20 16:40:57.191962] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/config.status, reason: lookup detected pending operations >[2012-05-20 16:40:57.193029] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.193116] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/config.status >[2012-05-20 16:40:57.194437] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/config.status (84572912-0980-4eb3-99bd-ed85cc87a93b). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.194525] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 113: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/config.status => -1 (No such file or directory) >[2012-05-20 16:40:57.197109] I [afr-common.c:1215:afr_detect_self_heal_by_lookup_status] 0-new-replicate-0: entries are missing in lookup of /glusterfs-3.0.0pre2/install-sh. >[2012-05-20 16:40:57.197200] I [afr-common.c:1341:afr_launch_self_heal] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal triggered. path: /glusterfs-3.0.0pre2/install-sh, reason: lookup detected pending operations >[2012-05-20 16:40:57.197972] I [afr-self-heal-common.c:1827:afr_sh_post_nb_entrylk_conflicting_sh_cbk] 0-new-replicate-0: Non blocking entrylks failed. >[2012-05-20 16:40:57.198040] E [afr-self-heal-common.c:2042:afr_self_heal_completion_cbk] 0-new-replicate-0: background meta-data data entry missing-entry gfid self-heal failed on /glusterfs-3.0.0pre2/install-sh >[2012-05-20 16:40:57.199139] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-new-client-2: remote operation failed: No such file or directory. Path: /glusterfs-3.0.0pre2/install-sh (0444359e-aef7-42fb-9ea3-7006f2573e18). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 16:40:57.199214] W [fuse-bridge.c:2811:fuse_xattr_cbk] 0-glusterfs-fuse: 115: GETXATTR(trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime) /glusterfs-3.0.0pre2/install-sh => -1 (No such file or directory) >[2012-05-20 16:41:30.329499] I [fuse-bridge.c:4037:fuse_thread_proc] 0-fuse: unmounting /tmp/gsyncd-aux-mount-PGpmVq >[2012-05-20 16:41:30.329644] W [glusterfsd.c:816:cleanup_and_exit] (-->/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fa97b19904d] (-->/lib/x86_64-linux-gnu/libpthread.so.0(+0x6d8c) [0x7fa97b7dcd8c] (-->/usr/local/sbin/glusterfs(glusterfs_sigwaiter+0xea) [0x407f1f]))) 0-: received signum (15), shutting down >[2012-05-20 16:41:42.454898] I [glusterfsd.c:1651:main] 0-/usr/local/sbin/glusterfs: Started running /usr/local/sbin/glusterfs version 3.3git >[2012-05-20 16:41:42.465581] I [io-cache.c:1549:check_cache_size_ok] 0-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:41:42.465805] I [io-cache.c:1549:check_cache_size_ok] 0-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:41:42.466072] I [quota.c:3085:quota_parse_limits] 0-new-quota: no "limit-set" option provided >[2012-05-20 16:41:42.476415] I [client.c:2142:notify] 0-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:41:42.477863] I [client.c:2142:notify] 0-new-client-1: parent translators are ready, attempting connect on transport >[2012-05-20 16:41:42.478981] I [client.c:2142:notify] 0-new-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-client-2 > 20: type protocol/client > 21: option remote-host hyperspace > 22: option remote-subvolume /mnt/sda10/export3 > 23: option transport-type tcp > 24: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 25: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 26: end-volume > 27: > 28: volume new-replicate-0 > 29: type cluster/replicate > 30: subvolumes new-client-0 new-client-1 new-client-2 > 31: end-volume > 32: > 33: volume new-quota > 34: type features/quota > 35: option timeout 0 > 36: subvolumes new-replicate-0 > 37: end-volume > 38: > 39: volume new-write-behind > 40: type performance/write-behind > 41: subvolumes new-quota > 42: end-volume > 43: > 44: volume new-read-ahead > 45: type performance/read-ahead > 46: subvolumes new-write-behind > 47: end-volume > 48: > 49: volume new-io-cache > 50: type performance/io-cache > 51: subvolumes new-read-ahead > 52: end-volume > 53: > 54: volume new-quick-read > 55: type performance/quick-read > 56: subvolumes new-io-cache > 57: end-volume > 58: > 59: volume new-md-cache > 60: type performance/md-cache > 61: subvolumes new-quick-read > 62: end-volume > 63: > 64: volume new > 65: type debug/io-stats > 66: option latency-measurement off > 67: option count-fop-hits off > 68: subvolumes new-md-cache > 69: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:41:42.481626] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:41:42.481811] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:41:42.482032] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 0-new-client-2: changing port to 24071 (from 0) >[2012-05-20 16:41:46.460556] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:41:46.460792] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:41:46.460897] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:41:46.460913] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:41:46.460964] I [afr-common.c:3628:afr_notify] 0-new-replicate-0: Subvolume 'new-client-1' came back up; going online. >[2012-05-20 16:41:46.461004] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-1: Server lk version = 1 >[2012-05-20 16:41:46.461049] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:41:46.461061] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:41:46.461297] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-0: Server lk version = 1 >[2012-05-20 16:41:46.461428] I [client-handshake.c:1628:select_server_supported_programs] 0-new-client-2: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:41:46.462005] I [client-handshake.c:1425:client_setvolume_cbk] 0-new-client-2: Connected to 127.0.1.1:24071, attached to remote volume '/mnt/sda10/export3'. >[2012-05-20 16:41:46.462050] I [client-handshake.c:1437:client_setvolume_cbk] 0-new-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:41:46.475758] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 0 >[2012-05-20 16:41:46.476267] I [client-handshake.c:453:client_set_lk_version_cbk] 0-new-client-2: Server lk version = 1 >[2012-05-20 16:41:46.476299] I [fuse-bridge.c:3376:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.16 >[2012-05-20 16:41:46.477056] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 0-new-replicate-0: added root inode >[2012-05-20 16:42:25.403766] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2012-05-20 16:42:25.406735] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2012-05-20 16:42:25.410407] I [io-cache.c:1549:check_cache_size_ok] 1-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:42:25.410502] I [io-cache.c:1549:check_cache_size_ok] 1-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:42:25.410624] I [quota.c:3085:quota_parse_limits] 1-new-quota: no "limit-set" option provided >[2012-05-20 16:42:25.415713] I [client.c:2142:notify] 1-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:42:25.416228] I [client.c:2142:notify] 1-new-client-1: parent translators are ready, attempting connect on transport >[2012-05-20 16:42:25.416560] I [client.c:2142:notify] 1-new-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-client-2 > 20: type protocol/client > 21: option remote-host hyperspace > 22: option remote-subvolume /mnt/sda10/export3 > 23: option transport-type tcp > 24: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 25: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 26: end-volume > 27: > 28: volume new-replicate-0 > 29: type cluster/replicate > 30: subvolumes new-client-0 new-client-1 new-client-2 > 31: end-volume > 32: > 33: volume new-quota > 34: type features/quota > 35: option timeout 0 > 36: subvolumes new-replicate-0 > 37: end-volume > 38: > 39: volume new-write-behind > 40: type performance/write-behind > 41: subvolumes new-quota > 42: end-volume > 43: > 44: volume new-read-ahead > 45: type performance/read-ahead > 46: subvolumes new-write-behind > 47: end-volume > 48: > 49: volume new-io-cache > 50: type performance/io-cache > 51: subvolumes new-read-ahead > 52: end-volume > 53: > 54: volume new-quick-read > 55: type performance/quick-read > 56: subvolumes new-io-cache > 57: end-volume > 58: > 59: volume new > 60: type debug/io-stats > 61: option latency-measurement off > 62: option count-fop-hits off > 63: subvolumes new-quick-read > 64: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:42:25.416986] I [glusterfsd-mgmt.c:1565:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing >[2012-05-20 16:42:25.417281] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 1-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:42:25.417341] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 1-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:42:25.417491] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 1-new-client-2: changing port to 24071 (from 0) >[2012-05-20 16:42:28.468063] I [client-handshake.c:1628:select_server_supported_programs] 1-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:42:28.468790] I [client-handshake.c:1425:client_setvolume_cbk] 1-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:42:28.468865] I [client-handshake.c:1437:client_setvolume_cbk] 1-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:42:28.469059] I [afr-common.c:3628:afr_notify] 1-new-replicate-0: Subvolume 'new-client-1' came back up; going online. >[2012-05-20 16:42:28.469305] I [client-handshake.c:453:client_set_lk_version_cbk] 1-new-client-1: Server lk version = 1 >[2012-05-20 16:42:28.469475] I [client-handshake.c:1628:select_server_supported_programs] 1-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:42:28.470061] I [client-handshake.c:1425:client_setvolume_cbk] 1-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:42:28.470131] I [client-handshake.c:1437:client_setvolume_cbk] 1-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:42:28.470395] I [client-handshake.c:453:client_set_lk_version_cbk] 1-new-client-0: Server lk version = 1 >[2012-05-20 16:42:28.471433] I [client-handshake.c:1628:select_server_supported_programs] 1-new-client-2: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:42:28.472135] I [client-handshake.c:1425:client_setvolume_cbk] 1-new-client-2: Connected to 127.0.1.1:24071, attached to remote volume '/mnt/sda10/export3'. >[2012-05-20 16:42:28.472213] I [client-handshake.c:1437:client_setvolume_cbk] 1-new-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:42:28.485887] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 1 >[2012-05-20 16:42:28.485990] I [client-handshake.c:453:client_set_lk_version_cbk] 1-new-client-2: Server lk version = 1 >[2012-05-20 16:42:29.021078] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 1-new-replicate-0: added root inode >[2012-05-20 16:42:29.021781] I [client.c:2151:notify] 0-new-client-0: current graph is no longer active, destroying rpc_client >[2012-05-20 16:42:29.021913] I [client.c:2151:notify] 0-new-client-1: current graph is no longer active, destroying rpc_client >[2012-05-20 16:42:29.021978] I [client.c:2151:notify] 0-new-client-2: current graph is no longer active, destroying rpc_client >[2012-05-20 16:42:29.022028] I [client.c:2090:client_rpc_notify] 0-new-client-0: disconnected >[2012-05-20 16:42:29.022225] I [client.c:2090:client_rpc_notify] 0-new-client-1: disconnected >[2012-05-20 16:42:29.022325] I [client.c:2090:client_rpc_notify] 0-new-client-2: disconnected >[2012-05-20 16:42:29.022381] E [afr-common.c:3665:afr_notify] 0-new-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up. >[2012-05-20 16:42:32.074623] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2012-05-20 16:42:32.076363] I [glusterfsd-mgmt.c:64:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2012-05-20 16:42:32.080727] I [io-cache.c:1549:check_cache_size_ok] 2-new-quick-read: Max cache size is 4066508800 >[2012-05-20 16:42:32.080997] I [io-cache.c:1549:check_cache_size_ok] 2-new-io-cache: Max cache size is 4066508800 >[2012-05-20 16:42:32.081355] I [quota.c:3085:quota_parse_limits] 2-new-quota: no "limit-set" option provided >[2012-05-20 16:42:32.087556] I [client.c:2142:notify] 2-new-client-0: parent translators are ready, attempting connect on transport >[2012-05-20 16:42:32.088111] I [client.c:2142:notify] 2-new-client-1: parent translators are ready, attempting connect on transport >[2012-05-20 16:42:32.088645] I [client.c:2142:notify] 2-new-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume new-client-0 > 2: type protocol/client > 3: option remote-host hyperspace > 4: option remote-subvolume /mnt/sda7/export3 > 5: option transport-type tcp > 6: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 7: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 8: end-volume > 9: > 10: volume new-client-1 > 11: type protocol/client > 12: option remote-host hyperspace > 13: option remote-subvolume /mnt/sda8/export3 > 14: option transport-type tcp > 15: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 16: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 17: end-volume > 18: > 19: volume new-client-2 > 20: type protocol/client > 21: option remote-host hyperspace > 22: option remote-subvolume /mnt/sda10/export3 > 23: option transport-type tcp > 24: option username 96fe88c4-f83b-459a-81a2-a3264d710686 > 25: option password c4097230-aa7a-4869-9698-58f1e2222a5e > 26: end-volume > 27: > 28: volume new-replicate-0 > 29: type cluster/replicate > 30: subvolumes new-client-0 new-client-1 new-client-2 > 31: end-volume > 32: > 33: volume new-quota > 34: type features/quota > 35: option timeout 0 > 36: subvolumes new-replicate-0 > 37: end-volume > 38: > 39: volume new-write-behind > 40: type performance/write-behind > 41: subvolumes new-quota > 42: end-volume > 43: > 44: volume new-read-ahead > 45: type performance/read-ahead > 46: subvolumes new-write-behind > 47: end-volume > 48: > 49: volume new-io-cache > 50: type performance/io-cache > 51: subvolumes new-read-ahead > 52: end-volume > 53: > 54: volume new-quick-read > 55: type performance/quick-read > 56: subvolumes new-io-cache > 57: end-volume > 58: > 59: volume new-md-cache > 60: type performance/md-cache > 61: subvolumes new-quick-read > 62: end-volume > 63: > 64: volume new > 65: type debug/io-stats > 66: option latency-measurement off > 67: option count-fop-hits off > 68: subvolumes new-md-cache > 69: end-volume > >+------------------------------------------------------------------------------+ >[2012-05-20 16:42:32.089589] I [glusterfsd-mgmt.c:1565:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing >[2012-05-20 16:42:32.089975] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 2-new-client-1: changing port to 24070 (from 0) >[2012-05-20 16:42:32.090033] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 2-new-client-0: changing port to 24069 (from 0) >[2012-05-20 16:42:32.090075] I [rpc-clnt.c:1660:rpc_clnt_reconfig] 2-new-client-2: changing port to 24071 (from 0) >[2012-05-20 16:42:35.473612] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-1: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:42:35.474432] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-1: Connected to 127.0.1.1:24070, attached to remote volume '/mnt/sda8/export3'. >[2012-05-20 16:42:35.474526] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:42:35.474701] I [afr-common.c:3628:afr_notify] 2-new-replicate-0: Subvolume 'new-client-1' came back up; going online. >[2012-05-20 16:42:35.474790] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-1: Server lk version = 1 >[2012-05-20 16:42:35.475086] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:42:35.475635] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-0: Connected to 127.0.1.1:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 16:42:35.475689] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:42:35.476016] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-0: Server lk version = 1 >[2012-05-20 16:42:35.476136] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-2: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 16:42:35.476767] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-2: Connected to 127.0.1.1:24071, attached to remote volume '/mnt/sda10/export3'. >[2012-05-20 16:42:35.476843] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 16:42:35.493871] I [fuse-bridge.c:4138:fuse_graph_setup] 0-fuse: switched to graph 2 >[2012-05-20 16:42:35.493961] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-2: Server lk version = 1 >[2012-05-20 16:42:36.065205] I [afr-common.c:1965:afr_set_root_inode_on_first_lookup] 2-new-replicate-0: added root inode >[2012-05-20 16:42:36.066469] I [client.c:2151:notify] 1-new-client-0: current graph is no longer active, destroying rpc_client >[2012-05-20 16:42:36.095542] I [client.c:2151:notify] 1-new-client-1: current graph is no longer active, destroying rpc_client >[2012-05-20 16:42:36.095703] I [client.c:2090:client_rpc_notify] 1-new-client-0: disconnected >[2012-05-20 16:42:36.095846] I [client.c:2090:client_rpc_notify] 1-new-client-1: disconnected >[2012-05-20 16:42:36.095709] I [client.c:2151:notify] 1-new-client-2: current graph is no longer active, destroying rpc_client >[2012-05-20 16:42:36.096729] I [client.c:2090:client_rpc_notify] 1-new-client-2: disconnected >[2012-05-20 16:42:36.096782] E [afr-common.c:3665:afr_notify] 1-new-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up. >[2012-05-20 17:11:12.661560] C [client-handshake.c:126:rpc_client_ping_timer_expired] 2-new-client-0: server 127.0.1.1:24069 has not responded in the last 42 seconds, disconnecting. >[2012-05-20 17:11:12.662565] E [rpc-clnt.c:373:saved_frames_unwind] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0x124) [0x7f1ad663fe78] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x155) [0x7f1ad663f3ea] (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0x1f) [0x7f1ad663ee6f]))) 2-new-client-0: forced unwinding frame type(GlusterFS 3.1) op(LOOKUP(27)) called at 2012-05-20 17:09:54.553216 (xid=0x4464x) >[2012-05-20 17:11:12.662679] W [client3_1-fops.c:2630:client3_1_lookup_cbk] 2-new-client-0: remote operation failed: Transport endpoint is not connected. Path: / (00000000-0000-0000-0000-000000000001) >[2012-05-20 17:11:12.663074] E [rpc-clnt.c:373:saved_frames_unwind] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_notify+0x124) [0x7f1ad663fe78] (-->/usr/local/lib/libgfrpc.so.0(rpc_clnt_connection_cleanup+0x155) [0x7f1ad663f3ea] (-->/usr/local/lib/libgfrpc.so.0(saved_frames_destroy+0x1f) [0x7f1ad663ee6f]))) 2-new-client-0: forced unwinding frame type(GlusterFS Handshake) op(PING(3)) called at 2012-05-20 17:10:30.657008 (xid=0x4465x) >[2012-05-20 17:11:12.663171] W [client-handshake.c:275:client_ping_cbk] 2-new-client-0: timer must have expired >[2012-05-20 17:11:12.663221] I [client.c:2090:client_rpc_notify] 2-new-client-0: disconnected >[2012-05-20 17:11:12.663836] I [socket.c:2315:socket_submit_request] 2-new-client-0: not connected (priv->connected = 0) >[2012-05-20 17:11:12.663914] W [rpc-clnt.c:1498:rpc_clnt_submit] 2-new-client-0: failed to submit rpc-request (XID: 0x4466x Program: GlusterFS 3.1, ProgVers: 330, Proc: 18) to rpc-transport (new-client-0) >[2012-05-20 17:11:12.663967] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 2-new-client-0: remote operation failed: Transport endpoint is not connected. Path: / (00000000-0000-0000-0000-000000000001). Key: trusted.glusterfs.volume-mark >[2012-05-20 17:11:39.187106] I [client-handshake.c:1628:select_server_supported_programs] 2-new-client-0: Using Program GlusterFS 3.3git, Num (1298437), Version (330) >[2012-05-20 17:11:39.187388] E [client3_1-fops.c:1032:client3_1_getxattr_cbk] 2-new-client-0: XDR decoding failed >[2012-05-20 17:11:39.187484] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 2-new-client-0: remote operation failed: Invalid argument. Path: / (00000000-0000-0000-0000-000000000001). Key: trusted.glusterfs.75160aa1-3053-44cb-973f-7fb0d63a93d3.xtime >[2012-05-20 17:11:39.187882] I [client-handshake.c:1425:client_setvolume_cbk] 2-new-client-0: Connected to 192.168.1.88:24069, attached to remote volume '/mnt/sda7/export3'. >[2012-05-20 17:11:39.187959] I [client-handshake.c:1437:client_setvolume_cbk] 2-new-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-05-20 17:11:39.188432] I [client-handshake.c:453:client_set_lk_version_cbk] 2-new-client-0: Server lk version = 1 >[2012-05-20 17:11:39.193537] E [mem-pool.c:503:mem_put] (-->/usr/local/lib/glusterfs/3.3git/xlator/cluster/replicate.so(afr_setxattr_wind_cbk+0xe8) [0x7f1ad1a96d66] (-->/usr/local/lib/glusterfs/3.3git/xlator/cluster/replicate.so(afr_setxattr_unwind+0x1fc) [0x7f1ad1a96c40] (-->/usr/local/lib/glusterfs/3.3git/xlator/features/quota.so(quota_setxattr_cbk+0x1bf) [0x7f1ad1863d5b]))) 2-mem-pool: invalid argument
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 823255
:
585625
|
585626
|
585629
| 585640