Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 617485 Details for
Bug 860599
DHT - Hash layout for Directory is getting changed when sub-volume is down and user performes rebalance operation, as a result at same level multiple files can be created having same name
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
rebalance log
test-rebalance.log (text/x-log), 92.71 KB, created by
Rachana Patel
on 2012-09-26 09:43:35 UTC
(
hide
)
Description:
rebalance log
Filename:
MIME Type:
Creator:
Rachana Patel
Created:
2012-09-26 09:43:35 UTC
Size:
92.71 KB
patch
obsolete
>[2012-09-25 14:28:07.117758] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-25 14:28:12.118379] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-25 14:28:12.118447] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-25 14:28:12.121072] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-25 14:28:12.121095] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-25 14:28:12.121108] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-25 14:28:12.121119] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-25 14:28:12.121130] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-25 14:28:12.121431] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-25 14:28:12.121495] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-25 14:28:12.123513] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-25 14:28:12.126042] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-25 14:28:12.128127] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-25 14:28:12.130841] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-25 14:28:16.129569] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-25 14:28:16.129805] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-25 14:28:16.129825] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-25 14:28:16.130078] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-25 14:28:17.211076] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-25 14:28:17.680335] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-25 14:28:21.134317] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-25 14:28:21.134743] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-25 14:28:21.134764] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-25 14:28:21.135022] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-25 14:28:21.137308] E [socket.c:1715:socket_connect_finish] 0-test-client-2: connection to 10.70.35.86:24007 failed (Connection refused) >[2012-09-25 14:28:21.137337] W [dht-common.c:4696:dht_notify] 0-test-dht: Received CHILD_DOWN. Exiting >[2012-09-25 14:28:21.141924] I [dht-layout.c:593:dht_layout_normalize] 0-test-dht: found anomalies in /. holes=1 overlaps=0 >[2012-09-25 14:28:21.141970] W [dht-selfheal.c:875:dht_selfheal_directory] 0-test-dht: 1 subvolumes down -- not fixing >[2012-09-25 14:28:21.142008] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-25 14:28:21.142857] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-25 14:28:21.154255] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-25 14:28:21.154290] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 15, failures: 0 >[2012-09-25 14:28:21.154619] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-25 15:32:18.528174] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-25 15:32:23.652825] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-25 15:32:23.652914] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-25 15:32:23.655059] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-25 15:32:23.655073] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-25 15:32:23.655082] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-25 15:32:23.655090] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-25 15:32:23.655098] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-25 15:32:23.655177] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-25 15:32:23.655219] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-25 15:32:23.656976] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-25 15:32:23.659141] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-25 15:32:23.661115] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-25 15:32:23.663487] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-25 15:32:23.663740] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-25 15:32:23.663852] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-25 15:32:27.541221] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-25 15:32:27.541504] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-25 15:32:27.541520] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-25 15:32:27.541705] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-25 15:32:27.545365] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-25 15:32:27.545531] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-25 15:32:27.545546] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-25 15:32:27.545657] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-25 15:32:27.548170] E [socket.c:1715:socket_connect_finish] 0-test-client-2: connection to 10.70.35.86:24007 failed (Connection refused) >[2012-09-25 15:32:27.548208] W [dht-common.c:4696:dht_notify] 0-test-dht: Received CHILD_DOWN. Exiting >[2012-09-25 15:32:27.552732] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-25 15:32:27.553399] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-25 15:32:27.556005] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-25 15:32:27.556033] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 0, failures: 0 >[2012-09-25 15:32:27.556388] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-25 15:34:56.636437] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-25 15:35:01.641649] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-25 15:35:01.641710] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-25 15:35:01.643318] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-25 15:35:01.643335] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-25 15:35:01.643343] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-25 15:35:01.643351] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-25 15:35:01.643359] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-25 15:35:01.643448] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-25 15:35:01.643507] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-25 15:35:01.645300] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-25 15:35:01.647637] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-25 15:35:01.650018] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-25 15:35:01.652291] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-25 15:35:01.652534] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-25 15:35:01.652564] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-25 15:35:05.645924] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-25 15:35:05.646097] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-25 15:35:05.646111] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-25 15:35:05.646204] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-25 15:35:05.648562] E [socket.c:1715:socket_connect_finish] 0-test-client-2: connection to 10.70.35.86:24007 failed (Connection refused) >[2012-09-25 15:35:05.648595] W [dht-common.c:4696:dht_notify] 0-test-dht: Received CHILD_DOWN. Exiting >[2012-09-25 15:35:05.651214] E [socket.c:1715:socket_connect_finish] 0-test-client-1: connection to 10.70.35.85:24007 failed (Connection refused) >[2012-09-25 15:35:05.651265] W [dht-common.c:4696:dht_notify] 0-test-dht: Received CHILD_DOWN. Exiting >[2012-09-25 15:35:05.656589] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-25 15:35:05.656987] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-25 15:35:05.672880] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-25 15:35:05.672912] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 41, failures: 0 >[2012-09-25 15:35:05.673223] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-26 10:04:12.679087] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-26 10:04:17.681304] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-26 10:04:17.681361] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-26 10:04:17.683536] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-26 10:04:17.683550] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-26 10:04:17.683559] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-26 10:04:17.683567] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-26 10:04:17.683575] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-26 10:04:17.683770] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-26 10:04:17.683813] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-26 10:04:17.685551] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-26 10:04:17.687759] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-26 10:04:17.689753] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-26 10:04:17.692118] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-26 10:04:17.692488] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-26 10:04:17.692562] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-26 10:04:20.697975] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 10:04:20.698206] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-26 10:04:20.698226] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 10:04:20.698499] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-26 10:04:20.701976] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-2: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 10:04:20.702365] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-2: Connected to 10.70.35.86:24211, attached to remote volume '/home/t1'. >[2012-09-26 10:04:20.702379] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 10:04:20.702637] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-2: Server lk version = 1 >[2012-09-26 10:04:20.704476] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 10:04:20.704820] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-26 10:04:20.704840] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 10:04:20.705240] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-26 10:04:20.708891] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-26 10:04:20.709638] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-26 10:04:20.715085] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d7: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:20.749111] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d7 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:20.818391] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d13: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:20.831396] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d13 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:20.842988] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d14: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:20.864827] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d14 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:20.875684] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d15: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:20.923064] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d15 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:20.945990] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d16: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:20.970890] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d16 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.028835] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d21: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.054050] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d21 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.078072] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d25: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.103876] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d25 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.139042] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d36: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.165225] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d36 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.222470] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f3: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.246302] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f3 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.258490] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f11: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.292169] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f11 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.311628] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f14: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.323944] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f14 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.356534] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f15: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.390299] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f15 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.422656] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f31: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.558117] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f31 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.575577] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f33: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.587004] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f33 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.600123] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f37: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.630938] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f37 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.649853] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /f38: attempting to move from test-client-0 to test-client-2 >[2012-09-26 10:04:21.677750] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /f38 from subvolume test-client-0 to test-client-2 >[2012-09-26 10:04:21.678579] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-test-client-1: remote operation failed: No such file or directory. Path: /d10 (00000000-0000-0000-0000-000000000000). Key: trusted.glusterfs.node-uuid >[2012-09-26 10:04:21.678607] E [dht-common.c:1748:dht_vgetxattr_cbk] 0-test-dht: Subvolume test-client-1 returned -1 (No such file or directory) >[2012-09-26 10:04:21.678626] E [dht-rebalance.c:1142:gf_defrag_migrate_data] 0-test-dht: Failed to get node-uuid for /d10 >[2012-09-26 10:04:21.679497] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-test-client-1: remote operation failed: No such file or directory. Path: /d11 (00000000-0000-0000-0000-000000000000). Key: trusted.glusterfs.node-uuid >[2012-09-26 10:04:21.679527] E [dht-common.c:1748:dht_vgetxattr_cbk] 0-test-dht: Subvolume test-client-1 returned -1 (No such file or directory) >[2012-09-26 10:04:21.679547] E [dht-rebalance.c:1142:gf_defrag_migrate_data] 0-test-dht: Failed to get node-uuid for /d11 >[2012-09-26 10:04:21.680194] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-test-client-1: remote operation failed: No such file or directory. Path: /f17 (00000000-0000-0000-0000-000000000000). Key: trusted.glusterfs.node-uuid >[2012-09-26 10:04:21.680217] E [dht-common.c:1748:dht_vgetxattr_cbk] 0-test-dht: Subvolume test-client-1 returned -1 (No such file or directory) >[2012-09-26 10:04:21.680235] E [dht-rebalance.c:1142:gf_defrag_migrate_data] 0-test-dht: Failed to get node-uuid for /f17 >[2012-09-26 10:04:21.930487] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-26 10:04:21.930538] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 16, size: 0, lookups: 102, failures: 0 >[2012-09-26 10:04:21.930929] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-26 11:24:06.254755] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-26 11:24:11.341593] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-26 11:24:11.341655] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-26 11:24:11.343457] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-26 11:24:11.343475] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-26 11:24:11.343493] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-26 11:24:11.343505] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-26 11:24:11.343515] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-26 11:24:11.343626] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-26 11:24:11.343685] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-26 11:24:11.345775] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-26 11:24:11.348818] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-26 11:24:11.350820] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-26 11:24:11.353094] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-26 11:24:11.353474] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-26 11:24:11.417912] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-26 11:24:15.265223] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 11:24:15.265467] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-26 11:24:15.265486] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 11:24:15.265606] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-26 11:24:15.269877] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 11:24:15.270058] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-26 11:24:15.270073] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 11:24:15.270182] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-26 11:24:15.273411] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-2: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 11:24:15.273651] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-2: Connected to 10.70.35.86:24211, attached to remote volume '/home/t1'. >[2012-09-26 11:24:15.273679] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 11:24:15.273904] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-2: Server lk version = 1 >[2012-09-26 11:24:15.279436] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-26 11:24:15.280493] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-26 11:24:15.293109] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1 >[2012-09-26 11:24:15.294226] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1 >[2012-09-26 11:24:15.303933] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level1 >[2012-09-26 11:24:15.304632] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level1 >[2012-09-26 11:24:15.312919] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level2 >[2012-09-26 11:24:15.313623] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level2 >[2012-09-26 11:24:15.318360] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level3 >[2012-09-26 11:24:15.319108] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level3 >[2012-09-26 11:24:15.323889] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level4 >[2012-09-26 11:24:15.325270] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level4 >[2012-09-26 11:24:15.339383] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level5 >[2012-09-26 11:24:15.340099] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level5 >[2012-09-26 11:24:15.347993] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level6 >[2012-09-26 11:24:15.348726] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level6 >[2012-09-26 11:24:15.354947] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level7 >[2012-09-26 11:24:15.355665] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level7 >[2012-09-26 11:24:15.362347] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level8 >[2012-09-26 11:24:15.362999] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level8 >[2012-09-26 11:24:15.380273] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level9 >[2012-09-26 11:24:15.381159] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level9 >[2012-09-26 11:24:15.402940] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level10 >[2012-09-26 11:24:15.403656] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level10 >[2012-09-26 11:24:15.414969] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level11 >[2012-09-26 11:24:15.415708] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level11 >[2012-09-26 11:24:15.421364] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level12 >[2012-09-26 11:24:15.422140] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level12 >[2012-09-26 11:24:15.431600] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level13 >[2012-09-26 11:24:15.432338] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level13 >[2012-09-26 11:24:15.452446] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level14 >[2012-09-26 11:24:15.453320] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level14 >[2012-09-26 11:24:15.464742] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level15 >[2012-09-26 11:24:15.465223] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level15 >[2012-09-26 11:24:15.470408] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level16 >[2012-09-26 11:24:15.471117] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level16 >[2012-09-26 11:24:15.479000] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level17 >[2012-09-26 11:24:15.479890] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level17 >[2012-09-26 11:24:15.489837] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level18 >[2012-09-26 11:24:15.490359] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level18 >[2012-09-26 11:24:15.495325] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level19 >[2012-09-26 11:24:15.496097] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level19 >[2012-09-26 11:24:15.505468] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/level20 >[2012-09-26 11:24:15.506249] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/level20 >[2012-09-26 11:24:15.516940] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2 >[2012-09-26 11:24:15.517582] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2 >[2012-09-26 11:24:15.533540] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level1 >[2012-09-26 11:24:15.534276] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level1 >[2012-09-26 11:24:15.540085] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level2 >[2012-09-26 11:24:15.543457] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level2 >[2012-09-26 11:24:15.556984] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level3 >[2012-09-26 11:24:15.562608] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level3 >[2012-09-26 11:24:15.570927] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level4 >[2012-09-26 11:24:15.571641] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level4 >[2012-09-26 11:24:15.582969] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level5 >[2012-09-26 11:24:15.583708] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level5 >[2012-09-26 11:24:15.590898] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level6 >[2012-09-26 11:24:15.591744] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level6 >[2012-09-26 11:24:15.600941] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level7 >[2012-09-26 11:24:15.601487] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level7 >[2012-09-26 11:24:15.605663] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level8 >[2012-09-26 11:24:15.606300] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level8 >[2012-09-26 11:24:15.624017] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level9 >[2012-09-26 11:24:15.624619] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level9 >[2012-09-26 11:24:15.642436] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level10 >[2012-09-26 11:24:15.643092] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level10 >[2012-09-26 11:24:15.658252] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level11 >[2012-09-26 11:24:15.659034] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level11 >[2012-09-26 11:24:15.672278] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level12 >[2012-09-26 11:24:15.673070] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level12 >[2012-09-26 11:24:15.690103] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level13 >[2012-09-26 11:24:15.690939] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level13 >[2012-09-26 11:24:15.711946] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level14 >[2012-09-26 11:24:15.712774] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level14 >[2012-09-26 11:24:15.720143] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level15 >[2012-09-26 11:24:15.720812] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level15 >[2012-09-26 11:24:15.725952] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level16 >[2012-09-26 11:24:15.726627] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level16 >[2012-09-26 11:24:15.731248] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level17 >[2012-09-26 11:24:15.735042] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level17 >[2012-09-26 11:24:15.744503] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level18 >[2012-09-26 11:24:15.745196] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level18 >[2012-09-26 11:24:15.750054] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level19 >[2012-09-26 11:24:15.750685] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level19 >[2012-09-26 11:24:15.757889] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2/level20 >[2012-09-26 11:24:15.758400] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2/level20 >[2012-09-26 11:24:15.765894] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d3 >[2012-09-26 11:24:15.769252] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d3 >[2012-09-26 11:24:15.773634] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d4 >[2012-09-26 11:24:15.774303] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d4 >[2012-09-26 11:24:15.781457] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d5 >[2012-09-26 11:24:15.781981] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d5 >[2012-09-26 11:24:15.785349] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d6 >[2012-09-26 11:24:15.785760] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d6 >[2012-09-26 11:24:15.789643] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d7 >[2012-09-26 11:24:15.792915] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d7 >[2012-09-26 11:24:15.797806] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d8 >[2012-09-26 11:24:15.798301] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d8 >[2012-09-26 11:24:15.802910] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d9 >[2012-09-26 11:24:15.803382] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d9 >[2012-09-26 11:24:15.810727] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d10 >[2012-09-26 11:24:15.813312] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d10 >[2012-09-26 11:24:15.821268] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d11 >[2012-09-26 11:24:15.821930] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d11 >[2012-09-26 11:24:15.825625] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d12 >[2012-09-26 11:24:15.826261] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d12 >[2012-09-26 11:24:15.830486] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d13 >[2012-09-26 11:24:15.830983] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d13 >[2012-09-26 11:24:15.843961] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d14 >[2012-09-26 11:24:15.845437] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d14 >[2012-09-26 11:24:15.859040] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d15 >[2012-09-26 11:24:15.859696] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d15 >[2012-09-26 11:24:15.868117] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d16 >[2012-09-26 11:24:15.868934] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d16 >[2012-09-26 11:24:15.877093] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d17 >[2012-09-26 11:24:15.877744] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d17 >[2012-09-26 11:24:15.891017] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d18 >[2012-09-26 11:24:15.891536] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d18 >[2012-09-26 11:24:15.903118] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d19 >[2012-09-26 11:24:15.905527] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d19 >[2012-09-26 11:24:15.917052] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d20 >[2012-09-26 11:24:15.917644] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d20 >[2012-09-26 11:24:15.929119] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d21 >[2012-09-26 11:24:15.929746] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d21 >[2012-09-26 11:24:15.936728] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d22 >[2012-09-26 11:24:15.937211] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d22 >[2012-09-26 11:24:15.940656] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d23 >[2012-09-26 11:24:15.941483] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d23 >[2012-09-26 11:24:15.948743] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d24 >[2012-09-26 11:24:15.949364] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d24 >[2012-09-26 11:24:15.956346] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d25 >[2012-09-26 11:24:15.956897] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d25 >[2012-09-26 11:24:15.964668] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d26 >[2012-09-26 11:24:15.965502] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d26 >[2012-09-26 11:24:15.977966] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d27 >[2012-09-26 11:24:15.978931] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d27 >[2012-09-26 11:24:15.988320] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d28 >[2012-09-26 11:24:15.989154] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d28 >[2012-09-26 11:24:15.997773] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d29 >[2012-09-26 11:24:15.998354] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d29 >[2012-09-26 11:24:16.012998] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d30 >[2012-09-26 11:24:16.014242] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d30 >[2012-09-26 11:24:16.022940] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d31 >[2012-09-26 11:24:16.023594] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d31 >[2012-09-26 11:24:16.027923] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d32 >[2012-09-26 11:24:16.028577] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d32 >[2012-09-26 11:24:16.036614] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33 >[2012-09-26 11:24:16.037751] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33 >[2012-09-26 11:24:16.048072] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level1 >[2012-09-26 11:24:16.048650] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level1 >[2012-09-26 11:24:16.053987] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level2 >[2012-09-26 11:24:16.054658] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level2 >[2012-09-26 11:24:16.063236] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level3 >[2012-09-26 11:24:16.064404] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level3 >[2012-09-26 11:24:16.074046] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level4 >[2012-09-26 11:24:16.074881] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level4 >[2012-09-26 11:24:16.083874] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level5 >[2012-09-26 11:24:16.084340] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level5 >[2012-09-26 11:24:16.092229] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level6 >[2012-09-26 11:24:16.092853] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level6 >[2012-09-26 11:24:16.097254] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level7 >[2012-09-26 11:24:16.099478] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level7 >[2012-09-26 11:24:16.105913] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level8 >[2012-09-26 11:24:16.106318] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level8 >[2012-09-26 11:24:16.123550] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level9 >[2012-09-26 11:24:16.124294] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level9 >[2012-09-26 11:24:16.141978] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level10 >[2012-09-26 11:24:16.142774] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level10 >[2012-09-26 11:24:16.150356] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level11 >[2012-09-26 11:24:16.153796] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level11 >[2012-09-26 11:24:16.162129] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level12 >[2012-09-26 11:24:16.162506] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level12 >[2012-09-26 11:24:16.173554] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level13 >[2012-09-26 11:24:16.174046] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level13 >[2012-09-26 11:24:16.189088] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level14 >[2012-09-26 11:24:16.189576] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level14 >[2012-09-26 11:24:16.193798] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level15 >[2012-09-26 11:24:16.195994] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level15 >[2012-09-26 11:24:16.203576] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level16 >[2012-09-26 11:24:16.204159] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level16 >[2012-09-26 11:24:16.212297] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level17 >[2012-09-26 11:24:16.216929] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level17 >[2012-09-26 11:24:16.220920] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level18 >[2012-09-26 11:24:16.223983] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level18 >[2012-09-26 11:24:16.230528] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level19 >[2012-09-26 11:24:16.231147] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level19 >[2012-09-26 11:24:16.239477] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/level20 >[2012-09-26 11:24:16.243954] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/level20 >[2012-09-26 11:24:16.248113] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/d33 >[2012-09-26 11:24:16.248545] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/d33 >[2012-09-26 11:24:16.252210] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/d2 >[2012-09-26 11:24:16.252813] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/d2 >[2012-09-26 11:24:16.272969] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d34 >[2012-09-26 11:24:16.338073] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d34 >[2012-09-26 11:24:16.346674] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d35 >[2012-09-26 11:24:16.347211] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d35 >[2012-09-26 11:24:16.356880] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d36 >[2012-09-26 11:24:16.357480] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d36 >[2012-09-26 11:24:16.365314] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d37 >[2012-09-26 11:24:16.368929] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d37 >[2012-09-26 11:24:16.377539] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d38 >[2012-09-26 11:24:16.378269] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d38 >[2012-09-26 11:24:16.386404] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d39 >[2012-09-26 11:24:16.386882] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d39 >[2012-09-26 11:24:16.390414] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d40 >[2012-09-26 11:24:16.390907] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d40 >[2012-09-26 11:24:16.400537] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-26 11:24:16.400564] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 180, failures: 0 >[2012-09-26 11:24:16.400944] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-26 11:42:44.653276] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-26 11:42:49.655606] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-26 11:42:49.655662] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-26 11:42:49.657664] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-26 11:42:49.657683] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-26 11:42:49.657696] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-26 11:42:49.657717] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-26 11:42:49.657729] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-26 11:42:49.657840] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-26 11:42:49.657922] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-26 11:42:49.659921] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-26 11:42:49.662949] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-26 11:42:49.665490] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-26 11:42:49.667720] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-26 11:42:49.759943] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-26 11:42:49.809739] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-26 11:42:53.663457] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 11:42:53.663683] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-26 11:42:53.663703] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 11:42:53.663973] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-26 11:42:53.667489] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 11:42:53.667774] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-26 11:42:53.667794] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 11:42:53.667971] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-26 11:42:53.672561] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-2: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 11:42:53.672898] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-2: Connected to 10.70.35.86:24211, attached to remote volume '/home/t1'. >[2012-09-26 11:42:53.672913] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 11:42:53.674014] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-2: Server lk version = 1 >[2012-09-26 11:42:53.679280] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-26 11:42:53.683501] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-26 11:42:53.700895] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d3 >[2012-09-26 11:42:53.702400] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d3 >[2012-09-26 11:42:53.717294] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d4 >[2012-09-26 11:42:53.720753] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d4 >[2012-09-26 11:42:53.728012] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d5 >[2012-09-26 11:42:53.728779] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d5 >[2012-09-26 11:42:53.739944] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d6 >[2012-09-26 11:42:53.740780] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d6 >[2012-09-26 11:42:53.755744] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d7 >[2012-09-26 11:42:53.759617] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d7 >[2012-09-26 11:42:53.772072] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d8 >[2012-09-26 11:42:53.772772] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d8 >[2012-09-26 11:42:53.786314] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d9 >[2012-09-26 11:42:53.792246] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d9 >[2012-09-26 11:42:53.801534] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d10 >[2012-09-26 11:42:53.802238] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d10 >[2012-09-26 11:42:53.810797] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d11 >[2012-09-26 11:42:53.811503] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d11 >[2012-09-26 11:42:53.827782] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d12 >[2012-09-26 11:42:53.828539] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d12 >[2012-09-26 11:42:53.837523] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d13 >[2012-09-26 11:42:53.838221] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d13 >[2012-09-26 11:42:53.854043] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d14 >[2012-09-26 11:42:53.857486] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d14 >[2012-09-26 11:42:53.867121] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d15 >[2012-09-26 11:42:53.873016] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d15 >[2012-09-26 11:42:53.880175] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d16 >[2012-09-26 11:42:53.880933] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d16 >[2012-09-26 11:42:53.896674] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d17 >[2012-09-26 11:42:53.898228] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d17 >[2012-09-26 11:42:53.910958] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d18 >[2012-09-26 11:42:53.911532] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d18 >[2012-09-26 11:42:53.929267] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d19 >[2012-09-26 11:42:53.930060] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d19 >[2012-09-26 11:42:53.947082] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d20 >[2012-09-26 11:42:53.950599] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d20 >[2012-09-26 11:42:53.963787] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d21 >[2012-09-26 11:42:53.967645] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d21 >[2012-09-26 11:42:53.985239] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d22 >[2012-09-26 11:42:53.985891] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d22 >[2012-09-26 11:42:53.997576] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d23 >[2012-09-26 11:42:53.998192] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d23 >[2012-09-26 11:42:54.005492] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d24 >[2012-09-26 11:42:54.006302] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d24 >[2012-09-26 11:42:54.015389] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d25 >[2012-09-26 11:42:54.017013] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d25 >[2012-09-26 11:42:54.034042] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d26 >[2012-09-26 11:42:54.034634] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d26 >[2012-09-26 11:42:54.048000] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d27 >[2012-09-26 11:42:54.048630] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d27 >[2012-09-26 11:42:54.062163] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d28 >[2012-09-26 11:42:54.062675] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d28 >[2012-09-26 11:42:54.077084] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d29 >[2012-09-26 11:42:54.077827] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d29 >[2012-09-26 11:42:54.089450] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d30 >[2012-09-26 11:42:54.090252] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d30 >[2012-09-26 11:42:54.097932] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d31 >[2012-09-26 11:42:54.104095] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d31 >[2012-09-26 11:42:54.113012] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d32 >[2012-09-26 11:42:54.113553] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d32 >[2012-09-26 11:42:54.120904] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d34 >[2012-09-26 11:42:54.123813] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d34 >[2012-09-26 11:42:54.140827] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d35 >[2012-09-26 11:42:54.141379] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d35 >[2012-09-26 11:42:54.151989] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d36 >[2012-09-26 11:42:54.155643] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d36 >[2012-09-26 11:42:54.168215] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d37 >[2012-09-26 11:42:54.171624] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d37 >[2012-09-26 11:42:54.185028] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d38 >[2012-09-26 11:42:54.188669] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d38 >[2012-09-26 11:42:54.195712] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d39 >[2012-09-26 11:42:54.196348] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d39 >[2012-09-26 11:42:54.206698] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d40 >[2012-09-26 11:42:54.207698] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d40 >[2012-09-26 11:42:54.220485] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1 >[2012-09-26 11:42:54.221449] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1 >[2012-09-26 11:42:54.228721] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/d33 >[2012-09-26 11:42:54.233035] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/d33 >[2012-09-26 11:42:54.242580] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/d2 >[2012-09-26 11:42:54.247893] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/d2 >[2012-09-26 11:42:54.255022] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/d1 >[2012-09-26 11:42:54.257556] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/d1 >[2012-09-26 11:42:54.268911] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/d1/abc >[2012-09-26 11:42:54.274273] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/d1/abc >[2012-09-26 11:42:54.283018] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/d1/d1 >[2012-09-26 11:42:54.283816] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/d1/d1 >[2012-09-26 11:42:54.297583] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1/d1/d2 >[2012-09-26 11:42:54.298362] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1/d1/d2 >[2012-09-26 11:42:54.317649] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33 >[2012-09-26 11:42:54.318724] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33 >[2012-09-26 11:42:54.327452] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/d2 >[2012-09-26 11:42:54.330981] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/d2 >[2012-09-26 11:42:54.343368] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d33/d33 >[2012-09-26 11:42:54.343932] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d33/d33 >[2012-09-26 11:42:54.365977] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d2 >[2012-09-26 11:42:54.369918] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d2 >[2012-09-26 11:42:54.392307] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-26 11:42:54.392335] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 0, failures: 0 >[2012-09-26 11:42:54.392635] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-26 12:13:46.592078] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-26 12:13:51.621730] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-26 12:13:51.621785] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-26 12:13:51.623920] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-26 12:13:51.623939] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-26 12:13:51.623952] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-26 12:13:51.623963] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-26 12:13:51.623974] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-26 12:13:51.624086] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-26 12:13:51.624147] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-26 12:13:51.626110] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-26 12:13:51.628261] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-26 12:13:51.630217] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-26 12:13:51.632503] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-26 12:13:51.632880] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-26 12:13:51.633061] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-26 12:13:55.605613] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 12:13:55.605829] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-26 12:13:55.605843] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 12:13:55.606063] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-26 12:13:55.610162] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 12:13:55.610438] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-26 12:13:55.610454] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 12:13:55.610587] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-26 12:13:55.615280] E [socket.c:1715:socket_connect_finish] 0-test-client-2: connection to 10.70.35.86:24007 failed (Connection refused) >[2012-09-26 12:13:55.615563] W [dht-common.c:4696:dht_notify] 0-test-dht: Received CHILD_DOWN. Exiting >[2012-09-26 12:13:55.620588] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-26 12:13:55.621572] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-26 12:13:55.638365] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-26 12:13:55.638392] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 0, size: 0, lookups: 20, failures: 0 >[2012-09-26 12:13:55.638673] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down >[2012-09-26 12:31:48.066280] I [glusterfsd.c:1666:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.3.0rhs >[2012-09-26 12:31:53.118163] W [socket.c:410:__socket_keepalive] 0-socket: failed to set keep idle on socket 8 >[2012-09-26 12:31:53.118229] W [socket.c:1876:socket_server_event_handler] 0-socket.glusterfsd: Failed to set keep-alive: Operation not supported >[2012-09-26 12:31:53.120098] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'node-uuid' for volume 'test-dht' with value '74702ea2-e56e-4663-b755-d60b8b1fa988' >[2012-09-26 12:31:53.120111] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'rebalance-cmd' for volume 'test-dht' with value '1' >[2012-09-26 12:31:53.120120] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'assert-no-child-down' for volume 'test-dht' with value 'yes' >[2012-09-26 12:31:53.120128] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'lookup-unhashed' for volume 'test-dht' with value 'yes' >[2012-09-26 12:31:53.120135] I [graph.c:241:gf_add_cmdline_options] 0-test-dht: adding option 'use-readdirp' for volume 'test-dht' with value 'yes' >[2012-09-26 12:31:53.120325] I [io-cache.c:1549:check_cache_size_ok] 0-test-quick-read: Max cache size is 1969029120 >[2012-09-26 12:31:53.120366] I [io-cache.c:1549:check_cache_size_ok] 0-test-io-cache: Max cache size is 1969029120 >[2012-09-26 12:31:53.121775] I [client.c:2142:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2012-09-26 12:31:53.124186] I [client.c:2142:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >[2012-09-26 12:31:53.126250] I [client.c:2142:notify] 0-test-client-2: parent translators are ready, attempting connect on transport >Given volfile: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host 10.70.35.81 > 4: option remote-subvolume /home/t1 > 5: option transport-type tcp > 6: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 7: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 8: end-volume > 9: > 10: volume test-client-1 > 11: type protocol/client > 12: option remote-host 10.70.35.85 > 13: option remote-subvolume /home/t1 > 14: option transport-type tcp > 15: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 16: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 17: end-volume > 18: > 19: volume test-client-2 > 20: type protocol/client > 21: option remote-host 10.70.35.86 > 22: option remote-subvolume /home/t1 > 23: option transport-type tcp > 24: option username 3ab7d152-afdc-4de0-914a-5a5669e0d612 > 25: option password 8a578c01-5da1-4f5a-90b0-ab4f185dae9b > 26: end-volume > 27: > 28: volume test-dht > 29: type cluster/distribute > 30: subvolumes test-client-0 test-client-1 test-client-2 > 31: end-volume > 32: > 33: volume test-write-behind > 34: type performance/write-behind > 35: subvolumes test-dht > 36: end-volume > 37: > 38: volume test-read-ahead > 39: type performance/read-ahead > 40: subvolumes test-write-behind > 41: end-volume > 42: > 43: volume test-io-cache > 44: type performance/io-cache > 45: subvolumes test-read-ahead > 46: end-volume > 47: > 48: volume test-quick-read > 49: type performance/quick-read > 50: subvolumes test-io-cache > 51: end-volume > 52: > 53: volume test-md-cache > 54: type performance/md-cache > 55: subvolumes test-quick-read > 56: end-volume > 57: > 58: volume test > 59: type debug/io-stats > 60: option latency-measurement off > 61: option count-fop-hits off > 62: subvolumes test-md-cache > 63: end-volume > >+------------------------------------------------------------------------------+ >[2012-09-26 12:31:53.128550] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-0: changing port to 24009 (from 0) >[2012-09-26 12:31:53.128912] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-1: changing port to 24211 (from 0) >[2012-09-26 12:31:53.141247] I [rpc-clnt.c:1659:rpc_clnt_reconfig] 0-test-client-2: changing port to 24211 (from 0) >[2012-09-26 12:31:57.081951] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 12:31:57.082423] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-0: Connected to 10.70.35.81:24009, attached to remote volume '/home/t1'. >[2012-09-26 12:31:57.082439] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 12:31:57.082919] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2012-09-26 12:31:57.085954] I [client-handshake.c:1636:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3.0rhs, Num (1298437), Version (330) >[2012-09-26 12:31:57.086236] I [client-handshake.c:1433:client_setvolume_cbk] 0-test-client-1: Connected to 10.70.35.85:24211, attached to remote volume '/home/t1'. >[2012-09-26 12:31:57.086251] I [client-handshake.c:1445:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2012-09-26 12:31:57.086398] I [client-handshake.c:453:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2012-09-26 12:31:57.088250] E [socket.c:1715:socket_connect_finish] 0-test-client-2: connection to 10.70.35.86:24007 failed (Connection refused) >[2012-09-26 12:31:57.088288] W [dht-common.c:4696:dht_notify] 0-test-dht: Received CHILD_DOWN. Exiting >[2012-09-26 12:31:57.094457] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of / >[2012-09-26 12:31:57.094893] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on / >[2012-09-26 12:31:57.118048] I [dht-common.c:2337:dht_setxattr] 0-test-dht: fixing the layout of /d1 >[2012-09-26 12:31:57.121262] I [dht-rebalance.c:1055:gf_defrag_migrate_data] 0-test-dht: migrate data called on /d1 >[2012-09-26 12:31:57.127897] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f3: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.161266] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f3 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.163427] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f11: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.194334] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f11 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.199503] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f14: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.247432] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f14 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.248529] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f15: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.285055] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f15 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.295473] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f31: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.354058] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f31 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.356526] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f33: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.381855] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f33 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.382982] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f37: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.413882] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f37 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.415278] I [dht-rebalance.c:639:dht_migrate_file] 0-test-dht: /d1/f38: attempting to move from test-client-0 to test-client-1 >[2012-09-26 12:31:57.448291] I [dht-rebalance.c:848:dht_migrate_file] 0-test-dht: completed migration of /d1/f38 from subvolume test-client-0 to test-client-1 >[2012-09-26 12:31:57.464813] I [dht-rebalance.c:1611:gf_defrag_status_get] 0-glusterfs: Rebalance is completed >[2012-09-26 12:31:57.464841] I [dht-rebalance.c:1614:gf_defrag_status_get] 0-glusterfs: Files migrated: 8, size: 0, lookups: 60, failures: 0 >[2012-09-26 12:31:57.465213] W [glusterfsd.c:831:cleanup_and_exit] (-->/lib64/libc.so.6(clone+0x6d) [0x3d814e811d] (-->/lib64/libpthread.so.0() [0x3d81c07851] (-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xdd) [0x405cfd]))) 0-: received signum (15), shutting down
<b>You cannot view the attachment while viewing its details because your browser does not support IFRAMEs. <a href="attachment.cgi?id=617485">View the attachment on a separate page</a>.</b>
View Attachment As Raw
Actions:
View
Attachments on
bug 860599
:
617484
| 617485