Bug 764864 (GLUSTER-3132)

Summary: issues of layout mismatch while using distributed-stripe volume
Product: [Community] GlusterFS Reporter: Amar Tumballi <amarts>
Component: stripeAssignee: Amar Tumballi <amarts>
Status: CLOSED DUPLICATE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 3.2.1CC: gluster-bugs, sgowda, vraman
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description shishir gowda 2011-07-06 02:33:28 UTC
This was fixed in master by this patch:
http://patches.gluster.com/patch/7573/

Marking it as dup of the bug 764798

*** This bug has been marked as a duplicate of bug 3066 ***

Comment 1 Amar Tumballi 2011-07-06 05:24:19 UTC
translators are ready, attempting connect on transport
[2011-06-24 13:10:12.121885] I [client.c:1935:notify] 0-tvol-client-4: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.125288] I [client.c:1935:notify] 0-tvol-client-5: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.128688] I [client.c:1935:notify] 0-tvol-client-6: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.132070] I [client.c:1935:notify] 0-tvol-client-7: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.135476] I [client.c:1935:notify] 0-tvol-client-8: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.138878] I [client.c:1935:notify] 0-tvol-client-9: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.142262] I [client.c:1935:notify] 0-tvol-client-10: parent translators are ready, attempting connect on transport
[2011-06-24 13:10:12.145652] I [client.c:1935:notify] 0-tvol-client-11: parent translators are ready, attempting connect on transport
Given volfile:
+------------------------------------------------------------------------------+
  1: volume tvol-client-0
  2:     type protocol/client
  3:     option remote-host 192.168.8.200
  4:     option remote-subvolume /mnt/sdb/tvol
  5:     option transport-type tcp
  6: end-volume
  7: 
  8: volume tvol-client-1
  9:     type protocol/client
 10:     option remote-host 192.168.8.201
 11:     option remote-subvolume /mnt/sdb/tvol
 12:     option transport-type tcp
 13: end-volume
 14: 
 15: volume tvol-client-2
 16:     type protocol/client
 17:     option remote-host 192.168.8.202
 18:     option remote-subvolume /mnt/sdb/tvol
 19:     option transport-type tcp
 20: end-volume
 21: 
 22: volume tvol-client-3
 23:     type protocol/client
 24:     option remote-host 192.168.8.203
 25:     option remote-subvolume /mnt/sdb/tvol
 26:     option transport-type tcp
 27: end-volume
 28: 
 29: volume tvol-client-4
 30:     type protocol/client
 31:     option remote-host 192.168.8.200
 32:     option remote-subvolume /mnt/sdc/tvol
 33:     option transport-type tcp
 34: end-volume
 35: 
 36: volume tvol-client-5
 37:     type protocol/client
 38:     option remote-host 192.168.8.201
 39:     option remote-subvolume /mnt/sdc/tvol
 40:     option transport-type tcp
 41: end-volume
 42: 
 43: volume tvol-client-6
 44:     type protocol/client
 45:     option remote-host 192.168.8.202
 46:     option remote-subvolume /mnt/sdc/tvol
 47:     option transport-type tcp
 48: end-volume
 49: 
 50: volume tvol-client-7
 51:     type protocol/client
 52:     option remote-host 192.168.8.203
 53:     option remote-subvolume /mnt/sdc/tvol
 54:     option transport-type tcp
 55: end-volume
 56: 
 57: volume tvol-client-8
 58:     type protocol/client
 59:     option remote-host 192.168.8.200
 60:     option remote-subvolume /mnt/sdd/tvol
 61:     option transport-type tcp
 62: end-volume
 63: 
 64: volume tvol-client-9
 65:     type protocol/client
 66:     option remote-host 192.168.8.201
 67:     option remote-subvolume /mnt/sdd/tvol
 68:     option transport-type tcp
 69: end-volume
 70: 
 71: volume tvol-client-10
 72:     type protocol/client
 73:     option remote-host 192.168.8.202
 74:     option remote-subvolume /mnt/sdd/tvol
 75:     option transport-type tcp
 76: end-volume
 77: 
 78: volume tvol-client-11
 79:     type protocol/client
 80:     option remote-host 192.168.8.203
 81:     option remote-subvolume /mnt/sdd/tvol
 82:     option transport-type tcp
 83: end-volume
 84: 
 85: volume tvol-stripe-0
 86:     type cluster/stripe
 87:     subvolumes tvol-client-0 tvol-client-1 tvol-client-2 tvol-client-3
 88: end-volume
 89: 
 90: volume tvol-stripe-1
 91:     type cluster/stripe
 92:     subvolumes tvol-client-4 tvol-client-5 tvol-client-6 tvol-client-7
 93: end-volume
 94: 
 95: volume tvol-stripe-2
 96:     type cluster/stripe
 97:     subvolumes tvol-client-8 tvol-client-9 tvol-client-10 tvol-client-11
 98: end-volume
 99: 
100: volume tvol-dht
101:     type cluster/distribute
102:     subvolumes tvol-stripe-0 tvol-stripe-1 tvol-stripe-2
103: end-volume
104: 
105: volume tvol-write-behind
106:     type performance/write-behind
107:     subvolumes tvol-dht
108: end-volume
109: 
110: volume tvol-read-ahead
111:     type performance/read-ahead
112:     subvolumes tvol-write-behind
113: end-volume
114: 
115: volume tvol-io-cache
116:     type performance/io-cache
117:     subvolumes tvol-read-ahead
118: end-volume
119: 
120: volume tvol-quick-read
121:     type performance/quick-read
122:     subvolumes tvol-io-cache
123: end-volume
124: 
125: volume tvol-stat-prefetch
126:     type performance/stat-prefetch
127:     subvolumes tvol-quick-read
128: end-volume
129: 
130: volume tvol
131:     type debug/io-stats
132:     option latency-measurement off
133:     option count-fop-hits off
134:     subvolumes tvol-stat-prefetch
135: end-volume
+------------------------------------------------------------------------------+
[2011-06-24 13:10:12.149687] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-10: changing port to 24021 (from 0)
[2011-06-24 13:10:12.149885] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-11: changing port to 24021 (from 0)
[2011-06-24 13:10:12.149922] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-7: changing port to 24020 (from 0)
[2011-06-24 13:10:12.149952] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-9: changing port to 24021 (from 0)
[2011-06-24 13:10:12.150023] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-8: changing port to 24021 (from 0)
[2011-06-24 13:10:12.150080] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-4: changing port to 24020 (from 0)
[2011-06-24 13:10:12.150128] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-0: changing port to 24019 (from 0)
[2011-06-24 13:10:12.150184] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-6: changing port to 24020 (from 0)
[2011-06-24 13:10:12.150235] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-2: changing port to 24019 (from 0)
[2011-06-24 13:10:12.150267] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-5: changing port to 24020 (from 0)
[2011-06-24 13:10:12.150294] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-1: changing port to 24019 (from 0)
[2011-06-24 13:10:12.150348] I [rpc-clnt.c:1531:rpc_clnt_reconfig] 0-tvol-client-3: changing port to 24019 (from 0)
[2011-06-24 13:10:16.95630] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-10: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.95924] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-10: Connected to 192.168.8.202:24021, attached to remote volume '/mnt/sdd/tvol'.
[2011-06-24 13:10:16.99358] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-11: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.99863] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-11: Connected to 192.168.8.203:24021, attached to remote volume '/mnt/sdd/tvol'.
[2011-06-24 13:10:16.103193] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-7: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.103586] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-7: Connected to 192.168.8.203:24020, attached to remote volume '/mnt/sdc/tvol'.
[2011-06-24 13:10:16.106797] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-9: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.107219] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-9: Connected to 192.168.8.201:24021, attached to remote volume '/mnt/sdd/tvol'.
[2011-06-24 13:10:16.110554] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-8: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.110996] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-8: Connected to 192.168.8.200:24021, attached to remote volume '/mnt/sdd/tvol'.
[2011-06-24 13:10:16.114348] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-4: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.114778] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-4: Connected to 192.168.8.200:24020, attached to remote volume '/mnt/sdc/tvol'.
[2011-06-24 13:10:16.118199] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-0: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.118554] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-0: Connected to 192.168.8.200:24019, attached to remote volume '/mnt/sdb/tvol'.
[2011-06-24 13:10:16.121783] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-6: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.122201] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-6: Connected to 192.168.8.202:24020, attached to remote volume '/mnt/sdc/tvol'.
[2011-06-24 13:10:16.125436] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-2: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.125807] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-2: Connected to 192.168.8.202:24019, attached to remote volume '/mnt/sdb/tvol'.
[2011-06-24 13:10:16.129272] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-5: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.129711] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-5: Connected to 192.168.8.201:24020, attached to remote volume '/mnt/sdc/tvol'.
[2011-06-24 13:10:16.133023] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-1: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.133351] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-1: Connected to 192.168.8.201:24019, attached to remote volume '/mnt/sdb/tvol'.
[2011-06-24 13:10:16.136606] I [client-handshake.c:1082:select_server_supported_programs] 0-tvol-client-3: Using Program GlusterFS-3.1.0, Num (1298437), Version (310)
[2011-06-24 13:10:16.136907] I [client-handshake.c:913:client_setvolume_cbk] 0-tvol-client-3: Connected to 192.168.8.203:24019, attached to remote volume '/mnt/sdb/tvol'.
[2011-06-24 13:10:16.140506] I [fuse-bridge.c:3316:fuse_graph_setup] 0-fuse: switched to graph 0
[2011-06-24 13:10:16.140834] I [fuse-bridge.c:2897:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.13 kernel 7.10
[2011-06-24 13:10:16.141984] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=1 overlaps=0
[2011-06-24 13:10:29.323589] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:29.323627] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:29.323685] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:29.323701] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:29.323740] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:29.323754] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:29.324581] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=2 overlaps=0
[2011-06-24 13:10:29.324624] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:34.885777] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:34.885820] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:34.886470] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=1 overlaps=0
[2011-06-24 13:10:34.886498] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:34.887498] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:34.888342] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:34.888383] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 9: /a => -1 (No such file or directory)
[2011-06-24 13:10:34.888467] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:34.889398] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1867878108
[2011-06-24 13:10:34.889950] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1867878108
[2011-06-24 13:10:34.889976] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 12: /b => -1 (No such file or directory)
[2011-06-24 13:10:34.890041] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1867878108
[2011-06-24 13:10:34.890644] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:34.891197] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:34.891224] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 15: /c => -1 (No such file or directory)
[2011-06-24 13:10:34.891290] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:47.666505] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-tvol-dht: subvol: tvol-stripe-2; inode layout - 0 - 0; disk layout - 1431655765 - 2863311529
[2011-06-24 13:10:47.666540] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:47.667129] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:51.70983] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-tvol-dht: subvol: tvol-stripe-0; inode layout - 0 - 0; disk layout - 2863311530 - 4294967295
[2011-06-24 13:10:51.71015] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:51.71046] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-tvol-dht: subvol: tvol-stripe-2; inode layout - 0 - 0; disk layout - 1431655765 - 2863311529
[2011-06-24 13:10:51.71062] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:51.71593] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=2 overlaps=0
[2011-06-24 13:10:51.71622] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:51.71700] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:51.72573] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:51.72603] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 24: /a => -1 (No such file or directory)
[2011-06-24 13:10:51.72680] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:51.77017] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:51.77049] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:51.77715] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:51.77808] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:51.78400] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:51.78426] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 33: /c => -1 (No such file or directory)
[2011-06-24 13:10:51.78492] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:53.468586] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-tvol-dht: subvol: tvol-stripe-0; inode layout - 0 - 0; disk layout - 2863311530 - 4294967295
[2011-06-24 13:10:53.468620] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:53.468659] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-tvol-dht: subvol: tvol-stripe-1; inode layout - 0 - 0; disk layout - 0 - 1431655764
[2011-06-24 13:10:53.468675] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:53.469259] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=1 overlaps=0
[2011-06-24 13:10:53.469282] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:53.469368] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:53.470134] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:53.470163] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 37: /a => -1 (No such file or directory)
[2011-06-24 13:10:53.470239] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:53.471204] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1867878108
[2011-06-24 13:10:53.473207] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:53.473872] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:53.473899] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 45: /c => -1 (No such file or directory)
[2011-06-24 13:10:53.473971] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:54.466427] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:54.467153] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:54.467181] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 48: /a => -1 (No such file or directory)
[2011-06-24 13:10:54.467253] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:54.469352] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:54.470055] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:54.470085] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 55: /c => -1 (No such file or directory)
[2011-06-24 13:10:54.470153] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:55.408662] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:55.408699] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:55.409381] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=2 overlaps=0
[2011-06-24 13:10:55.409411] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:10:55.409498] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:55.410183] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:55.410212] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 59: /a => -1 (No such file or directory)
[2011-06-24 13:10:55.410313] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 974644454
[2011-06-24 13:10:55.413071] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:55.413769] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:55.413796] W [fuse-bridge.c:1567:fuse_create_cbk] 0-glusterfs-fuse: 67: /c => -1 (No such file or directory)
[2011-06-24 13:10:55.413867] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1086513363
[2011-06-24 13:10:56.433213] I [dht-layout.c:682:dht_layout_dir_mismatch] 0-tvol-dht: subvol: tvol-stripe-0; inode layout - 0 - 0; disk layout - 2863311530 - 4294967295
[2011-06-24 13:10:56.433245] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:56.433347] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:10:56.433366] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:10:56.433913] I [dht-layout.c:581:dht_layout_normalize] 0-tvol-dht: found anomalies in /. holes=1 overlaps=0
[2011-06-24 13:10:56.433942] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:11:00.598420] I [dht-layout.c:655:dht_layout_dir_mismatch] 0-tvol-dht: / - disk layout missing
[2011-06-24 13:11:00.598453] I [dht-common.c:523:dht_revalidate_cbk] 0-tvol-dht: mismatching layouts for /
[2011-06-24 13:11:00.599001] I [dht-common.c:361:dht_lookup_root_dir_cbk] 0-tvol-dht: fixing assignment on /
[2011-06-24 13:11:00.599766] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1867878108
[2011-06-24 13:11:00.600345] I [dht-layout.c:192:dht_layout_search] 0-tvol-dht: no subvolume for hash (value) = 1867878108
[2011-06-24 13:11:00.600372] W [fuse-bridge.c:905:fuse_unlink_cbk] 0-glusterfs-fuse: 79: UNLINK() /b => -1 (Invalid argument)