Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 926217 Details for
Bug 1129486
adding bricks to mounted volume causes client failures
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
mnt.log from the client
mnt.log (text/plain), 55.65 KB, created by
Scott Merrill
on 2014-08-12 21:17:02 UTC
(
hide
)
Description:
mnt.log from the client
Filename:
MIME Type:
Creator:
Scott Merrill
Created:
2014-08-12 21:17:02 UTC
Size:
55.65 KB
patch
obsolete
>[2014-08-12 21:02:27.766344] I [glusterfsd.c:1959:main] 0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version 3.5.2 (/usr/sbin/glusterfs --volfile-server=gluster1 --volfile-id=/test /mnt) >[2014-08-12 21:02:27.769890] I [socket.c:3561:socket_init] 0-glusterfs: SSL support is NOT enabled >[2014-08-12 21:02:27.769912] I [socket.c:3576:socket_init] 0-glusterfs: using system polling thread >[2014-08-12 21:02:27.777300] I [dht-shared.c:311:dht_init_regex] 0-test-dht: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >[2014-08-12 21:02:27.778506] I [socket.c:3561:socket_init] 0-test-client-1: SSL support is NOT enabled >[2014-08-12 21:02:27.778517] I [socket.c:3576:socket_init] 0-test-client-1: using system polling thread >[2014-08-12 21:02:27.778698] I [socket.c:3561:socket_init] 0-test-client-0: SSL support is NOT enabled >[2014-08-12 21:02:27.778706] I [socket.c:3576:socket_init] 0-test-client-0: using system polling thread >[2014-08-12 21:02:27.778723] I [client.c:2294:notify] 0-test-client-0: parent translators are ready, attempting connect on transport >[2014-08-12 21:02:27.781312] I [client.c:2294:notify] 0-test-client-1: parent translators are ready, attempting connect on transport >Final graph: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host gluster1 > 4: option remote-subvolume /bricks/brick1/brick > 5: option transport-type socket > 6: option send-gids true > 7: end-volume > 8: > 9: volume test-client-1 > 10: type protocol/client > 11: option remote-host gluster2 > 12: option remote-subvolume /bricks/brick1/brick > 13: option transport-type socket > 14: option send-gids true > 15: end-volume > 16: > 17: volume test-replicate-0 > 18: type cluster/replicate > 19: subvolumes test-client-0 test-client-1 > 20: end-volume > 21: > 22: volume test-dht > 23: type cluster/distribute > 24: subvolumes test-replicate-0 > 25: end-volume > 26: > 27: volume test-write-behind > 28: type performance/write-behind > 29: subvolumes test-dht > 30: end-volume > 31: > 32: volume test-read-ahead > 33: type performance/read-ahead > 34: subvolumes test-write-behind > 35: end-volume > 36: > 37: volume test-io-cache > 38: type performance/io-cache > 39: subvolumes test-read-ahead > 40: end-volume > 41: > 42: volume test-quick-read > 43: type performance/quick-read > 44: subvolumes test-io-cache > 45: end-volume > 46: > 47: volume test-open-behind > 48: type performance/open-behind > 49: subvolumes test-quick-read > 50: end-volume > 51: > 52: volume test-md-cache > 53: type performance/md-cache > 54: subvolumes test-open-behind > 55: end-volume > 56: > 57: volume test > 58: type debug/io-stats > 59: option latency-measurement off > 60: option count-fop-hits off > 61: subvolumes test-md-cache > 62: end-volume > 63: >+------------------------------------------------------------------------------+ >[2014-08-12 21:02:27.784811] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-test-client-0: changing port to 49152 (from 0) >[2014-08-12 21:02:27.787407] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 0-test-client-1: changing port to 49152 (from 0) >[2014-08-12 21:02:27.790247] I [client-handshake.c:1677:select_server_supported_programs] 0-test-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) >[2014-08-12 21:02:27.790338] I [client-handshake.c:1677:select_server_supported_programs] 0-test-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) >[2014-08-12 21:02:27.790695] I [client-handshake.c:1462:client_setvolume_cbk] 0-test-client-1: Connected to 192.168.30.108:49152, attached to remote volume '/bricks/brick1/brick'. >[2014-08-12 21:02:27.790707] I [client-handshake.c:1474:client_setvolume_cbk] 0-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2014-08-12 21:02:27.790778] I [afr-common.c:4131:afr_notify] 0-test-replicate-0: Subvolume 'test-client-1' came back up; going online. >[2014-08-12 21:02:27.790806] I [client-handshake.c:1462:client_setvolume_cbk] 0-test-client-0: Connected to 192.168.30.107:49152, attached to remote volume '/bricks/brick1/brick'. >[2014-08-12 21:02:27.790827] I [client-handshake.c:1474:client_setvolume_cbk] 0-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2014-08-12 21:02:27.793556] I [fuse-bridge.c:4977:fuse_graph_setup] 0-fuse: switched to graph 0 >[2014-08-12 21:02:27.793603] I [client-handshake.c:450:client_set_lk_version_cbk] 0-test-client-0: Server lk version = 1 >[2014-08-12 21:02:27.793616] I [client-handshake.c:450:client_set_lk_version_cbk] 0-test-client-1: Server lk version = 1 >[2014-08-12 21:02:27.793749] I [fuse-bridge.c:3914:fuse_init] 0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 kernel 7.13 >[2014-08-12 21:02:27.795069] I [dht-layout.c:640:dht_layout_normalize] 0-test-dht: found anomalies in /. holes=1 overlaps=0 >[2014-08-12 21:04:23.227185] I [glusterfsd-mgmt.c:56:mgmt_cbk_spec] 0-mgmt: Volume file changed >[2014-08-12 21:04:23.232454] I [dht-shared.c:311:dht_init_regex] 1-test-dht: using regex rsync-hash-regex = ^\.(.+)\.[^.]+$ >[2014-08-12 21:04:23.237454] I [socket.c:3561:socket_init] 1-test-client-3: SSL support is NOT enabled >[2014-08-12 21:04:23.237466] I [socket.c:3576:socket_init] 1-test-client-3: using system polling thread >[2014-08-12 21:04:23.237775] I [socket.c:3561:socket_init] 1-test-client-2: SSL support is NOT enabled >[2014-08-12 21:04:23.237784] I [socket.c:3576:socket_init] 1-test-client-2: using system polling thread >[2014-08-12 21:04:23.238048] I [socket.c:3561:socket_init] 1-test-client-1: SSL support is NOT enabled >[2014-08-12 21:04:23.238057] I [socket.c:3576:socket_init] 1-test-client-1: using system polling thread >[2014-08-12 21:04:23.238256] I [socket.c:3561:socket_init] 1-test-client-0: SSL support is NOT enabled >[2014-08-12 21:04:23.238266] I [socket.c:3576:socket_init] 1-test-client-0: using system polling thread >[2014-08-12 21:04:23.238278] I [client.c:2294:notify] 1-test-client-0: parent translators are ready, attempting connect on transport >[2014-08-12 21:04:23.241006] I [client.c:2294:notify] 1-test-client-1: parent translators are ready, attempting connect on transport >[2014-08-12 21:04:23.246067] I [client.c:2294:notify] 1-test-client-2: parent translators are ready, attempting connect on transport >[2014-08-12 21:04:23.248585] I [client.c:2294:notify] 1-test-client-3: parent translators are ready, attempting connect on transport >Final graph: >+------------------------------------------------------------------------------+ > 1: volume test-client-0 > 2: type protocol/client > 3: option remote-host gluster1 > 4: option remote-subvolume /bricks/brick1/brick > 5: option transport-type socket > 6: option send-gids true > 7: end-volume > 8: > 9: volume test-client-1 > 10: type protocol/client > 11: option remote-host gluster2 > 12: option remote-subvolume /bricks/brick1/brick > 13: option transport-type socket > 14: option send-gids true > 15: end-volume > 16: > 17: volume test-replicate-0 > 18: type cluster/replicate > 19: subvolumes test-client-0 test-client-1 > 20: end-volume > 21: > 22: volume test-client-2 > 23: type protocol/client > 24: option remote-host gluster1 > 25: option remote-subvolume /bricks/brick2/brick > 26: option transport-type socket > 27: option send-gids true > 28: end-volume > 29: > 30: volume test-client-3 > 31: type protocol/client > 32: option remote-host gluster2 > 33: option remote-subvolume /bricks/brick2/brick > 34: option transport-type socket > 35: option send-gids true > 36: end-volume > 37: > 38: volume test-replicate-1 > 39: type cluster/replicate > 40: subvolumes test-client-2 test-client-3 > 41: end-volume > 42: > 43: volume test-dht > 44: type cluster/distribute > 45: subvolumes test-replicate-0 test-replicate-1 > 46: end-volume > 47: > 48: volume test-write-behind > 49: type performance/write-behind > 50: subvolumes test-dht > 51: end-volume > 52: > 53: volume test-read-ahead > 54: type performance/read-ahead > 55: subvolumes test-write-behind > 56: end-volume > 57: > 58: volume test-io-cache > 59: type performance/io-cache > 60: subvolumes test-read-ahead > 61: end-volume > 62: > 63: volume test-quick-read > 64: type performance/quick-read > 65: subvolumes test-io-cache > 66: end-volume > 67: > 68: volume test-open-behind > 69: type performance/open-behind > 70: subvolumes test-quick-read > 71: end-volume > 72: > 73: volume test-md-cache > 74: type performance/md-cache > 75: subvolumes test-open-behind > 76: end-volume > 77: > 78: volume test > 79: type debug/io-stats > 80: option latency-measurement off > 81: option count-fop-hits off > 82: subvolumes test-md-cache > 83: end-volume > 84: >+------------------------------------------------------------------------------+ >[2014-08-12 21:04:23.255419] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 1-test-client-0: changing port to 49152 (from 0) >[2014-08-12 21:04:23.258052] E [client-handshake.c:1760:client_query_portmap_cbk] 1-test-client-2: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running. >[2014-08-12 21:04:23.258155] I [client.c:2229:client_rpc_notify] 1-test-client-2: disconnected from 192.168.30.107:24007. Client process will keep trying to connect to glusterd until brick's port is available >[2014-08-12 21:04:23.258173] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 1-test-client-1: changing port to 49152 (from 0) >[2014-08-12 21:04:23.258239] E [client-handshake.c:1760:client_query_portmap_cbk] 1-test-client-3: failed to get the port number for remote subvolume. Please run 'gluster volume status' on server to see if brick process is running. >[2014-08-12 21:04:23.263012] I [client.c:2229:client_rpc_notify] 1-test-client-3: disconnected from 192.168.30.108:24007. Client process will keep trying to connect to glusterd until brick's port is available >[2014-08-12 21:04:23.263026] E [afr-common.c:4168:afr_notify] 1-test-replicate-1: All subvolumes are down. Going offline until atleast one of them comes back up. >[2014-08-12 21:04:23.266198] I [client-handshake.c:1677:select_server_supported_programs] 1-test-client-0: Using Program GlusterFS 3.3, Num (1298437), Version (330) >[2014-08-12 21:04:23.266428] I [client-handshake.c:1677:select_server_supported_programs] 1-test-client-1: Using Program GlusterFS 3.3, Num (1298437), Version (330) >[2014-08-12 21:04:23.266531] I [client-handshake.c:1462:client_setvolume_cbk] 1-test-client-0: Connected to 192.168.30.107:49152, attached to remote volume '/bricks/brick1/brick'. >[2014-08-12 21:04:23.266542] I [client-handshake.c:1474:client_setvolume_cbk] 1-test-client-0: Server and Client lk-version numbers are not same, reopening the fds >[2014-08-12 21:04:23.266611] I [afr-common.c:4131:afr_notify] 1-test-replicate-0: Subvolume 'test-client-0' came back up; going online. >[2014-08-12 21:04:23.266728] I [client-handshake.c:1462:client_setvolume_cbk] 1-test-client-1: Connected to 192.168.30.108:49152, attached to remote volume '/bricks/brick1/brick'. >[2014-08-12 21:04:23.266739] I [client-handshake.c:1474:client_setvolume_cbk] 1-test-client-1: Server and Client lk-version numbers are not same, reopening the fds >[2014-08-12 21:04:23.269708] I [fuse-bridge.c:4977:fuse_graph_setup] 0-fuse: switched to graph 1 >[2014-08-12 21:04:23.269736] I [client-handshake.c:450:client_set_lk_version_cbk] 1-test-client-1: Server lk version = 1 >[2014-08-12 21:04:23.269747] I [client-handshake.c:450:client_set_lk_version_cbk] 1-test-client-0: Server lk version = 1 >[2014-08-12 21:04:23.335548] I [fuse-bridge.c:4189:fuse_migrate_fd_open] 0-glusterfs-fuse: migrated basefd (0x15e72cc) to newfd (0x16b320c) (inode-gfid:e5e936cb-4ad9-4e56-8ace-980b74ce19af)(old-subvolume:test-0 new-subvolume:test-1) >[2014-08-12 21:04:23.335604] I [client.c:2303:notify] 0-test-client-0: current graph is no longer active, destroying rpc_client >[2014-08-12 21:04:23.335653] I [client.c:2303:notify] 0-test-client-1: current graph is no longer active, destroying rpc_client >[2014-08-12 21:04:23.335889] I [client.c:2229:client_rpc_notify] 0-test-client-0: disconnected from 192.168.30.107:49152. Client process will keep trying to connect to glusterd until brick's port is available >[2014-08-12 21:04:23.335932] I [client.c:2229:client_rpc_notify] 0-test-client-1: disconnected from 192.168.30.108:49152. Client process will keep trying to connect to glusterd until brick's port is available >[2014-08-12 21:04:23.335941] E [afr-common.c:4168:afr_notify] 0-test-replicate-0: All subvolumes are down. Going offline until atleast one of them comes back up. >[2014-08-12 21:04:25.263497] W [defaults.c:1381:default_release] (-->/usr/lib64/glusterfs/3.5.2/xlator/protocol/client.so(client3_3_finodelk_cbk+0x153) [0x7f5600da68f3] (-->/usr/lib64/glusterfs/3.5.2/xlator/protocol/client.so(client_local_wipe+0x28) [0x7f5600d9ab88] (-->/usr/lib64/libglusterfs.so.0(fd_unref+0x10e) [0x7f56059f5fbe]))) 0-fuse: xlator does not implement release_cbk >[2014-08-12 21:04:25.274366] I [afr-common.c:4299:afr_local_init] 1-test-replicate-1: no subvolumes up >[2014-08-12 21:04:25.274390] W [dht-diskusage.c:45:dht_du_info_cbk] 1-test-dht: failed to get disk info from test-replicate-1 >[2014-08-12 21:04:25.275285] I [afr-common.c:4299:afr_local_init] 1-test-replicate-1: no subvolumes up >[2014-08-12 21:04:26.789919] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 1-test-client-2: changing port to 49153 (from 0) >[2014-08-12 21:04:26.795465] I [client-handshake.c:1677:select_server_supported_programs] 1-test-client-2: Using Program GlusterFS 3.3, Num (1298437), Version (330) >[2014-08-12 21:04:26.795747] I [rpc-clnt.c:1729:rpc_clnt_reconfig] 1-test-client-3: changing port to 49153 (from 0) >[2014-08-12 21:04:26.798329] I [client-handshake.c:1462:client_setvolume_cbk] 1-test-client-2: Connected to 192.168.30.107:49153, attached to remote volume '/bricks/brick2/brick'. >[2014-08-12 21:04:26.798341] I [client-handshake.c:1474:client_setvolume_cbk] 1-test-client-2: Server and Client lk-version numbers are not same, reopening the fds >[2014-08-12 21:04:26.798462] I [afr-common.c:4131:afr_notify] 1-test-replicate-1: Subvolume 'test-client-2' came back up; going online. >[2014-08-12 21:04:26.798593] I [client-handshake.c:450:client_set_lk_version_cbk] 1-test-client-2: Server lk version = 1 >[2014-08-12 21:04:26.798711] I [client-handshake.c:1677:select_server_supported_programs] 1-test-client-3: Using Program GlusterFS 3.3, Num (1298437), Version (330) >[2014-08-12 21:04:26.802172] I [client-handshake.c:1462:client_setvolume_cbk] 1-test-client-3: Connected to 192.168.30.108:49153, attached to remote volume '/bricks/brick2/brick'. >[2014-08-12 21:04:26.802184] I [client-handshake.c:1474:client_setvolume_cbk] 1-test-client-3: Server and Client lk-version numbers are not same, reopening the fds >[2014-08-12 21:04:26.802378] I [client-handshake.c:450:client_set_lk_version_cbk] 1-test-client-3: Server lk version = 1 >[2014-08-12 21:04:30.216748] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:30.216775] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:32.685662] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:32.685686] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:35.130443] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:35.130467] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:37.576254] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:37.576281] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.034290] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.034314] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.037510] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.037522] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.046247] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.046262] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.054673] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.054688] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.063094] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.063108] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.071334] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.071348] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.079758] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.079773] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.087923] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.087938] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.096348] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.096362] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.104941] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.104954] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.113231] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.113245] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.121508] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.121522] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.130456] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.130470] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.138609] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.138624] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.146921] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.146935] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.154890] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.154904] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.162869] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.162882] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.170836] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.170849] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.178772] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.178786] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.186960] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.186973] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.195313] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.195337] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.203556] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.203569] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.211950] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.211962] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.220952] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.220965] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.229210] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.229222] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.237367] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.237379] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.245510] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.245522] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.253964] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.253976] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.262371] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.262383] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.270526] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.270538] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.278780] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.278793] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.286970] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.286983] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.295227] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.295239] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.303160] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.303173] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.311856] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.311868] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.319751] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.319764] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.327381] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.327394] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.335284] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.335296] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.343179] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.343192] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.351115] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.351165] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.359225] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.359237] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.362432] W [client-rpc-fops.c:322:client3_3_mkdir_cbk] 1-test-client-2: remote operation failed: Permission denied. Path: /14 >[2014-08-12 21:04:40.362532] W [client-rpc-fops.c:322:client3_3_mkdir_cbk] 1-test-client-3: remote operation failed: Permission denied. Path: /14 >[2014-08-12 21:04:40.364074] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:40.364085] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:40.365039] W [client-rpc-fops.c:2690:client3_3_opendir_cbk] 1-test-client-2: remote operation failed: Stale file handle. Path: /14 (ff6dee96-1569-4589-a6b4-5b3abe92d026) >[2014-08-12 21:04:40.365215] W [client-rpc-fops.c:2690:client3_3_opendir_cbk] 1-test-client-3: remote operation failed: Stale file handle. Path: /14 (ff6dee96-1569-4589-a6b4-5b3abe92d026) >[2014-08-12 21:04:40.367817] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:40.367954] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:40.368192] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:40.368536] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:40.368548] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:40.368554] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:42.804349] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:42.804374] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:42.808070] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:42.813595] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:42.813976] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:42.814400] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:42.814412] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:42.814417] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:45.256454] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:45.256478] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:45.260275] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:45.260369] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:45.260707] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:45.261150] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:45.261161] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:45.261167] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:47.699766] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:47.699789] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:47.703337] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:47.703495] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:47.703823] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:47.704270] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:47.704282] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:47.704288] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:50.142598] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:50.142623] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:50.146398] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:50.146532] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:50.146895] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:50.147365] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:50.147377] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:50.147383] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:52.598947] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.598970] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.602269] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.602280] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.611287] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.611299] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.619817] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.619829] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.628295] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.628306] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.636809] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.636822] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.645244] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.645256] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.653370] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.653381] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.661766] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.661778] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.669824] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.669848] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.678246] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.678258] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.686573] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.686584] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.695495] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.695507] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.703891] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.703903] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.712327] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.712339] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.720806] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.720818] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.729338] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.729350] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.737748] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.737759] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.745949] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.745961] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.754271] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.754283] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.762594] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.762606] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.770924] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.770936] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.779203] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.779215] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.787948] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.787960] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.796209] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.796222] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.804538] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.804550] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.813006] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.813018] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.821344] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.821357] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.829791] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.829815] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.838098] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.838110] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.846477] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.846489] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.854973] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.854985] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.863270] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.863282] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.871723] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.871735] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.880495] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.880508] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.889018] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.889054] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.897748] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.897760] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.905797] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.905809] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.913808] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.913821] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.921807] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.921819] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.929986] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.929998] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.938314] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.938326] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.946832] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.946845] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.955276] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.955289] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.963535] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.963547] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.972598] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.972610] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.980923] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.980935] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.989121] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.989170] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:52.997505] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:52.997518] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:53.006056] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:53.006069] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:53.014385] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:53.014397] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:53.022376] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:53.022387] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:53.030470] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:53.030482] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:53.033558] W [client-rpc-fops.c:322:client3_3_mkdir_cbk] 1-test-client-2: remote operation failed: Permission denied. Path: /19 >[2014-08-12 21:04:53.033661] W [client-rpc-fops.c:322:client3_3_mkdir_cbk] 1-test-client-3: remote operation failed: Permission denied. Path: /19 >[2014-08-12 21:04:53.035084] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:53.035095] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:53.036109] W [client-rpc-fops.c:2690:client3_3_opendir_cbk] 1-test-client-2: remote operation failed: Stale file handle. Path: /19 (a5c50e9c-07da-4533-8003-9c0ff96c7919) >[2014-08-12 21:04:53.036130] W [client-rpc-fops.c:2690:client3_3_opendir_cbk] 1-test-client-3: remote operation failed: Stale file handle. Path: /19 (a5c50e9c-07da-4533-8003-9c0ff96c7919) >[2014-08-12 21:04:53.038932] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:53.039001] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:53.039245] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:53.039540] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:53.039552] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:53.039557] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:55.491895] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:55.491921] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:55.495657] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:55.495749] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:55.496030] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:55.496427] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:55.496443] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:55.496451] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:04:58.015086] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:04:58.015121] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:04:58.018816] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:58.018908] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:58.019161] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:04:58.019505] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:04:58.019517] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:04:58.019522] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:00.476057] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:00.476082] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:00.479840] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:00.479927] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:00.480214] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:00.480663] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:00.480675] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:00.480681] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:02.969695] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:02.969724] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:02.972902] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:02.972914] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:02.981407] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:02.981420] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:02.990007] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:02.990019] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:02.998238] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:02.998251] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:03.001573] W [client-rpc-fops.c:322:client3_3_mkdir_cbk] 1-test-client-2: remote operation failed: Permission denied. Path: /20 >[2014-08-12 21:05:03.001699] W [client-rpc-fops.c:322:client3_3_mkdir_cbk] 1-test-client-3: remote operation failed: Permission denied. Path: /20 >[2014-08-12 21:05:03.003114] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:03.003126] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:03.004184] W [client-rpc-fops.c:2690:client3_3_opendir_cbk] 1-test-client-2: remote operation failed: Stale file handle. Path: /20 (f7f6d19d-9191-44fb-8bf5-f76d7780eda9) >[2014-08-12 21:05:03.004202] W [client-rpc-fops.c:2690:client3_3_opendir_cbk] 1-test-client-3: remote operation failed: Stale file handle. Path: /20 (f7f6d19d-9191-44fb-8bf5-f76d7780eda9) >[2014-08-12 21:05:03.007102] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:03.007184] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:03.007448] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:03.007846] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:03.007857] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:03.007863] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:05.512594] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:05.512619] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:05.516294] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:05.516424] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:05.516752] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:05.517266] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:05.517278] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:05.517284] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:07.968322] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:07.968345] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:07.972021] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:07.972173] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:07.973402] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:07.973825] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:07.973837] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:07.973843] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:10.442867] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:10.442891] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:10.446658] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:10.446699] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:10.447082] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:10.447495] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:10.447507] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:10.447513] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:12.890874] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:12.890898] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:12.894622] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:12.894786] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:12.895155] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:12.895582] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:12.895593] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:12.895599] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:15.338535] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:15.338562] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:15.342410] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:15.342452] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:15.342702] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:15.343107] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:15.343119] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:15.343129] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:17.880922] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:17.880946] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:17.884475] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:17.884571] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:17.884856] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:17.885240] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:17.885251] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:17.885257] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed. >[2014-08-12 21:05:20.375505] I [dht-layout.c:727:dht_layout_dir_mismatch] 1-test-dht: / - disk layout missing >[2014-08-12 21:05:20.375532] I [dht-common.c:635:dht_revalidate_cbk] 1-test-dht: mismatching layouts for / >[2014-08-12 21:05:20.379181] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:20.381205] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:20.381530] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-2: remote operation failed: Stale file handle >[2014-08-12 21:05:20.381973] W [client-rpc-fops.c:1632:client3_3_entrylk_cbk] 1-test-client-3: remote operation failed: Stale file handle >[2014-08-12 21:05:20.381985] I [afr-lk-common.c:1092:afr_lock_blocking] 1-test-replicate-1: unable to lock on even one child >[2014-08-12 21:05:20.381991] I [afr-transaction.c:1295:afr_post_blocking_entrylk_cbk] 1-test-replicate-1: Blocking entrylks failed.
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1129486
: 926217 |
926218
|
926219