Hide Forgot
Description of problem: After committing replace-brick operation, any operation on the mount point returned ENOTCONN Version-Release number of selected component (if applicable): Upstream How reproducible: Consistently Steps to Reproduce: 1. start replace-brick 2. commit replace-brick 3. ls /mnt/gluster Actual results: Return ENOTCONN Expected results: Shouldn't error out Additional info: Client log: [2012-04-04 12:43:27.444919] D [glusterfsd-mgmt.c:1608:mgmt_getspec_cbk] 0-glusterfsd-mgmt: No need to re-load volfile, reconfigure done [2012-04-04 12:43:27.444956] I [glusterfsd-mgmt.c:1584:mgmt_getspec_cbk] 0-glusterfs: No change in volfile, continuing [2012-04-04 12:43:29.582652] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:29.582715] W [client3_1-fops.c:2525:client3_1_opendir_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: / [2012-04-04 12:43:29.582742] W [fuse-bridge.c:694:fuse_fd_cbk] 0-glusterfs-fuse: 54: OPENDIR() / => -1 (Transport endpoint is not connected) [2012-04-04 12:43:30.160428] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:30.160485] W [client3_1-fops.c:2525:client3_1_opendir_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: / [2012-04-04 12:43:30.160511] W [fuse-bridge.c:694:fuse_fd_cbk] 0-glusterfs-fuse: 55: OPENDIR() / => -1 (Transport endpoint is not connected) [2012-04-04 12:43:30.571687] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:30.571746] W [client3_1-fops.c:2525:client3_1_opendir_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: / [2012-04-04 12:43:30.571772] W [fuse-bridge.c:694:fuse_fd_cbk] 0-glusterfs-fuse: 56: OPENDIR() / => -1 (Transport endpoint is not connected) [2012-04-04 12:43:32.248989] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:32.249069] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: /dot [2012-04-04 12:43:32.249155] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:32.249178] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: /dot [2012-04-04 12:43:32.249203] W [fuse-bridge.c:272:fuse_entry_cbk] 0-glusterfs-fuse: 57: LOOKUP() /dot => -1 (Transport endpoint is not connected) [2012-04-04 12:43:32.249349] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:32.249374] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: /dot [2012-04-04 12:43:32.249439] D [client.c:189:client_submit_request] 0-test2-client-0: connection in disconnected state [2012-04-04 12:43:32.249460] W [client3_1-fops.c:2607:client3_1_lookup_cbk] 0-test2-client-0: remote operation failed: Transport endpoint is not connected. Path: /dot [2012-04-04 12:43:32.249484] W [fuse-bridge.c:272:fuse_entry_cbk] 0-glusterfs-fuse: 58: LOOKUP() /dot => -1 (Transport endpoint is not connected)
Works for me in dc4c44d51e3eb4b53e78840bf223911e48ce62b5.
Hi KP, I still see this issue with 7ef32ae76d1c1e4a5ff47899d175be9fdeb73bc8 Reopening it.
Anush, Could you provide more information, like volume configuration (volume info)? It would be helpful if you could attach all logs (client, bricks and glusterd) when the problem is observed.
The reason the lookup on the mount point fails for a short period of time after replace-brick commit, is because the commit operation restarts the destination brick. This 'disconnection' is noticed by the application only when you perform replace-brick on a 'single-brick volume'. This is a known limitation of replace-brick.