Bug 800381

Summary: [228d01916c57d5a5716e1097e39e7aa06f31f3e4] Read is hung while doing search and replace with perl script
Product: [Community] GlusterFS Reporter: Anush Shetty <ashetty>
Component: distributeAssignee: shishir gowda <sgowda>
Status: CLOSED DUPLICATE QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: mainlineCC: amarts, gluster-bugs, nsathyan
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-03-29 07:21:41 UTC Type: ---
Regression: --- Mount Type: fuse
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Statedump of the backend none

Description Anush Shetty 2012-03-06 12:33:21 UTC
Created attachment 567935 [details]
Statedump of the backend

Description of problem: While using the perl script to do do search and replace replace, the script hung at the read call. This was only seen on distribute volumes with quick-read on. 

How reproducible: Consistently


Steps to Reproduce:
1. echo 'test' > dot
2. perl -i -pe 's/test/somethingelse/' dot
  
Actual results:

The script was hung at the read call. 


Additional info:

Attaching the statedump of the bricks.

Client log-
[2012-03-06 17:40:24.340042] D [dht-diskusage.c:88:dht_du_info_cbk] 1-test2-dht: on subvolume 'test2-client-0': avail_percent is: 94.00 and avail_space is: 96749436928 and avail_inodes is: 99.00
[2012-03-06 17:40:24.340124] D [dht-diskusage.c:88:dht_du_info_cbk] 1-test2-dht: on subvolume 'test2-client-1': avail_percent is: 94.00 and avail_space is: 96749436928 and avail_inodes is: 99.00
[2012-03-06 17:40:26.804931] W [client.c:112:client_grace_timeout] 0-test2-client-0: client grace timer expired, updating the lk-version to 2
[2012-03-06 17:40:26.805034] W [client.c:112:client_grace_timeout] 0-test2-client-1: client grace timer expired, updating the lk-version to 2
[2012-03-06 17:40:27.051189] D [dht-diskusage.c:88:dht_du_info_cbk] 1-test2-dht: on subvolume 'test2-client-1': avail_percent is: 94.00 and avail_space is: 96749436928 and avail_inodes is: 99.00
[2012-03-06 17:40:27.051255] D [dht-diskusage.c:88:dht_du_info_cbk] 1-test2-dht: on subvolume 'test2-client-0': avail_percent is: 94.00 and avail_space is: 96749436928 and avail_inodes is: 99.00
[2012-03-06 17:40:27.052061] W [client3_1-fops.c:375:client3_1_open_cbk] 1-test2-client-0: remote operation failed: No such file or directory. Path: /dot
[2012-03-06 17:40:27.052437] W [client3_1-fops.c:880:client3_1_getxattr_cbk] 1-test2-client-0: remote operation failed: No such file or directory. Path: (null)
[2012-03-06 17:40:27.052665] W [client3_1-fops.c:2180:client3_1_lookup_cbk] 1-test2-client-0: remote operation failed: Invalid argument. Path: /dot
[2012-03-06 17:40:27.052694] D [dht-common.c:269:dht_discover_cbk] 1-test2-dht: lookup of /dot on test2-client-0 returned error (Invalid argument)
[2012-03-06 17:40:27.052797] W [client3_1-fops.c:2180:client3_1_lookup_cbk] 1-test2-client-1: remote operation failed: Invalid argument. Path: /dot
[2012-03-06 17:40:27.052820] D [dht-common.c:269:dht_discover_cbk] 1-test2-dht: lookup of /dot on test2-client-1 returned error (Invalid argument)
[2012-03-06 17:40:27.052841] I [dht-layout.c:600:dht_layout_normalize] 1-test2-dht: found anomalies in /dot. holes=1 overlaps=0
[2012-03-06 17:40:27.052862] D [dht-layout.c:614:dht_layout_normalize] 1-test2-dht: path=/dot err=Invalid argument on subvol=test2-client-0
[2012-03-06 17:40:27.052881] D [dht-layout.c:614:dht_layout_normalize] 1-test2-dht: path=/dot err=Invalid argument on subvol=test2-client-1
[2012-03-06 17:40:27.052900] D [dht-common.c:197:dht_discover_complete] 1-test2-dht: normalizing failed on /dot

Comment 1 Amar Tumballi 2012-03-08 06:41:22 UTC
Shishir,

I guess some of the outstanding patches from Raghu (Du) should already fix it. http://review.gluster.com/#dashboard,1000007 (in quick-read)

Please try to review it and see for anything obvious.

Avati,
Can you send some thoughts on utilizing anon-fd for quick-read, which we believe should reduce the complexity involved.

Comment 2 shishir gowda 2012-03-08 08:23:20 UTC
This seems to be a dup of bug 765363.
http://review.gluster.com/#change,506,patchset=6 patch fixes this issue

Comment 3 shishir gowda 2012-03-29 07:21:41 UTC

*** This bug has been marked as a duplicate of bug 765363 ***