Bug 810502

Summary: ping_pong application hangs on fuse mounts
Product: [Community] GlusterFS Reporter: Shwetha Panduranga <shwetha.h.panduranga>
Component: locksAssignee: Pranith Kumar K <pkarampu>
Status: CLOSED CURRENTRELEASE QA Contact: Shwetha Panduranga <shwetha.h.panduranga>
Severity: high Docs Contact:
Priority: high    
Version: mainlineCC: amarts, gluster-bugs, rfortier
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.4.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-07-24 17:42:27 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 817967    

Description Shwetha Panduranga 2012-04-06 13:15:23 UTC
Description of problem:

Even after all locks (both blocked and active locks) on the ping_pong file are cleared, ping_pong application doesn't exit but hangs on the fuse mounts. 

Before the clear-locks, the replace-brick operation was also initiated. 

Version-Release number of selected component (if applicable):
3.3.0qa33

How reproducible:
often

Steps to Reproduce:
1.create a distribute-replicate volume(3x3).
2.set auth.allow option to <ip_address_of_client)

[04/06/12 - 21:57:30 root@APP-SERVER1 ~]# gluster volume info
 
Volume Name: dstore
Type: Distributed-Replicate
Volume ID: f69cd573-751f-45dd-b741-4bb9caa7cffc
Status: Started
Number of Bricks: 3 x 3 = 9
Transport-type: tcp
Bricks:
Brick1: 192.168.2.35:/export1/dstore1
Brick2: 192.168.2.36:/export1/dstore1
Brick3: 192.168.2.37:/export1/dstore1
Brick4: 192.168.2.35:/export2/dstore1
Brick5: 192.168.2.36:/export2/dstore1
Brick6: 192.168.2.37:/export2/dstore1
Brick7: 192.168.2.35:/export1/dstore2
Brick8: 192.168.2.36:/export1/dstore2
Brick9: 192.168.2.37:/export1/dstore2
Options Reconfigured:
auth.allow: 192.168.2.34

3.create 1 fuse mount from a machine.

4.start 4-5 instances of ping_pong from the same mount ("/usr/sbin/ping_pong ping_pong_file -rw 100 50 300")

5.gluster volume statedump <volume_name>. check if there are blocked locks on ping_pong file

6.while there are blocked locks on ping_pong file execute:
"gluster volume replace-brick <volume_name> <old_brick> <new_brick> start" 
(select the brick which has ping_pong file to replace)

7.gluster volume clear-locks <volume_name> /ping_pong_file kind blocked inode

8.gluster volume replace-brick <volume_name> <old_brick> <new_brick> commit

9.gluster volume clear-locks <volume_name> /ping_pong_file kind blocked posix

10.gluster volume clear-locks  <volume_name> /ping_pong_file kind all posix

Comment 1 Anand Avati 2012-05-07 07:46:58 UTC
CHANGE: http://review.gluster.com/3221 (cluster/afr: Perform Flush with lk-owner given by parent xlator.) merged in master by Vijay Bellur (vijay)

Comment 2 Anand Avati 2012-05-07 07:48:55 UTC
CHANGE: http://review.gluster.com/3228 (cluster/afr: Fix inodelk-trace logs to print lk-owners) merged in master by Vijay Bellur (vijay)

Comment 3 Shwetha Panduranga 2012-05-14 05:50:17 UTC
This bug still exists on 3.3.0qa41

Comment 4 Anand Avati 2012-05-19 02:04:06 UTC
CHANGE: http://review.gluster.com/3365 (features/locks: insert_and_merge should not operate on blocked locks) merged in master by Anand Avati (avati)

Comment 5 Anand Avati 2012-05-19 02:04:38 UTC
CHANGE: http://review.gluster.com/3366 (features/locks: Don't delete blocked locks in pl_flush) merged in master by Anand Avati (avati)

Comment 6 Shwetha Panduranga 2012-05-24 12:02:48 UTC
Bug is fixed. Verified on 3.3.0qa43