Bug 1596040 - eagerlock seems to lose effect when the file being written is also being read
Summary: eagerlock seems to lose effect when the file being written is also being read
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: replicate
Version: rhgs-3.3
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: RHGS 3.4.z Batch Update 3
Assignee: Ravishankar N
QA Contact: Anees Patel
URL:
Whiteboard:
Depends On: 1630688
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-28 06:40 UTC by Nag Pavan Chilakam
Modified: 2019-02-04 07:41 UTC (History)
10 users (show)

Fixed In Version: glusterfs-3.12.2-33
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-02-04 07:41:25 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2019:0263 None None None 2019-02-04 07:41:37 UTC

Description Nag Pavan Chilakam 2018-06-28 06:40:22 UTC
Description of problem:
=========================
If a file being written to is being read from another client, I see that finodelk count is increasing.

read doesn't require locks in afr and also if we read a file (no other operation on this file), we don't see any finodelk in profiling

 eager locking is lost if we read a file being written to, which may need to be improvised


Version-Release number of selected component (if applicable):
-----------
3.8.4-54.12

How reproducible:
==============
always

Steps to Reproduce:
1.create a 1x3 volume enable profiling 
2. mount vol on 2 clients
3. write to a file using dd say " dd if=/dev/urandom of=prof bs=1024 count=1000000"
4. from another client do a md5sum on same file



Actual results:
-----------
finodelk count would be huge

Expected results:
----------
findoelk count should be almost zero

Additional info:
----------------
FYI(may not be useful)
you can even check that dirty bit is incrementing in a  rangebound  sequence and then being reset ie it increase from about 0-5 and then again goes back to zero

Comment 10 Ravishankar N 2018-09-19 06:45:27 UTC
1630688 makes eager locking for data transactions independent of fd count, which should fix this BZ. Hence marking this as dependent on it.

Comment 11 Atin Mukherjee 2018-11-11 19:40:26 UTC
So isn't this a good candidate for qualification for 3.4.3 considering the content is already part of the latest release?

Comment 12 Ravishankar N 2018-11-15 05:11:31 UTC
(In reply to Atin Mukherjee from comment #11)
> So isn't this a good candidate for qualification for 3.4.3 considering the
> content is already part of the latest release?

Agree. Nag, Irrespective of the broader changes mentioned in comment#5 and 7, Pranith's eager lock patch does fix this particular bug. i.e. eager lock won't be lost when we read a file that is being written to simultaneously. Please provide ack for 3.4.3 if there is agreement to test it.

Comment 13 Nag Pavan Chilakam 2018-12-03 08:41:45 UTC
(In reply to Ravishankar N from comment #12)
> (In reply to Atin Mukherjee from comment #11)
> > So isn't this a good candidate for qualification for 3.4.3 considering the
> > content is already part of the latest release?
> 
> Agree. Nag, Irrespective of the broader changes mentioned in comment#5 and
> 7, Pranith's eager lock patch does fix this particular bug. i.e. eager lock
> won't be lost when we read a file that is being written to simultaneously.
> Please provide ack for 3.4.3 if there is agreement to test it.

I believe this can be taken in 3.4.3 only after assessing other bugs and timelines.
Kindly go ahead and propose for 3.4.3 from dev side, and then QE would look into this if it is possible to take in 3.4.3 or not

Comment 16 Sunil Kumar Acharya 2018-12-17 09:37:13 UTC
Fix is already part of downstream code as mentioned in comment #10

Comment 19 Anees Patel 2018-12-26 05:21:57 UTC
Hi Ravi,

I have created a test-plan to verify the fix,
https://docs.google.com/spreadsheets/d/1bX1EyPNnVUU_3PJa-HfvzJe4stMWQix7G02YSSPAROE/edit?usp=sharing
Kindly go thru the test-plan and please provide your feedback

Comment 20 Anees Patel 2018-12-26 05:23:07 UTC
Setting need_info on Ravi for the above comment

Comment 21 Ravishankar N 2018-12-27 12:09:07 UTC
Hi Anees, The first 3 tests look fine. The 4th one seems to be multiple writers to the same file and no reads, which does not seem to relate to this specific bug. You might experience more inodelks/xattrops with multiple writers than with single writer.

Comment 22 Anees Patel 2018-12-28 10:23:44 UTC
Hi Ravi,

Thank-you for the feed-back,

I verified the fix on the latest build
# rpm -qa | grep gluster
gluster-nagios-common-0.2.4-1.el7rhgs.noarch
glusterfs-rdma-3.12.2-34.el7rhgs.x86_64
glusterfs-server-3.12.2-34.el7rhgs.x86_64
glusterfs-client-xlators-3.12.2-34.el7rhgs.x86_64
glusterfs-fuse-3.12.2-34.el7rhgs.x86_64
glusterfs-events-3.12.2-34.el7rhgs.x86_64
glusterfs-devel-3.12.2-34.el7rhgs.x86_64


finodelks were well close to one or two with the above test-cases.
The max count that I was able to get with single writer was 3.

 %-latency   Avg-latency   Min-Latency   Max-Latency   No. of calls         Fop
 ---------   -----------   -----------   -----------   ------------        ----
      0.00       0.00 us       0.00 us       0.00 us              2     RELEASE
      0.00       0.00 us       0.00 us       0.00 us              8  RELEASEDIR
      0.00      36.00 us      30.00 us      42.00 us              2       FLUSH
      0.00     181.00 us     181.00 us     181.00 us              1        OPEN
      0.00     101.50 us      70.00 us     133.00 us              2     INODELK
      0.00      72.67 us      53.00 us     108.00 us              3    FINODELK
      0.00     180.67 us     140.00 us     209.00 us              3    FXATTROP
      0.00     308.50 us     281.00 us     336.00 us              2     XATTROP
      0.00     317.00 us     282.00 us     352.00 us              2      RENAME
      0.00      86.88 us      38.00 us     155.00 us              8     ENTRYLK
      0.00     142.12 us     103.00 us     204.00 us              8     OPENDIR
      0.02     175.77 us      81.00 us     477.00 us             26      LOOKUP
      0.34  101170.00 us  101170.00 us  101170.00 us              1    TRUNCATE
     99.63     168.52 us      67.00 us 1083136.00 us         176836       WRITE


Hence setting this bug to verified

Comment 24 errata-xmlrpc 2019-02-04 07:41:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0263


Note You need to log in before you can comment on or make changes to this bug.