Bug 1473968 - rda-cache-limit filled with (null) value after use parallel-readdir
rda-cache-limit filled with (null) value after use parallel-readdir
Status: NEW
Product: GlusterFS
Classification: Community
Component: glusterd (Show other bugs)
3.10
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: bugs@gluster.org
: Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2017-07-22 16:26 EDT by Vitaly Lipatov
Modified: 2017-11-29 18:53 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Vitaly Lipatov 2017-07-22 16:26:48 EDT
Description of problem:

Any operations failed with rda-cache-limit related error:
# gluster volume set ftp-pvt performance.rda-cache-limit 10M
volume set: failed: invalid number format "5242880(null)" in option "rda-cache-limit"

Version-Release number of selected component (if applicable):
3.10.1, 3.10.3

How reproducible:

I did on 3.10.1
# gluster volume set ftp-pub performance.parallel-readdir on
volume set: success

After it I can't disable it:
# gluster volume set ftp-pub performance.parallel-readdir off
volume set: failed: Staging failed on euclid. Error: invalid number format "5242880(null)" in option "rda-cache-limit"
Staging failed on asu. Error: invalid number format "5242880(null)" in option "rda-cache-limit"
Staging failed on nun. Error: invalid number format "5242880(null)" in option "rda-cache-limit"
Staging failed on cellar. Error: invalid number format "5242880(null)" in option "rda-cache-limit"
Staging failed on multi. Error: invalid number format "5242880(null)" in option "rda-cache-limit"
Staging failed on server. Error: invalid number format "5242880(null)" in option "rda-cache-limit"


Actual results:
files like /var/lib/glusterd/vols/ftp-pub/trusted-ftp-pub.tcp-fuse.vol contains

volume ftp-pvt-readdir-ahead-0
    type performance/readdir-ahead
    option rda-cache-limit 5242880(null)
    option rda-request-size 131072
    option parallel-readdir on
    subvolumes ftp-pvt-replicate-0
end-volume



Expected results:
correct value (without (null))
    option rda-cache-limit 5242880


Additional info:
Comment 1 Atin Mukherjee 2017-07-24 00:54:43 EDT
I believe https://review.gluster.org/17338 fixes this problem.

Poornima - can you please confirm?
Comment 2 Poornima G 2017-07-24 01:04:27 EDT
Yes, this bug is fixed. Could you please try it on 3.11.X latest version?
Comment 3 Vitaly Lipatov 2017-07-24 10:30:55 EDT
I can test 3.10.4 with patch from https://review.gluster.org/17338
but it seems there are no fixes against strange (null) string after number in
in volume info like option rda-cache-limit 5242880(null)
Comment 4 Vitaly Lipatov 2017-07-26 16:58:25 EDT
That patch not about this problem and did not fix it (I have adopted it for 3.10.4 and tried with it).

The problem is left after I ran 
# find /var/lib/glusterd/vols -type f | xargs -n1 subst "s|readdir\([= ]\)on|readdir\1off|g"
on all gluster's servers and restart it.

After it I got
# gluster volume set ftp-pub performance.rda-cache-limit 5MB
volume set: success
Comment 5 Atin Mukherjee 2017-08-08 11:44:35 EDT
Poornima - any comments?
Comment 6 Poornima G 2017-08-09 01:08:07 EDT
I think in 3.10 parallel readdir was only experimental feature, after 3.10 lots of fixes have gone in parallel readdir. Could you please check if the bug is seen on 3.11? If you are doing source install on a test machine, 3.12 will also be a good candidate to test with. Will be interested in knowing if its seen on 3.12/11?
Comment 7 Vitaly Lipatov 2017-11-29 18:46:41 EST
I just got the same with 3.12.3 after
# gluster volume set ftp-pvt performance.parallel-readdir on


[root@server ftp]# gluster volume set ftp-pvt server.root-squash on
volume set: failed: invalid number format "2621440(null)" in option "rda-cache-limit"

[root@server ftp]# gluster volume get ftp-pvt rda-cache-limit
Option                                  Value                                   
------                                  -----                                   
performance.rda-cache-limit             5MB   

# gluster volume set ftp-pvt rda-cache-limit 10MB
volume set: failed: invalid number format "5242880(null)" in option "rda-cache-limit"
Comment 8 Vitaly Lipatov 2017-11-29 18:53:14 EST
It seem, the problem near 32 bit system.

vol file on 32 bit host in cluster:
# grep -A5 readdir /var/lib/glusterd/vols/ftp-pvt/trusted-ftp-pvt.tcp-fuse.vol
volume ftp-pvt-readdir-ahead-0
    type performance/readdir-ahead
    option rda-cache-limit 2621440(null)
    option rda-request-size 131072
    option parallel-readdir on
    subvolumes ftp-pvt-replicate-0
end-volume

vol file on 64 bit host in cluster:
# grep -A5 readdir /var/lib/glusterd/vols/ftp-pvt/trusted-ftp-pvt.tcp-fuse.vol
volume ftp-pvt-readdir-ahead-0
    type performance/readdir-ahead
    option rda-cache-limit 2621440B
    option rda-request-size 131072
    option parallel-readdir on
    subvolumes ftp-pvt-replicate-0
end-volume

Note You need to log in before you can comment on or make changes to this bug.