Bug 1545570 - DHT calls dht_lookup_everywhere for 1xn volumes
Summary: DHT calls dht_lookup_everywhere for 1xn volumes
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: distribute
Version: rhgs-3.4
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: RHGS 3.4.0
Assignee: Nithya Balachandran
QA Contact: Prasad Desala
URL:
Whiteboard:
Depends On: 1546620 1548270 1548271
Blocks: 1503137
TreeView+ depends on / blocked
 
Reported: 2018-02-15 09:58 UTC by Nithya Balachandran
Modified: 2018-09-17 09:53 UTC (History)
3 users (show)

Fixed In Version: glusterfs-3.12.2-5
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1546620 (view as bug list)
Environment:
Last Closed: 2018-09-04 06:42:41 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2607 None None None 2018-09-04 06:43:59 UTC

Description Nithya Balachandran 2018-02-15 09:58:23 UTC
Description of problem:

DHT lookup does not handle lookups on 1xn volumes optimally. It calls a dht_lookup_everywhere even though it has already checked the only subvol leading to repeated, redundant lookups for entries.

Version-Release number of selected component (if applicable):


How reproducible:
Consistently

Steps to Reproduce:
1. Create a 1x3 volume and set client-log-level to DEBUG
2. Fuse mount the volume and create some files (touch file-{1..10})
3. Check the mnt log for dht messages

Actual results:
[2018-02-15 09:53:04.799711] D [MSGID: 0] [dht-common.c:2761:dht_lookup] 0-gitmo-dht: Calling fresh lookup for /file-10 on gitmo-replicate-0
[2018-02-15 09:53:04.800179] D [MSGID: 0] [dht-common.c:2344:dht_lookup_cbk] 0-gitmo-dht: fresh_lookup returned for /file-10 with op_ret -1 [No such file or directory]
[2018-02-15 09:53:04.800188] D [MSGID: 0] [dht-common.c:2357:dht_lookup_cbk] 0-gitmo-dht: Entry /file-10 missing on subvol gitmo-replicate-0
[2018-02-15 09:53:04.800196] D [MSGID: 0] [dht-common.c:2128:dht_lookup_everywhere] 0-gitmo-dht: winding lookup call to 1 subvols
[2018-02-15 09:53:04.800670] D [MSGID: 0] [dht-common.c:1930:dht_lookup_everywhere_cbk] 0-gitmo-dht: returned with op_ret -1 and op_errno 2 (/file-10) from subvol gitmo-replicate-0
[2018-02-15 09:53:04.800679] D [MSGID: 0] [dht-common.c:1594:dht_lookup_everywhere_done] 0-gitmo-dht: STATUS: hashed_subvol gitmo-replicate-0 cached_subvol null
[2018-02-15 09:53:04.800684] D [MSGID: 0] [dht-common.c:1655:dht_lookup_everywhere_done] 0-gitmo-dht: There was no cached file and  unlink on hashed is not skipped /file-10
[2018-02-15 09:53:04.800691] D [MSGID: 0] [dht-common.c:1658:dht_lookup_everywhere_done] 0-stack-trace: stack-address: 0x7f03e4000d50, gitmo-dht returned -1 error: No such file or directory [No such file or directory]


Expected results:

DHT need not call dht_lookup_everywhere for 1xn volumes

Additional info:

Comment 5 Nithya Balachandran 2018-02-23 09:24:15 UTC
This patch is very specific to lookups on files that do not exist (data or linkto). It does not handle directories or linkto files.

Comment 7 Prasad Desala 2018-04-16 10:39:31 UTC
Verified this BZ on glusterfs version 3.12.2-7.el7rhgs.x86_64. 
Now, dht_lookup_everywhere is not called for 1xn volumes.

[2018-04-16 10:25:11.829200] D [MSGID: 0] [dht-common.c:3419:dht_lookup] 0-1x3-dht: Calling fresh lookup for /file-10 on 1x3-replicate-0
[2018-04-16 10:25:11.831602] D [MSGID: 0] [dht-common.c:2972:dht_lookup_cbk] 0-1x3-dht: fresh_lookup returned for /file-10 with op_ret -1 [No such file or directory]
[2018-04-16 10:25:11.831918] D [fuse-resolve.c:61:fuse_resolve_entry_cbk] 0-fuse: 00000000-0000-0000-0000-000000000001/file-10: failed to resolve (No such file or directory)
[2018-04-16 10:25:11.832104] D [MSGID: 0] [dht-common.c:3419:dht_lookup] 0-1x3-dht: Calling fresh lookup for /file-10 on 1x3-replicate-0
[2018-04-16 10:25:11.834616] D [MSGID: 0] [dht-common.c:2972:dht_lookup_cbk] 0-1x3-dht: fresh_lookup returned for /file-10 with op_ret -1 [No such file or directory]
[2018-04-16 10:25:11.835342] D [MSGID: 0] [dht-common.c:3419:dht_lookup] 0-1x3-dht: Calling fresh lookup for /file-10 on 1x3-replicate-0
[2018-04-16 10:25:11.837554] D [MSGID: 0] [dht-common.c:2972:dht_lookup_cbk] 0-1x3-dht: fresh_lookup returned for /file-10 with op_ret -1 [No such file or directory]
[2018-04-16 10:25:11.837859] D [fuse-resolve.c:61:fuse_resolve_entry_cbk] 0-fuse: 00000000-0000-0000-0000-000000000001/file-10: failed to resolve (No such file or directory)
[2018-04-16 10:25:11.837961] D [MSGID: 0] [dht-common.c:8074:dht_create_wind_to_avail_subvol] 0-1x3-dht: creating /file-10 on 1x3-replicate-0
[2018-04-16 10:25:11.854286] D [fuse-bridge.c:1359:fuse_removexattr_cbk] 0-glusterfs-fuse: 87: REMOVEXATTR() of security.ima on /file-10 => -1 (No data available)

Moving this BZ to Verified.

Comment 9 errata-xmlrpc 2018-09-04 06:42:41 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2607


Note You need to log in before you can comment on or make changes to this bug.