Bug 1499520
Summary: | Geo-rep: Lot of changelog(rsync) retries and failure during directory syncing. | |||
---|---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Kotresh HR <khiremat> | |
Component: | distribute | Assignee: | Mohit Agrawal <moagrawa> | |
Status: | CLOSED UPSTREAM | QA Contact: | ||
Severity: | high | Docs Contact: | ||
Priority: | high | |||
Version: | mainline | CC: | bugs, khiremat, moagrawa, vfarias | |
Target Milestone: | --- | Flags: | khiremat:
needinfo-
|
|
Target Release: | --- | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | If docs needed, set a value | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1600671 (view as bug list) | Environment: | ||
Last Closed: | 2020-03-12 12:37:31 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1510012, 1565577, 1571069, 1576767, 1580215, 1600671 |
Description
Kotresh HR
2017-10-08 11:23:09 UTC
> [2017-10-08 11:12:31.687599] I [dict.c:471:dict_get] (-->/usr/local/lib/glusterfs/4.0dev/xlator/cluster/distribute.so(+0x28c31) [0x7f4af9601c31] -->/usr/local/lib/glusterfs/4.0dev/xlator/cluster/distribute.so(+0x28448) [0x7f4af9601448] -->/usr/local/lib/libglusterfs.so.0(dict_get+0x79) [0x7f4b0765662d] ) 0-dict: !this || key=trusted.glusterfs.dht.mds [Invalid argument]
I think we are not creating "trusted.glusterfs.dht.mds" in discover codepath. Note that dht in geo-rep slave mounts receives only nameless lookup. We do create mds xattr in nameless lookup codepath, but only if directory is missing on one of the subvolumes. I guess in this case directory is present on all subvols, but only mds xattr is missing.
@Kotresh,
The above hypothesis is only true for those directories that were created before the patch in question is merged. If a directory is created _after_ upgrading, you shouldn't see these errors. Can you please check whether this issue is seen for directories that were created after upgrade is done?
(In reply to Raghavendra G from comment #1) > @Kotresh, > > The above hypothesis is only true for those directories that were created > before the patch in question is merged. If a directory is created _after_ > upgrading, you shouldn't see these errors. Can you please check whether this > issue is seen for directories that were created after upgrade is done? Retries are happening for the new directory creations as well. As per slave logs in #c1 it seems xattr is NULL on local frame,I think because xattr is NULL so retries are happening. (In reply to Mohit Agrawal from comment #3) > As per slave logs in #c1 it seems xattr is NULL on local frame,I think > because xattr is NULL so retries are happening. The retries are happening in normal use case for every directory being synced. It needs to be analysed and should be fixed. Could you take a look at this further? -Kotresh HR Hi Mohit, Even with the patch https://review.gluster.org/#/c/18436/, the issue is reproducible. Please take a look at it. I have made this a blocker for 3.13.0 release. Thanks, Kotresh HR This bug is moved to https://github.com/gluster/glusterfs/issues/914, and will be tracked there from now on. Visit GitHub issues URL for further details |