Bug 1333023
Summary: | readdir-ahead does not fetch xattrs that md-cache needs in it's internal calls | ||||||
---|---|---|---|---|---|---|---|
Product: | [Community] GlusterFS | Reporter: | Prashanth Pai <ppai> | ||||
Component: | md-cache | Assignee: | Prashanth Pai <ppai> | ||||
Status: | CLOSED CURRENTRELEASE | QA Contact: | |||||
Severity: | high | Docs Contact: | |||||
Priority: | high | ||||||
Version: | mainline | CC: | bugs, mzywusko, nlevinki, ppai, rcyriac, rhinduja, rhs-bugs, sbhaloth, storage-qa-internal, vbellur | ||||
Target Milestone: | --- | Keywords: | Triaged | ||||
Target Release: | --- | ||||||
Hardware: | All | ||||||
OS: | Linux | ||||||
Whiteboard: | |||||||
Fixed In Version: | Doc Type: | Bug Fix | |||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | 1331260 | ||||||
: | 1334699 1334700 (view as bug list) | Environment: | |||||
Last Closed: | 2016-11-22 13:03:51 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | v3.9.0 | Category: | --- | ||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Bug Depends On: | |||||||
Bug Blocks: | 1331260, 1334699, 1334700 | ||||||
Attachments: |
|
Comment 1
Prashanth Pai
2016-05-04 14:30:17 UTC
Created attachment 1153898 [details]
tcpdump of issue
Notice that in tcpdump traces, readdirp() calls do not contain "user.swift.metadata" in xdata.
REVIEW: http://review.gluster.org/14213 (md-cache: Don't cache swift metadata by default) posted (#1) for review on master by Prashanth Pai (ppai) REVIEW: http://review.gluster.org/14214 (readdir-ahead: Prefetch xattrs needed by md-cache) posted (#1) for review on master by Prashanth Pai (ppai) REVIEW: http://review.gluster.org/14213 (md-cache: Don't cache swift metadata by default) posted (#2) for review on master by Prashanth Pai (ppai) REVIEW: http://review.gluster.org/14214 (readdir-ahead: Prefetch xattrs needed by md-cache) posted (#2) for review on master by Prashanth Pai (ppai) REVIEW: http://review.gluster.org/14213 (md-cache: Don't cache swift metadata by default) posted (#3) for review on master by Prashanth Pai (ppai) REVIEW: http://review.gluster.org/14214 (readdir-ahead: Prefetch xattrs needed by md-cache) posted (#3) for review on master by Prashanth Pai (ppai) COMMIT: http://review.gluster.org/14214 committed in master by Raghavendra G (rgowdapp) ------ commit 0c73e7050c4d30ace0c39cc9b9634e9c1b448cfb Author: Prashanth Pai <ppai> Date: Wed May 4 16:56:50 2016 +0530 readdir-ahead: Prefetch xattrs needed by md-cache Problem: Negative cache feature implementation in md-cache requires xattrs returned by posix to be intercepted for every call that can possibly return xattrs. This includes readdirp(). This is crucial to treat missing keys in cache as a case of negative entry (returns ENODATA) md-cache puts names of xattrs that it wants to cache in xdata and passes it down to posix which returns the specified xattrs in the callback. This is done in lookup() and readdirp(). Hence, a xattr that is cached can be invalidated during readdirp_cbk too. This is based on the assumption that readdirp() will always return all xattrs that md-cache is interested in. However, this is not the case when readdirp() call is served from readdir-ahead's cache. readdir-ahead xlator will pre-fetch dentries during opendir_cbk and readdirp. These internal readdirp() calls made by readdir-ahead xlator does not set xdata in it's requests. Hence, no xattrs are fetched and stored in it's internal cache. This causes metadata loss in gluster-swift. md-cache returns ENODATA during getxattr() call even though the xattr for that object exists on the brick. On receiving ENODATA, gluster-swift will create new metadata and do setxattr(). This results in loss of information stored in existing xattr. Fix: During opendir, md-cache will communicate to readdir-ahead asking it to store the names of xattrs it's interested in so that readdir-ahead can fetch those in all subsequent internal readdirp() calls issued by it. This stored names of xattrs is invalidated/updated on the next real readdirp() call issued by application. This readdirp() call will have xdata set correctly by md-cache xlator. BUG: 1333023 Change-Id: I32d46f93a99d4ec34c741f3c52b0646d141614f9 Reviewed-on: http://review.gluster.org/14214 Tested-by: Prashanth Pai <ppai> NetBSD-regression: NetBSD Build System <jenkins.org> CentOS-regression: Gluster Build System <jenkins.com> Tested-by: Gluster Build System <jenkins.com> Smoke: Gluster Build System <jenkins.com> Reviewed-by: Raghavendra G <rgowdapp> Tested-by: Raghavendra G <rgowdapp> |