Bug 1317788

Summary:	Cache swift xattrs
Product:	[Community] GlusterFS	Reporter:	Prashanth Pai <ppai>
Component:	md-cache	Assignee:	Prashanth Pai <ppai>
Status:	CLOSED CURRENTRELEASE	QA Contact:
Severity:	low	Docs Contact:
Priority:	urgent
Version:	3.7.8	CC:	bugs, sasundar
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:	glusterfs-3.7.10	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:	1317785
Clones:	1317790 (view as bug list)		Environment:
Last Closed:	2016-04-11 06:07:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1317785
Bug Blocks:	1317790, 1318376

Description Prashanth Pai 2016-03-15 08:35:57 UTC

+++ This bug was initially created as a clone of Bug #1317785 +++

Description of problem:

gluster-swift extensively uses "user.swift.metadata" xattr in it's functionality. This is not cached and hence the performance is usually terrible.

Every operation (GET,PUT,POST,DELETE,HEAD) in gluster-swift issues getxattr() as follows:

getxattr("/mnt/gluster-object/test/c1/o8", "user.swift.metadata", 0x0, 0) = 190
getxattr("/mnt/gluster-object/test/c1/o8", "user.swift.metadata", "{"Content-Length":"11","ETag":"5eb63bbbe01eeed093cb22bb8f5acdc3","X-Timestamp":"1458021187.45380","X-Object-Type":"file","X-Type":"Object","Content-Type":"application/x-www-form-urlencoded"}", 190) = 190

The first getxattr() is to get the size of xattr value and the second getxattr() is to fetch the actual value itself.

The entire xattr information is sent thrice by brick to FUSE mount. Once on lookup, again on the first getxattr() to get size and then again on second gexattr() call. These three network calls can be reduced to just one if the xattr information is cached when it's fetched for the first time.

--- Additional comment from Prashanth Pai on 2016-03-15 04:34 EDT ---

Attached tcpdump trace of DELETE object operation from gluster-swift

Comment 1 Vijay Bellur 2016-03-16 12:55:52 UTC

REVIEW: http://review.gluster.org/13753 (md-cache: Cache gluster-swift metadata) posted (#1) for review on release-3.7 by Prashanth Pai (ppai)

Comment 2 Vijay Bellur 2016-03-16 16:55:54 UTC

REVIEW: http://review.gluster.org/13753 (md-cache: Cache gluster-swift metadata) posted (#2) for review on release-3.7 by Niels de Vos (ndevos)

Comment 3 Vijay Bellur 2016-03-17 04:26:49 UTC

REVIEW: http://review.gluster.org/13753 (md-cache: Cache gluster-swift metadata) posted (#3) for review on release-3.7 by Prashanth Pai (ppai)

Comment 4 Vijay Bellur 2016-03-28 05:26:33 UTC

REVIEW: http://review.gluster.org/13753 (md-cache: Cache gluster-swift metadata) posted (#4) for review on release-3.7 by Prashanth Pai (ppai)

Comment 5 Mike McCune 2016-03-28 23:21:47 UTC

This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions

Comment 6 Vijay Bellur 2016-03-30 10:38:44 UTC

REVIEW: http://review.gluster.org/13753 (md-cache: Cache gluster-swift metadata) posted (#5) for review on release-3.7 by Niels de Vos (ndevos)

Comment 7 Vijay Bellur 2016-03-31 07:48:58 UTC

COMMIT: http://review.gluster.org/13753 committed in release-3.7 by Kaushal M (kaushal) 
------
commit 116419f3cb7b5711b0881428c2deda829b43e782
Author: Prashanth Pai <ppai>
Date:   Tue Mar 15 14:21:18 2016 +0530

    md-cache: Cache gluster-swift metadata
    
    > BUG: 1317785
    > Change-Id: Ie02b8fc294802f8fdf49dee8bf97f1e6177d92bd
    > Signed-off-by: Prashanth Pai <ppai>
    > Reviewed-on: http://review.gluster.org/13735
    > Smoke: Gluster Build System <jenkins.com>
    > Reviewed-by: Poornima G <pgurusid>
    > NetBSD-regression: NetBSD Build System <jenkins.org>
    > CentOS-regression: Gluster Build System <jenkins.com>
    > Reviewed-by: Niels de Vos <ndevos>
    > Reviewed-by: Gaurav Kumar Garg <ggarg>
    
    BUG: 1317788
    Change-Id: I86efca6a829cdda70ec7ed7fe0a16b73c25888c3
    Signed-off-by: Prashanth Pai <ppai>
    Reviewed-on: http://review.gluster.org/13753
    NetBSD-regression: NetBSD Build System <jenkins.org>
    CentOS-regression: Gluster Build System <jenkins.com>
    Reviewed-by: Niels de Vos <ndevos>
    Smoke: Gluster Build System <jenkins.com>

Comment 8 Kaushal 2016-04-19 06:59:50 UTC

This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.10, please open a new bug report.

glusterfs-3.7.10 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-users/2016-April/026164.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user