Bug 1317790 - Cache swift xattrs
Summary: Cache swift xattrs
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: fuse
Version: rhgs-3.1
Hardware: All
OS: All
Target Milestone: ---
: RHGS 3.1.3
Assignee: Bug Updates Notification Mailing List
QA Contact: surabhi
Depends On: 1317785 1317788
Blocks: 1299184 1314171 1328444
TreeView+ depends on / blocked
Reported: 2016-03-15 08:40 UTC by Prashanth Pai
Modified: 2016-06-23 05:10 UTC (History)
9 users (show)

Fixed In Version: glusterfs-3.7.9-2
Doc Type: Bug Fix
Doc Text:
Clone Of: 1317788
Last Closed: 2016-06-23 05:10:59 UTC
Target Upstream Version:

Attachments (Terms of Use)
pcap to show the reduced xattr calls while PUT and Delete (40.32 KB, application/octet-stream)
2016-05-10 11:26 UTC, surabhi
no flags Details

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1240 0 normal SHIPPED_LIVE Red Hat Gluster Storage 3.1 Update 3 2016-06-23 08:51:28 UTC

Description Prashanth Pai 2016-03-15 08:40:25 UTC
+++ This bug was initially created as a clone of Bug #1317788 +++

+++ This bug was initially created as a clone of Bug #1317785 +++

Description of problem:

gluster-swift extensively uses "user.swift.metadata" xattr in it's functionality. This is not cached and hence the performance is usually terrible.

Every operation (GET,PUT,POST,DELETE,HEAD) in gluster-swift issues getxattr() as follows:

getxattr("/mnt/gluster-object/test/c1/o8", "user.swift.metadata", 0x0, 0) = 190
getxattr("/mnt/gluster-object/test/c1/o8", "user.swift.metadata", "{"Content-Length":"11","ETag":"5eb63bbbe01eeed093cb22bb8f5acdc3","X-Timestamp":"1458021187.45380","X-Object-Type":"file","X-Type":"Object","Content-Type":"application/x-www-form-urlencoded"}", 190) = 190

The first getxattr() is to get the size of xattr value and the second getxattr() is to fetch the actual value itself.

The entire xattr information is sent thrice by brick to FUSE mount. Once on lookup, again on the first getxattr() to get size and then again on second gexattr() call. These three network calls can be reduced to just one if the xattr information is cached when it's fetched for the first time.

--- Additional comment from Prashanth Pai on 2016-03-15 04:34 EDT ---

Attached tcpdump trace of DELETE object operation from gluster-swift

Comment 3 Prashanth Pai 2016-03-15 09:24:04 UTC
Upstream change on review: http://review.gluster.org/13735

Comment 5 Mike McCune 2016-03-28 23:26:37 UTC
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions

Comment 9 surabhi 2016-05-09 14:19:14 UTC
Performed GET,PUT and DELETE operations and saw only one getxattr calls happening for the specific operations which may improve the performance. Currently verified on the basis os reduced xattr calls. Will take up the basic perf in regression cycle. marking the BZ verified.

[pid 19074] open("/mnt/gluster-object/test/c1/o1", O_RDONLY|O_CLOEXEC) = 10
[pid 19074] fstat(10, {st_mode=S_IFREG|0755, st_size=8, ...}) = 0
[pid 19074] fgetxattr(10, "user.swift.metadata", 0x0, 0) = 178
[pid 19074] fgetxattr(10, "user.swift.metadata", "{"Content-Length":8,"ETag":"15fe514867dd5b4a1abf91ea35ff9e22","X-Object-Type":"file","X-Timestamp":"1462801876.06361","X-Type":"Object","Content-Type":"application/octet-stream"}", 178) = 178
[pid 19074] close(10)                   = 0
[pid 19074] setxattr("/mnt/gluster-object/test/c1/o1", "user.swift.metadata", "{"Content-Length":8,"ETag":"15fe514867dd5b4a1abf91ea35ff9e22","X-Timestamp":"1462802038.60981","X-Object-Type":"file","X-Type":"Object","Content-Type":"application/octet-stream","X-Object-Meta-Hello":"World"}", 208, 0) = 0
[pid 19074] poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP}, {fd=13, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 2, 0) = 0 (Timeout)
[pid 19074] sendto(6, "HTTP/1.1 202 Accepted\r\nContent-L"..., 217, 0, NULL, 0) = 217
[pid 19074] shutdown(6, SHUT_RDWR)      = 0
[pid 19074] close(6)                    = 0
[pid 19074] poll([{fd=4, events=POLLIN|POLLPRI|POLLERR|POLLHUP}, {fd=13, events=POLLIN|POLLPRI|POLLERR|POLLHUP}], 2, 60000

Comment 10 surabhi 2016-05-10 11:26:09 UTC
Created attachment 1155681 [details]
pcap to show the reduced xattr calls while PUT and Delete

Comment 12 errata-xmlrpc 2016-06-23 05:10:59 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.