Red Hat Bugzilla – Bug 903396
Tracker for gluster-swift refactoring work (PDQ)
Last modified: 2015-10-22 11:46:38 EDT
The UFO code, include some code in Openstack Swift proper, end up invoking a number of system calls that are not necessary, adding unnecessary load to the system and/or hampering performance for both small and large object transfers.
In some cases, this results in severe behaviors where GlusterFS becomes unavailable or crashes. In other cases, performance degrades to the point where it is not usable (60+ seconds to fulfill a request).
A first pass analysis of the system calls being made needs to be done to track down where they originate and remove the code the is responsible for any unwanted invocations.
CHANGE: http://review.gluster.org/4641 (storage/posix: Remove a redundant lstat in posix_handle_hard.) merged in master by Anand Avati (firstname.lastname@example.org)
REVIEW: http://review.gluster.org/4787 (object-storage: turn off stat() for container list) posted (#1) for review on master by Mohammed Junaid (email@example.com)
REVIEW: http://review.gluster.org/4787 (object-storage: turn off stat() for container list) posted (#2) for review on master by Mohammed Junaid (firstname.lastname@example.org)
This issue is not blocked by a RHEL 6 issue for running out of inodes (#928377) for two reasons:
1. Does not seem right for an upstream patch to be blocked by a downstream
2. The actions taken for running out inodes in Gluster, given the state of
#928377, are no longer needed
COMMIT: http://review.gluster.org/4787 committed in master by Anand Avati (email@example.com)
Author: Mohammed Junaid <firstname.lastname@example.org>
Date: Sun Apr 7 06:05:56 2013 +0530
object-storage: turn off stat() for container list
Turn of stat() system calls used to fetch the file size during a
container listing operation since these system calls can swamp Gluster
and the result is most often not used.
When a GET or HEAD request is made on a container, stat() system calls
are made during the Python standard library method, os.walk, to
determine if a given directory entry is another directory to recurse
into, and then utils._update_list() will stat() each file to get it
size, and finally utils.get_container_details_from_fs() will stat()
each directory encountered.
For most installations we have seen so far, we don't need the
container listing to accurately return the size of all the objects in
the container, so we can reduce the number of stat() system calls by
not fetching the size of the object.
For now, turn it off by default, and provide an /etc/swift/fs.conf
configuration parameter to turn it back on:
accurate_size_in_listing = yes
The default for the above is "no".
Signed-off-by: Mohammed Junaid <email@example.com>
Tested-by: Gluster Build System <firstname.lastname@example.org>
Reviewed-by: Kaleb KEITHLEY <email@example.com>
Reviewed-by: Anand Avati <firstname.lastname@example.org>
REVIEW: http://review.gluster.org/4859 (object-storage: remove stat of directories) posted (#1) for review on master by Mohammed Junaid (email@example.com)
*** Bug 978061 has been marked as a duplicate of this bug. ***
because of the large number of bugs filed against mainline version\ is ambiguous and about to be removed as a choice.
If you believe this is still a bug, please change the status back to NEW and choose the appropriate, applicable version for it.