Bug 903396 - Tracker for gluster-swift refactoring work (PDQ)
Summary: Tracker for gluster-swift refactoring work (PDQ)
Keywords:
Status: CLOSED EOL
Alias: None
Product: GlusterFS
Classification: Community
Component: object-storage
Version: mainline
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ---
Assignee: Luis Pabón
QA Contact:
URL:
Whiteboard:
: 978061 (view as bug list)
Depends On: 865619 865858 865926 868086 868120 894674 904370 904629 911443 911446 821310 827410 844805 858413 865867 885281 890348 907072 911448 928919 951661
Blocks: 956991 978061
TreeView+ depends on / blocked
 
Reported: 2013-01-23 21:52 UTC by Peter Portante
Modified: 2018-12-06 14:55 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 978061 (view as bug list)
Environment:
Last Closed: 2015-10-22 15:46:38 UTC
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 158823 0 None None None Never

Description Peter Portante 2013-01-23 21:52:47 UTC
The UFO code, include some code in Openstack Swift proper, end up invoking a number of system calls that are not necessary, adding unnecessary load to the system and/or hampering performance for both small and large object transfers.

In some cases, this results in severe behaviors where GlusterFS becomes unavailable or crashes. In other cases, performance degrades to the point where it is not usable (60+ seconds to fulfill a request).

A first pass analysis of the system calls being made needs to be done to track down where they originate and remove the code the is responsible for any unwanted invocations.

Comment 1 Vijay Bellur 2013-03-07 05:08:51 UTC
CHANGE: http://review.gluster.org/4641 (storage/posix: Remove a redundant lstat in posix_handle_hard.) merged in master by Anand Avati (avati)

Comment 2 Anand Avati 2013-04-08 02:26:35 UTC
REVIEW: http://review.gluster.org/4787 (object-storage: turn off stat() for container list) posted (#1) for review on master by Mohammed Junaid (junaid)

Comment 3 Anand Avati 2013-04-12 07:08:13 UTC
REVIEW: http://review.gluster.org/4787 (object-storage: turn off stat() for container list) posted (#2) for review on master by Mohammed Junaid (junaid)

Comment 4 Peter Portante 2013-04-12 18:43:01 UTC
This issue is not blocked by a RHEL 6 issue for running out of inodes (#928377) for two reasons:

 1. Does not seem right for an upstream patch to be blocked by a downstream
    issue

 2. The actions taken for running out inodes in Gluster, given the state of
    #928377, are no longer needed

Comment 5 Anand Avati 2013-04-12 20:49:12 UTC
COMMIT: http://review.gluster.org/4787 committed in master by Anand Avati (avati) 
------
commit bbaa273468f8e5377027aedcabcaa076dd7fec7e
Author: Mohammed Junaid <junaid>
Date:   Sun Apr 7 06:05:56 2013 +0530

    object-storage: turn off stat() for container list
    
    Turn of stat() system calls used to fetch the file size during a
    container listing operation since these system calls can swamp Gluster
    and the result is most often not used.
    
    When a GET or HEAD request is made on a container, stat() system calls
    are made during the Python standard library method, os.walk, to
    determine if a given directory entry is another directory to recurse
    into, and then utils._update_list() will stat() each file to get it
    size, and finally utils.get_container_details_from_fs() will stat()
    each directory encountered.
    
    For most installations we have seen so far, we don't need the
    container listing to accurately return the size of all the objects in
    the container, so we can reduce the number of stat() system calls by
    not fetching the size of the object.
    
    For now, turn it off by default, and provide an /etc/swift/fs.conf
    configuration parameter to turn it back on:
    
      accurate_size_in_listing = yes
    
    The default for the above is "no".
    
    Change-Id: I7dde11e14bb32ecafa3eabb08852f1ffc4366b35
    BUG: 903396
    Signed-off-by: Mohammed Junaid <junaid>
    Reviewed-on: http://review.gluster.org/4787
    Tested-by: Gluster Build System <jenkins.com>
    Reviewed-by: Kaleb KEITHLEY <kkeithle>
    Reviewed-by: Anand Avati <avati>

Comment 6 Anand Avati 2013-04-19 18:17:34 UTC
REVIEW: http://review.gluster.org/4859 (object-storage: remove stat of directories) posted (#1) for review on master by Mohammed Junaid (junaid)

Comment 7 Luis Pabón 2013-08-02 14:56:00 UTC
*** Bug 978061 has been marked as a duplicate of this bug. ***

Comment 8 Kaleb KEITHLEY 2015-10-22 15:46:38 UTC
because of the large number of bugs filed against mainline version\ is ambiguous and about to be removed as a choice.

If you believe this is still a bug, please change the status back to NEW and choose the appropriate, applicable version for it.


Note You need to log in before you can comment on or make changes to this bug.