Bug 986456 - HTTP error 503 when accessing an account/volume not in the ring
HTTP error 503 when accessing an account/volume not in the ring
Status: CLOSED DUPLICATE of bug 986458
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: gluster-swift (Show other bugs)
2.1
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Luis Pabón
pushpesh sharma
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-19 16:33 EDT by Luis Pabón
Modified: 2016-11-08 17:25 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Rings file in /etc/swift were not updated after preparing a new volume. Consequence: It fails all the REST apis with error code 503. Fix: Fix is to change gluster-swift-gen-builders to prepare ring files in /etc/swift directory irrespective of where it runs. Result: No more 503 due to volume not in ring although when it is set for use with swift.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-10 13:16:34 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Luis Pabón 2013-07-19 16:33:55 EDT
Description of problem:
If a new glusterFS volume is setup with swift, but the ring is not updated, the server returns a 503 because of an assert.

Few things are bad here:
* User has to run gluster-swift-gen-buiders <accounts...> must be run in /etc/swift.  The user may not know that.  Software must inform user where the files have been stored - New bug to be created:
* Assert is executed on RHS.  Optimized python should not have executed the assert. Also if the assert is removed, an exception occurs.  New bug to be created.
* Even if the assert is removed, the server returns a 503 when the volume/account has passed authentication but is not in the ring.

Traceback for the assert:
Jul 19 16:01:26 heka-client-09 account-server ERROR __call__ error with HEAD /volume_not_in_ring/0/AUTH_glustervol : 
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 333, in __call__
    res = method(req)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 1558, in wrapped
    return func(*a, **kw)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 520, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 153, in HEAD
    broker = self._get_account_broker(drive, part, account)
  File "/usr/lib/python2.6/site-packages/gluster/swift/account/server.py", line 38, in _get_account_broker
    return DiskAccount(self.root, drive, account, self.logger)
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/DiskDir.py", line 554, in __init__
    assert self._dir_exists_read_metadata()


Trackeback when the assert is removed:
Jul 19 16:25:10 heka-client-09 proxy-server ERROR 500 Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 333, in __call__
    res = method(req)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 1558, in wrapped
    return func(*a, **kw)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 520, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 189, in GET
    broker = self._get_account_broker(drive, part, account)
  File "/usr/lib/python2.6/site-packages/gluster/swift/account/server.py", line 38, in _get_account_broker
    return DiskAccount(self.root, drive, account, self.logger)
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/DiskDir.py", line 558, in __init__
    create_account_metadata(self.datadir)
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/utils.py", line 451, in create_account_metadata
    metadata = get_account_metadata(acc_path)
  File "/usr/lib/python2. From Account Server 127.0.0.1:6012 (txn: tx99201998f70c45caab24efc76813f981)
Jul 19 16:25:10 heka-client-09 proxy-server Account GET returning 503 for [500] (txn: tx99201998f70c45caab24efc76813f981) (client_ip: 127.0.0.1)


Version-Release number of selected component (if applicable):
[root@heka-client-09 swift]# rpm -qa | grep "gluster"
gluster-swift-object-1.8.0-6.3.el6rhs.noarch
vdsm-gluster-4.10.2-22.6.el6rhs.noarch
glusterfs-geo-replication-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-doc-1.4.8-4.el6.noarch
gluster-swift-1.8.0-6.3.el6rhs.noarch
gluster-swift-container-1.8.0-6.3.el6rhs.noarch
gluster-swift-account-1.8.0-6.3.el6rhs.noarch
glusterfs-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-server-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-proxy-1.8.0-6.3.el6rhs.noarch
gluster-swift-plugin-1.8.0-1.el6rhs.noarch
glusterfs-fuse-3.4.0.12rhs.beta3-1.el6rhs.x86_64


How reproducible:
Very


Steps to Reproduce:
1. add a new glusterfs mount (or xfs directory) to the root of the gluster-swift directory
2. setup authentication accordingly 
3. Run gluster-swift-gen-buiders <accounts...> but NOT in the /etc/swift directory
4. swift-init main restart
5. Get a AUTH token
6. Create a container on that volume/account --- should return a 503

Actual results:
Returns a 503

Expected results:
Should return a 201

Additional info:
Comment 2 Luis Pabón 2013-07-19 16:53:42 EDT
Expected Results is incorrect.   It should return a 201 only when the ring is updated.  It should return the appropriate http code for an account that does not exist in the ring.
Comment 3 Divya 2013-09-10 07:49:40 EDT
Luis, 

Please add the known issue details in the Doc Text field.
Comment 4 Luis Pabón 2013-09-10 13:16:34 EDT

*** This bug has been marked as a duplicate of bug 986458 ***
Comment 5 Divya 2013-09-11 04:49:52 EDT
This bug/issue has been fixed now. Hence, no more a known issue and will not be documented in Release Notes.

Note You need to log in before you can comment on or make changes to this bug.