Bug 986456 - HTTP error 503 when accessing an account/volume not in the ring
Summary: HTTP error 503 when accessing an account/volume not in the ring
Keywords:
Status: CLOSED DUPLICATE of bug 986458
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: gluster-swift
Version: 2.1
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: ---
Assignee: Luis Pabón
QA Contact: pushpesh sharma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-07-19 20:33 UTC by Luis Pabón
Modified: 2016-11-08 22:25 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: Rings file in /etc/swift were not updated after preparing a new volume. Consequence: It fails all the REST apis with error code 503. Fix: Fix is to change gluster-swift-gen-builders to prepare ring files in /etc/swift directory irrespective of where it runs. Result: No more 503 due to volume not in ring although when it is set for use with swift.
Clone Of:
Environment:
Last Closed: 2013-09-10 17:16:34 UTC
Embargoed:


Attachments (Terms of Use)

Description Luis Pabón 2013-07-19 20:33:55 UTC
Description of problem:
If a new glusterFS volume is setup with swift, but the ring is not updated, the server returns a 503 because of an assert.

Few things are bad here:
* User has to run gluster-swift-gen-buiders <accounts...> must be run in /etc/swift.  The user may not know that.  Software must inform user where the files have been stored - New bug to be created:
* Assert is executed on RHS.  Optimized python should not have executed the assert. Also if the assert is removed, an exception occurs.  New bug to be created.
* Even if the assert is removed, the server returns a 503 when the volume/account has passed authentication but is not in the ring.

Traceback for the assert:
Jul 19 16:01:26 heka-client-09 account-server ERROR __call__ error with HEAD /volume_not_in_ring/0/AUTH_glustervol : 
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 333, in __call__
    res = method(req)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 1558, in wrapped
    return func(*a, **kw)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 520, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 153, in HEAD
    broker = self._get_account_broker(drive, part, account)
  File "/usr/lib/python2.6/site-packages/gluster/swift/account/server.py", line 38, in _get_account_broker
    return DiskAccount(self.root, drive, account, self.logger)
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/DiskDir.py", line 554, in __init__
    assert self._dir_exists_read_metadata()


Trackeback when the assert is removed:
Jul 19 16:25:10 heka-client-09 proxy-server ERROR 500 Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 333, in __call__
    res = method(req)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 1558, in wrapped
    return func(*a, **kw)
  File "/usr/lib/python2.6/site-packages/swift/common/utils.py", line 520, in _timing_stats
    resp = func(ctrl, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/swift/account/server.py", line 189, in GET
    broker = self._get_account_broker(drive, part, account)
  File "/usr/lib/python2.6/site-packages/gluster/swift/account/server.py", line 38, in _get_account_broker
    return DiskAccount(self.root, drive, account, self.logger)
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/DiskDir.py", line 558, in __init__
    create_account_metadata(self.datadir)
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/utils.py", line 451, in create_account_metadata
    metadata = get_account_metadata(acc_path)
  File "/usr/lib/python2. From Account Server 127.0.0.1:6012 (txn: tx99201998f70c45caab24efc76813f981)
Jul 19 16:25:10 heka-client-09 proxy-server Account GET returning 503 for [500] (txn: tx99201998f70c45caab24efc76813f981) (client_ip: 127.0.0.1)


Version-Release number of selected component (if applicable):
[root@heka-client-09 swift]# rpm -qa | grep "gluster"
gluster-swift-object-1.8.0-6.3.el6rhs.noarch
vdsm-gluster-4.10.2-22.6.el6rhs.noarch
glusterfs-geo-replication-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-doc-1.4.8-4.el6.noarch
gluster-swift-1.8.0-6.3.el6rhs.noarch
gluster-swift-container-1.8.0-6.3.el6rhs.noarch
gluster-swift-account-1.8.0-6.3.el6rhs.noarch
glusterfs-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-server-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-rdma-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-proxy-1.8.0-6.3.el6rhs.noarch
gluster-swift-plugin-1.8.0-1.el6rhs.noarch
glusterfs-fuse-3.4.0.12rhs.beta3-1.el6rhs.x86_64


How reproducible:
Very


Steps to Reproduce:
1. add a new glusterfs mount (or xfs directory) to the root of the gluster-swift directory
2. setup authentication accordingly 
3. Run gluster-swift-gen-buiders <accounts...> but NOT in the /etc/swift directory
4. swift-init main restart
5. Get a AUTH token
6. Create a container on that volume/account --- should return a 503

Actual results:
Returns a 503

Expected results:
Should return a 201

Additional info:

Comment 2 Luis Pabón 2013-07-19 20:53:42 UTC
Expected Results is incorrect.   It should return a 201 only when the ring is updated.  It should return the appropriate http code for an account that does not exist in the ring.

Comment 3 Divya 2013-09-10 11:49:40 UTC
Luis, 

Please add the known issue details in the Doc Text field.

Comment 4 Luis Pabón 2013-09-10 17:16:34 UTC

*** This bug has been marked as a duplicate of bug 986458 ***

Comment 5 Divya 2013-09-11 08:49:52 UTC
This bug/issue has been fixed now. Hence, no more a known issue and will not be documented in Release Notes.


Note You need to log in before you can comment on or make changes to this bug.