Bug 983467 - [Builders] swift services can not be started without running 'gluster-swift-gen-builder' for volumes
[Builders] swift services can not be started without running 'gluster-swift-g...
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: gluster-swift (Show other bugs)
2.1
x86_64 Linux
high Severity high
: ---
: ---
Assigned To: Luis Pabón
pushpesh sharma
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-11 05:32 EDT by pushpesh sharma
Modified: 2016-11-08 17:25 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-23 18:32:31 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description pushpesh sharma 2013-07-11 05:32:56 EDT
Description of problem:


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.Fix the confs files of the fresh RHS2.1 install
2. service memcached start.
3. swift-init main start

[root@dhcp207-9 swift]# swift-init main start
Starting proxy-server...(/etc/swift/proxy-server.conf)
Starting container-server...(/etc/swift/container-server.conf)
Starting account-server...(/etc/swift/account-server.conf)
Starting object-server...(/etc/swift/object-server.conf)
Traceback (most recent call last):
  File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 130, in run_wsgi
    loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 272, in loadobj
    return context.create()
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 203, in invoke
    app = context.app_context.create()
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 710, in create
    return self.object_type.invoke(self)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 146, in invoke
    return fix_call(context.object, context.global_conf, **context.local_conf)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/util.py", line 56, in fix_call
    val = callable(*args, **kw)
  File "/usr/lib/python2.6/site-packages/gluster/swift/proxy/server.py", line 28, in app_factory
    return server.Application(conf)
  File "/usr/lib/python2.6/site-packages/swift/proxy/server.py", line 80, in __init__
    self.object_ring = object_ring or Ring(swift_dir, ring_name='object')
  File "/usr/lib/python2.6/site-packages/gluster/swift/common/ring.py", line 47, in __init__
    ring.Ring.__init__(self, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/swift/common/ring/ring.py", line 141, in __init__
    self._reload(force=True)
  File "/usr/lib/python2.6/site-packages/swift/common/ring/ring.py", line 146, in _reload
    ring_data = RingData.load(self.serialized_path)
  File "/usr/lib/python2.6/site-packages/swift/common/ring/ring.py", line 63, in load
    gz_file = GzipFile(filename, 'rb')
  File "/usr/lib64/python2.6/gzip.py", line 79, in __init__
    fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')
IOError: [Errno 2] No such file or directory: '/etc/swift/object.ring.gz'


Actual results:

However after running, gen-builders like below:-
gluster-swift-gen-builders test test2

swift services start properly. 

[root@dhcp207-9 swift]# swift-init main start
Starting proxy-server...(/etc/swift/proxy-server.conf)
Starting container-server...(/etc/swift/container-server.conf)
Starting account-server...(/etc/swift/account-server.conf)
Starting object-server...(/etc/swift/object-server.conf)


Expected results:
There was no need to run gen-builders in RHS2.0 and there is no mention of the same in user guide as well. 

Additional info:

[root@dhcp207-9 swift]# rpm -qa|grep gluster
gluster-swift-object-1.8.0-6.3.el6rhs.noarch
vdsm-gluster-4.10.2-22.7.el6rhs.noarch
gluster-swift-plugin-1.8.0-2.el6rhs.noarch
glusterfs-geo-replication-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-1.8.0-6.3.el6rhs.noarch
glusterfs-server-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-proxy-1.8.0-6.3.el6rhs.noarch
gluster-swift-account-1.8.0-6.3.el6rhs.noarch
glusterfs-rdma-3.4.0.12rhs.beta3-1.el6rhs.x86_64
glusterfs-fuse-3.4.0.12rhs.beta3-1.el6rhs.x86_64
gluster-swift-container-1.8.0-6.3.el6rhs.noarch
Comment 2 Luis Pabón 2013-07-12 14:34:53 EDT
Very good find.  This will require both documentation and software fix.
Comment 3 Junaid 2013-07-17 02:20:23 EDT
Yes, this was not a requirement in RHS 2.0. In RHS 2.0, the ring files were statically created and placed in /etc/swift by the gluster-swift-plugin rpm.

There are two basic reason we used gluster-swift-gen-builders 
1. To provide a means to add a check on the gluster volumes exposed through swift. Admins may not want to expose all the volumes, so this is a handy tool to achieve this goal.

2. In RHS 2.0, the gluster-swift code made modifications to openstack swift code. So, it was easier to manipulate the parameters to file operation calls when we required them. For example, we had to replace the drive name with account name in the code which called DiskFile class init since we were using static ring files. With gluster-swift-gen-builders, this is no longer required and hence a cleaner code. For more information, check gluster-swift.git/gluster/swift/common/ring.py file.

Definitely, we should update the documentation. Calling gluster-swift-gen-builders is similar to swift's requirement to update the ring files as and when a new drive is added. So, we can correlate it to same with an added benifit of a ready made script that would take care of doing it to the users.
Comment 4 Peter Portante 2013-07-17 07:16:05 EDT
Agree with Junaid: we need documentation to tell people that upgrading from old to new version of gluster-swift-plugin requires the ring build to be run listing all the gluster volumes that will be served.

We should probably check to see if the old ring files are in use, and fail the proxy server startup, but that is going to be a bit more work and possibly requiring some upstream effort.
Comment 5 Junaid 2013-07-17 07:33:59 EDT
Peter,

The swift code keeps checking the ring files for any modification every 15sec if I am not wrong. So, its even more easier to upgrade.
Comment 6 Peter Portante 2013-07-17 10:00:50 EDT
Not sure that our ring module does that.
Comment 8 Peter Portante 2013-07-17 11:56:17 EDT
Yes, but look at our subclass and check to see the methods we override don't also check for a reload.
Comment 9 Luis Pabón 2013-07-19 11:58:19 EDT
Junaid, 
  Probably we should check for the existence of the file in gluster/swift/common/ring.py::Ring.__init__() before it calls the super class initialization method.
Comment 10 Luis Pabón 2013-07-29 14:35:15 EDT
We have informed documentation to make sure to run the ring builder script before starting up the system.

The code fix is still going to be an error, but a 'nicer' one.  The correct fix, is to educate the client/user by providing the correct documentation.

Please make sure gluster-swift-gen-builders is run *before* starting swift.
Comment 11 pushpesh sharma 2013-08-05 05:09:58 EDT
Documentation is done perfectly to as per the new requirement to run gluster-swift-gen builders .
Comment 14 Scott Haines 2013-09-23 18:32:31 EDT
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html

Note You need to log in before you can comment on or make changes to this bug.