Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1062599

Summary: swift-proxy does not start because _ is undefined
Product: Red Hat OpenStack Reporter: Matthew Mosesohn <mmosesohn>
Component: python-keystoneclientAssignee: Jakub Ruzicka <jruzicka>
Status: CLOSED ERRATA QA Contact: Ami Jeain <ajeain>
Severity: high Docs Contact:
Priority: high    
Version: 4.0CC: apevec, ayoung, breeler, jagee, jruzicka, mmosesohn, pbrady, sclewis, yeylon, zaitcev
Target Milestone: z3Keywords: ZStream
Target Release: 4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-keystoneclient-0.6.0-1.el6ost Doc Type: Bug Fix
Doc Text:
Previously, updates to Identity made it impossible to include WSGI middleware directly from the openstack-keystone package into unrelated services such as Object Storage. The python-keystoneclient package version 0.4.x did not include the middleware package. In consequence, Object Storage services fail to start. This has been fixed in python-keystoneclient version 0.6.0. Note: Legacy proxy-server.conf may need changes in pilelines from keystone.middleware to keystoneclient.middleware in the [authtoken] and [s3token] sections, depending on how long ago the original installation was made. Now, when using python-keystoneclient version 0.6.0. and having made any necessary changes to pipelines, OpenStack Storage services start and function normally.
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-03-25 19:23:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1073572    
Bug Blocks:    
Attachments:
Description Flags
swift proxy-server.conf none

Description Matthew Mosesohn 2014-02-07 12:24:46 UTC
Description of problem:
swift-proxy fails to import 

Version-Release number of selected component (if applicable):
openstack-keystone-2013.2.1-1.el6ost.noarch
openstack-swift-1.10.0-2.el6ost.noarch
openstack-swift-account-1.10.0-2.el6ost.noarch
openstack-swift-container-1.10.0-2.el6ost.noarch
openstack-swift-object-1.10.0-2.el6ost.noarch
openstack-swift-plugin-swift3-1.0.0-0.20120711git.1.el6ost.noarch
openstack-swift-proxy-1.10.0-2.el6ost.noarch
python-keystone-2013.2.1-1.el6ost.noarch
python-keystoneclient-0.4.1-3.el6ost.noarch
python-swiftclient-1.8.0-1.el6ost.noarch


How reproducible:
always

Steps to Reproduce:
1. service openstack-swift-proxy start
2.
3.

Actual results:
swift-proxy starts

Expected results:
Service fails to start and is in a failed state with pid file in place

Additional info:
Traceback captured in /var/log/swift-startup.log
# cat /var/log/swift-startup.log 
Traceback (most recent call last):
  File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 256, in run_wsgi
    loadapp(conf_path, global_conf={'log_name': log_name})
  File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 107, in wrapper
    return f(conf_uri, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 450, in get_context
    global_additions=global_additions)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 562, in _pipeline_app_context
    for name in pipeline[:-1]]
  File "/usr/lib/python2.6/site-packages/swift/common/wsgi.py", line 55, in get_context
    object_type, name=name, global_conf=global_conf)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 458, in get_context
    section)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 517, in _context_from_explicit
    value = import_string(found_expr)
  File "/usr/lib/python2.6/site-packages/PasteDeploy-1.5.0-py2.6.egg/paste/deploy/loadwsgi.py", line 22, in import_string
    return pkg_resources.EntryPoint.parse("x=" + s).load(False)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 1948, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/lib/python2.6/site-packages/keystone/middleware/__init__.py", line 18, in <module>
    from keystone.middleware.core import *
  File "/usr/lib/python2.6/site-packages/keystone/middleware/core.py", line 21, in <module>
    from keystone.common import utils
  File "/usr/lib/python2.6/site-packages/keystone/common/utils.py", line 32, in <module>
    from keystone import exception
  File "/usr/lib/python2.6/site-packages/keystone/exception.py", line 63, in <module>
    class ValidationError(Error):
  File "/usr/lib/python2.6/site-packages/keystone/exception.py", line 64, in ValidationError
    message_format = _("Expecting to find %(attribute)s in %(target)s."
NameError: name '_' is not defined


The simplest fix is https://bugs.launchpad.net/ubuntu/+source/swift/+bug/1231339
but this was handled upstream in stable/havana with a different approach by 
handling the import in keystone.openstack.common.strutils

Comment 2 Perry Myers 2014-02-17 15:10:08 UTC
@abaron/zaitcev/apevec, can one of you take a look at this?

Comment 3 Pete Zaitcev 2014-02-18 06:37:21 UTC
I found no trace of any fix in Havana branch of keystone or
python-keystoneclient. The keystoneclient may work by virtue
of not doing the "from keystone.middleware.core import *".
I don't understand what it does there, everything seems to
load with that statement commented out from
keystone/middleware/__init__.py.

Anynow, I request that our Keystone wizards add
from keystone.openstack.common.gettextutils import _
into keystone/common/utils.py and keystone/exception.py.
That appears sufficient.

Comment 5 Pete Zaitcev 2014-02-19 16:19:16 UTC
Talked it over with Adam on IRC. The decision we need to make is this:
Either
 - ship a fix-up for keystone RPM that I mentioned in comment #3, or
 - upgrade python-keystoneclient to 0.6.0
Adam said we need to figure out which, propose a patch, and he'll
review and approve.

Padraig/Alan: is it possible to ship python-swiftclient in EPEL 6?
And same for Fedora 20 RDO?

Currently I have python-keystoneclient-0.4.1-4.fc20.noarch FYI

Comment 6 Pádraig Brady 2014-02-19 16:46:16 UTC
I see the 1.8.0 swiftclient update for F20 is blocked with reported failure:
https://admin.fedoraproject.org/updates/FEDORA-2013-23116/python-swiftclient-1.8.0-1.fc20

I see keystoneclient 0.6.0 available upstream,
0.4.2 available in koji,
and 0.4.1 available in repos

jruzicka handles all the clients now

Comment 7 Jakub Ruzicka 2014-02-19 21:06:26 UTC
This is a good reason to update keystoneclient with nice side-effect of being closer to upstream version.

However, following versions were bumped in requirements.txt between 0.4.1 and 0.6.0:

* PrettyTable  >=0.6,<0.8  ->  >=0.7,<0.8
  [NOT MET] 0.6.1
* netaddr  N/A  ->  >=0.7.6
  [NOT MET] 0.7.5
* iso8601  >=0.1.4  ->  >=0.1.8
  [OK] 0.1.8

Often, required versions aren't strictly needed so I'm gonna build and test 0.6.0. If it works or can be made to work with small fixes, I'm fine with solution.

Comment 8 Alan Pevec 2014-02-21 14:29:34 UTC
Note that keystoneclient >= 0.4.2 breaks puppet-keystone.
Puppet module fix is not merged yet https://review.openstack.org/73970

Comment 9 Alan Pevec 2014-02-21 14:42:00 UTC
> swift-proxy fails to import 

Please attach your proxy-server.conf

> this was handled upstream in stable/havana with a different approach by 
> handling the import in keystone.openstack.common.strutils

You mean https://review.openstack.org/60386 ? How is that fixing reported issue?

Comment 10 Matthew Mosesohn 2014-02-21 14:59:00 UTC
I'm having some deployment issues. I will get this to you tomorrow.

Comment 11 Matthew Mosesohn 2014-02-22 15:15:51 UTC
Created attachment 866427 [details]
swift proxy-server.conf

Comment 12 Matthew Mosesohn 2014-03-05 15:49:42 UTC
Any updates?

Comment 13 Pete Zaitcev 2014-03-05 16:07:27 UTC
I didn't realize that swift3 was separate story. I only noticed it
yesterday, coincidentially. Matthew's proxy-server.conf in comment #11
includes swift3, so our fixes aren't going to be sufficient to cure him.
I'll look at it today.

For now I worked around by adding an explicit import in exceptions.py,
which is sufficient, provided the rest of packages are up to date.

Comment 14 Pete Zaitcev 2014-03-06 00:43:42 UTC
Er, I take this back. Moving to keystoneclient 0.6 per Jakub's
comment #7 is sufficient to resolve everything, because s3_token
is also relocated to keystoneclient (at a cost of a small change
in proxy-server.conf from keystone.foo to keystoneclient.foo).

The fix for Puppet that Alan mentioned in comment #9 is merged.

Comment 15 Jakub Ruzicka 2014-03-06 15:23:41 UTC
As keystoneclient rebase to 0.6.0 will also fix #1066608, let's do this in 4.0.z.

Required puppet module fix is merged so we should be good to go.

Comment 18 Ami Jeain 2014-03-12 13:37:55 UTC
verified in the openstack-swift-1.10.0-3.el6ost.noarch and python-keystoneclient-0.6.0-1.el6ost.noarch:
# openstack-status |grep swift
openstack-swift-proxy:                  active
openstack-swift-account:                active
openstack-swift-container:              active
openstack-swift-object:                 active

# /etc/init.d/openstack-swift-proxy status
openstack-swift-proxy (pid  28969) is running...

Comment 20 errata-xmlrpc 2014-03-25 19:23:45 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0334.html