Bug 947469

Summary: openstack-swift-proxy fails to start because of an ImportError
Product: [Fedora] Fedora Reporter: Martina Kollarova <mkollaro>
Component: openstack-packstackAssignee: Martin Magr <mmagr>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 19CC: aortega, apevec, derekh, Jan.van.Eldik, jkt, jonathan.barber, mkollaro, mmagr, ncredi, p, sandro, zaitcev
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-10-14 20:05:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/etc/swift/proxy-server.conf
none
proxy-server.conf after packstack run none

Description Martina Kollarova 2013-04-02 13:43:59 UTC
Description of problem:
The service openstack-swift-proxy fails to start after installing it using packstack. A manual restart of the service doesn't help. Also, packstack doesn't report that it failed.

Version-Release number of selected component (if applicable):
openstack-swift-proxy-1.7.4-2.fc19.noarch
openstack-packstack-2013.1.1-0.3.dev527.fc19.noarch
Fedora 18 in a VM

How reproducible:


Steps to Reproduce:
1. Use packstack on fedora 18 using the grizzly repo (as in https://fedoraproject.org/wiki/Test_Day:2013-04-02_OpenStack)
2. $ systemctl status openstack-swift-proxy.service
  
Actual results:
$ swift list
[Errno 111] ECONNREFUSED

$ systemctl status openstack-swift-proxy.service
openstack-swift-proxy.service - OpenStack Object Storage (swift) - Proxy Server
	  Loaded: loaded (/usr/lib/systemd/system/openstack-swift-proxy.service; enabled)
	  Active: failed (Result: exit-code) since Tue 2013-04-02 15:23:24 CEST; 1s ago
	 Process: 20227 ExecStart=/usr/bin/swift-proxy-server /etc/swift/proxy-server.conf (code=exited, status=1/FAILURE)


Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: section)
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 517, in _con...xplicit
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: value = import_string(found_expr)
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 22, in import_string
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: return pkg_resources.EntryPoint.parse("x=" + s).load(False)
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 2017, in load
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: entry = __import__(self.module_name, globals(),globals(), ['__name__'])
Apr 02 15:23:24 localhost.localdomain swift-proxy-server[20227]: ImportError: No module named swift_auth
Apr 02 15:23:24 localhost.localdomain systemd[1]: openstack-swift-proxy.service: main process exited, code=exited, status=1/FAILURE
Apr 02 15:23:24 localhost.localdomain systemd[1]: Unit openstack-swift-proxy.service entered failed state


Expected results:
The service should be in an active state.


Additional info:

Comment 2 Pete Zaitcev 2013-04-02 15:42:23 UTC
Please attach /etc/swift/proxy-server.conf. No idea what "swift_auth" is
supposed to mean. Sounds like something from Swift 1.4.x times. In Folsom
we use authtoken, so let's see.

Also, moving the bug to Derek for triaging if Packstack needs to distinguish
Essex (swift_auth) and Folsom (authtoken).

Comment 3 Martina Kollarova 2013-04-02 16:22:58 UTC
Created attachment 730855 [details]
/etc/swift/proxy-server.conf

Comment 4 Alan Pevec 2013-04-03 18:10:17 UTC
In Grizzly keystone.middleware.swift_auth was removed in https://github.com/openstack/keystone/commit/e577cd60871e1810b45236d3642d60e460dc4858

swift_auth (not to be confused with older swauth) is part of Swift since 1.6.0
see http://docs.openstack.org/developer/swift/overview_auth.html#keystone-auth

Comment 5 Derek Higgins 2013-04-04 10:50:33 UTC
This was using the grizzly side repo, in addition to fixing the config file is the correct thing to do here to package swift 1.8 for the side repo ?

Comment 6 Alan Pevec 2013-04-04 13:04:49 UTC
> is the correct thing to do here to package swift 1.8 for the side repo ?

Yes, please update master/f19 to https://launchpad.net/swift/grizzly/1.8.0-rc2
Swift 1.8.0 is part of Grizzly release.

Comment 7 Pete Zaitcev 2013-04-09 18:26:17 UTC
Derek built 1.8.0
 https://koji.fedoraproject.org/koji/buildinfo?buildID=409603
I verified it uses the new-style proxy-server.conf without swift_auth.

Martina, could you check if that one works ok?

Comment 8 Martina Kollarova 2013-04-10 15:15:45 UTC
Now it fails because of something else. The tool swift-ring-builder returns 2 when no region is specified (a new feature in Grizzly). This produces a warning and the default region is used, yet packstack fails because of the exit code.

Now I'm not sure if this is packstack's problem or if swift-ring-builder should return 0 in this case, as to not break compatibility (as it is, old commands in scripts won't work in Grizzly and new commands won't work in Folsom, because it doesn't have the region feature).

Comment 9 Pete Zaitcev 2013-04-10 17:09:14 UTC
Please file the bug for the swift-ring-builder exit code, then we can mark
this bug as depending on that one. It sounds like a genuine problem that
I'll love to fix for you quickly.

Comment 10 Martina Kollarova 2013-04-12 14:41:07 UTC
So I went around the other problem (#951077) and tried 1.8.0-2, it still gives me the same ImportError.

Comment 11 Pete Zaitcev 2013-04-12 15:04:48 UTC
Is there /etc/swift/proxy-server.conf.rpmnew by any chance?

Comment 12 Martina Kollarova 2013-04-12 15:51:20 UTC
Created attachment 734808 [details]
proxy-server.conf after packstack run

Comment 13 Martina Kollarova 2013-04-12 15:53:00 UTC
Yes, I have an .rpmnew file there and I tried using it, but packstack overrides it with it's own conf file (see attachment).

Comment 14 Pete Zaitcev 2013-04-25 02:01:12 UTC
I don't understand one thing. Derek swears that Packstack does not do
anything of the sort. Supposedly, it only modifies the configuration
that my RPMs install. So, 1.8.0 must work -- unless we install on a
dirty system and thus its proxy-server.conf ends with .rpmnew attached.
Could we verify that everything works on a freshly installed box, at least?

Comment 15 Derek Higgins 2013-04-25 11:12:56 UTC
packstack does keep its own version of the swift config files (or more accurately the puppet modules do), but the file you have attached looks like it is from an old version of packstack (without the swift grizzly changes) can you confirm the version your using and I'll take a look. Basically you should have 

[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory
...
[filter:keystone]
use = egg:swift#keystoneauth

not 

[filter:keystone]
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
...
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory

Comment 16 Jonathan Barber 2013-07-02 09:05:38 UTC
I see the same problem (swift list => [Errno 111] ECONNREFUSED) on f19 with openstack-packstack-2013.1.1-0.3.dev527.fc19.noarch and openstack-swift-1.8.0-2.fc19.noarch.

The problem appears to be in the packstack puppet template file keystone.conf.erb which contains the lines:

"""
[filter:keystone]
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
"""

Changing this in the /etc/swift/proxy-swift.conf to:
"""
[filter:keystone]
use = egg:swift#keystoneauth
"""

Results in the following error from swift-proxy-server:
"""
Traceback (most recent call last):
  File "/usr/bin/swift-proxy-server", line 22, in <module>
    run_wsgi(conf_file, 'proxy-server', default_port=8080, **options)
  File "/usr/lib/python2.7/site-packages/swift/common/wsgi.py", line 119, in run_wsgi
    init_request_processor(conf_file, app_section, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/swift/common/wsgi.py", line 246, in init_request_processor
    app = loadapp('config:%s' % conf_file, global_conf={'log_name': log_name})
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 247, in loadapp
    return loadobj(APP, uri, name=name, **kw)
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 271, in loadobj
    global_conf=global_conf)
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 296, in loadcontext
    global_conf=global_conf)
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 320, in _loadconfig
    return loader.get_context(object_type, name, global_conf)
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 450, in get_context
    global_additions=global_additions)
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 562, in _pipeline_app_context
    for name in pipeline[:-1]]
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 458, in get_context
    section)
  File "/usr/lib/python2.7/site-packages/paste/deploy/loadwsgi.py", line 514, in _context_from_explicit
    "No loader given in section %r" % section)
LookupError: No loader given in section 'filter:keystone'
"""

(I tried with the filter:keystone section before and after filter:authtoken).

Comment 17 Jonathan Barber 2013-07-03 09:40:40 UTC
My previous comment is bogus, I had a "." instead of a space between the "use" and "=", i.e.:
use.= egg:swift#keystoneauth

Changing the configuration to:
"use = egg:swift#keystoneauth"

works.

Apologies for the noise.

Comment 18 Pete Zaitcev 2013-09-03 14:32:24 UTC
So, do we still have a problem here or not, for Swift or Packastack?

Comment 19 Fedora Admin XMLRPC Client 2013-10-14 09:50:46 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 20 Fedora Admin XMLRPC Client 2013-10-14 09:51:35 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 21 Pete Zaitcev 2013-10-14 20:05:22 UTC
We have this in 1.10.0-0.1.rc1:

[filter:authtoken]
paste.filter_factory = keystoneclient.middleware.auth_token:filter_factory

Comment 22 Martina Kollarova 2014-06-30 16:46:13 UTC
The problem seems to be gone.