Bug 824034

Summary: auth_token middleware should be in its own subpackage
Product: [Fedora] Fedora Reporter: Chmouel Boudjnah <chmouel>
Component: openstack-keystoneAssignee: Alan Pevec <apevec>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: apevec, apevec, bfilippov, breu, jonathansteffan, markmc, matt_domsch, pbrady, p, rbryant
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-keystone-2012.1-5.fc17 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 832536 844508 (view as bug list) Environment:
Last Closed: 2012-07-26 03:59:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
proposed spec patch
none
proposed spec patch
none
final proposed patch
none
final final patch none

Description Chmouel Boudjnah 2012-05-22 15:37:28 UTC
Description of problem:

The auth_token middleware is a shared middleware used by different OpenStack components as WSGI middleware for validating credentials/tokens.

Currently a user needs to install the full python-keystone package to get only the middleware when installing a swift proxy (or glance controller etc...).

It would be nice if this is broken out to its own.

Comment 1 Fedora Admin XMLRPC Client 2012-05-22 17:34:15 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 2 Pádraig Brady 2012-05-23 15:04:54 UTC
So this package would just contain keystone/middleware/auth_token.py ?

The main reason one would split up is to minimize dependencies,
and this does seem reasonable.

I was wondering about splitting all of keystone/middleware
to a keystone-middleware package, but the other modules there
look to have more extensive dependencies:
s3_token,swift_auth on swift and ec2_token on nova.

Considering the reduced dependencies on a "swift proxy" node:

openstack-swift-proxy (through openstack-swift) depends on:
python-configobj  
python-eventlet >= 0.9.8
python-greenlet >= 0.3.1
python-netifaces  
python-paste-deploy1.5  
python-setuptools  
python-simplejson  
python-webob1.0  
pyxattr  

while python-keystone depends on:
MySQL-python  
python-crypto  
python-dateutil  
python-eventlet  
python-httplib2  
python-ldap  
python-lxml  
python-memcached  
python-migrate  
python-passlib  
python-paste  
python-paste-deploy1.5  
python-paste-script  
python-prettytable  
python-routes1.12  
python-setuptools  
python-sqlalchemy0.7  
python-webob1.0  

And it seems that auth_token should depend on:
python-webob1.0  
python-memcached  
python-iso8601

So that's a reduction of

MySQL-python  
python-crypto  
python-dateutil  
python-httplib2  
python-ldap  
python-lxml  
python-migrate  
python-passlib  
python-paste-script  
python-prettytable  
python-routes1.12  
python-sqlalchemy0.7  

p.s. if we don't make this split,
perhaps python-iso8601 should be added to python-keystone

Comment 3 Pádraig Brady 2012-05-23 15:26:50 UTC
BTW the guard around the import of memcache and iso8601 in auth_token.py seems weird. Shouldn't it be catching ImportError rather than NameError?

Comment 4 Alan Pevec 2012-05-23 15:31:04 UTC
> So this package would just contain keystone/middleware/auth_token.py ?

yes

> I was wondering about splitting all of keystone/middleware
> to a keystone-middleware package, but the other modules there
> look to have more extensive dependencies:
> s3_token,swift_auth on swift and ec2_token on nova.

yes, BTW swift middleware is going to be moved to swift
from http://eavesdrop.openstack.org/meetings/openstack-meeting/2012/openstack-meeting.2012-05-16-20.31.html

"Voted on "move swift_auth.py middleware from keystone to swift?" Results are, yes: 5"

> perhaps python-iso8601 should be added to python-keystone

I've fixed that in openstack-keystone-2012.1-2
http://pkgs.fedoraproject.org/gitweb/?p=openstack-keystone.git;a=commitdiff;h=edd22d669d2c0bb8d591d0da673ea536fa88a46a#patch4

> Shouldn't it be catching ImportError rather than NameError?

Let's file LP bug upstream.

Comment 5 Chmouel Boudjnah 2012-05-23 17:53:46 UTC
If you guys want to create subpackage for s3_token and ec2_token that would be ideal tool and indeed we are working on moving swift_auth to swift so no need to create a subpackage (perhaps another bug report when moved to swift).

PS: it should indeed be ImportError

Comment 6 Alan Pevec 2012-05-23 23:43:55 UTC
Created attachment 586483 [details]
proposed spec patch

Comment 7 Alan Pevec 2012-05-23 23:47:57 UTC
Created attachment 586484 [details]
proposed spec patch

Comment 8 Pádraig Brady 2012-05-24 09:21:50 UTC
The milestone bump is redundant and a bit confusing given there never was an essex rc3 for keystone. Otherwise patch looks good.

cheers

Comment 9 Alan Pevec 2012-05-24 12:37:14 UTC
(In reply to comment #8)
> The milestone bump is redundant and a bit confusing

Count me confused too, that was wrong line :) I meant to bump Release only of course.

One thing I missed, yum update on existing Keystone installation will not pull python-keystone-auth-token:

@@ -56,6 +56,9 @@ Group:            Applications/System
 # python-keystone added in 2012.1-0.2.e3
 Conflicts:      openstack-keystone < 2012.1-0.2.e3
 
+# to pull middleware on yum update
+Requires:       python-keystone-auth-token = %{version}-%{release}
+
 Requires:       python-eventlet
 Requires:       python-ldap
 Requires:       python-lxml

Comment 10 Alan Pevec 2012-05-24 12:39:56 UTC
Created attachment 586618 [details]
final proposed patch

Comment 11 Alan Pevec 2012-05-24 12:50:35 UTC
Created attachment 586621 [details]
final final patch

* yum update
Updating:
 openstack-keystone             noarch     2012.1-3.fc16
 python-keystone                noarch     2012.1-3.fc16
Installing for dependencies:
 python-keystone-auth-token     noarch     2012.1-3.fc16

* yum install python-keystone-auth-token
Installing:
 python-keystone-auth-token     noarch     2012.1-3.fc16
Updating:
 python-keystone                noarch     2012.1-3.fc16
Updating for dependencies:
 openstack-keystone             noarch     2012.1-3.fc16

Comment 12 Fedora Update System 2012-05-25 10:40:03 UTC
openstack-keystone-2012.1-3.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/FEDORA-2012-8283/openstack-keystone-2012.1-3.fc17

Comment 13 Fedora Update System 2012-05-26 07:04:32 UTC
Package openstack-keystone-2012.1-3.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing openstack-keystone-2012.1-3.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-8283/openstack-keystone-2012.1-3.fc17
then log in and leave karma (feedback).

Comment 14 Pádraig Brady 2012-05-31 11:52:23 UTC
This will need to be updated, as the package is missing two empty __init__.py files in the keystone/ and keystone/middleware/ directories

I tested with those in place and glance was able to start OK

Note these empty files will need to not conflict with those from python-keystone.
I've not looked into how best to achieve that.

Comment 15 Alan Pevec 2012-05-31 12:51:16 UTC
What about small subpackage python-keystone-common which contains only:
keystone/__init__.py*
keystone/middleware/__init__.py*

Comment 16 Alan Pevec 2012-05-31 13:11:42 UTC
(In reply to comment #15)
> keystone/middleware/__init__.py*

Bad idea, it contains "from keystone.middleware.core import *" and core.py imports from keystone.

Only clean way seems to move auth_token out of keystone.middleware but that requires changes in paste-deploy configs of all apps.
Might be worth to propose that change upstream.

Comment 17 Alan Pevec 2012-06-01 14:18:02 UTC
(In reply to comment #16)
> (In reply to comment #15)
> > keystone/middleware/__init__.py*
> 
> Bad idea, it contains "from keystone.middleware.core import *" and core.py
> imports from keystone.

I tried to get rid of that but that would require keystone.conf changes which is config{noreplace} so existing setups would be broken after update.

This is yet another reason why mixing user-configurable settings and paste-deploy is bad, but that's another story.

Only quick fix here is what Pádraig suggested:
let python-keystone-auth-token create empty __init__.py if missing in %post script.
That way it can stand alone and if full python-keystone gets installed, it will overwrite those empty files.

Comment 18 Alan Pevec 2012-06-11 21:13:45 UTC
> Only quick fix here is what Pádraig suggested:

http://pkgs.fedoraproject.org/gitweb/?p=openstack-keystone.git;a=commitdiff;h=55247fe77e53bfdf58c19078a3f289ba357bb0e3

Pádraig, I feel rather dirty after this, please review :)

Comment 19 Pádraig Brady 2012-06-12 00:31:07 UTC
Nice work on the triggerpostun edge case.
I learn something every day :)

The logic looks sound, so looks good to me

Comment 20 Alan Pevec 2012-06-13 10:31:02 UTC
Reopening, triggerpostun has a side-effect on el6 where keystone/__init__.py is patched to import parallel versions of few python libs.

Comment 21 Fedora Update System 2012-06-13 21:38:39 UTC
Package openstack-keystone-2012.1-4.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing openstack-keystone-2012.1-4.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-8283/openstack-keystone-2012.1-4.fc17
then log in and leave karma (feedback).

Comment 22 Alan Pevec 2012-06-15 18:31:47 UTC
(In reply to comment #20)
> Reopening, triggerpostun has a side-effect on el6 where keystone/__init__.py
> is patched to import parallel versions of few python libs.

The issue is that triggerpostun was running on upgrades, breaking python-keystone:
http://pkgs.fedoraproject.org/gitweb/?p=openstack-keystone.git;a=commitdiff;h=fa729f8c9e4761ebb3b51eb38030defe48f328bf

Comment 23 Fedora Update System 2012-06-16 22:25:30 UTC
openstack-keystone-2012.1-8.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/openstack-keystone-2012.1-8.el6

Comment 24 Fedora Update System 2012-06-16 23:59:34 UTC
Package openstack-keystone-2012.1-5.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing openstack-keystone-2012.1-5.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-8283/openstack-keystone-2012.1-5.fc17
then log in and leave karma (feedback).

Comment 25 Fedora Update System 2012-07-16 12:52:44 UTC
openstack-keystone-2012.1.1-1.el6 has been submitted as an update for Fedora EPEL 6.
https://admin.fedoraproject.org/updates/openstack-keystone-2012.1.1-1.el6

Comment 26 Fedora Update System 2012-07-16 14:08:07 UTC
openstack-keystone-2012.1.1-1.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/openstack-keystone-2012.1.1-1.fc17

Comment 27 Fedora Update System 2012-07-26 03:59:22 UTC
openstack-keystone-2012.1.1-1.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 28 Alan Pevec 2012-07-30 23:46:28 UTC
Fun never ends:

folsom-2 introduced new keystone-internal dependecy, breaking auth-token stand-alone:

 from keystone.openstack.common import jsonutils

and folsom-3 adds one more:

 from keystone.common import cms

I'll look into further subpackaging common parts as python-keystone-common but this is getting messy.

Comment 29 Alan Pevec 2012-07-30 23:50:35 UTC
Filed bug 844508 to track the issue described in comment 28

Comment 30 Fedora Update System 2012-07-31 17:01:12 UTC
openstack-keystone-2012.1.1-1.el6 has been pushed to the Fedora EPEL 6 stable repository.  If problems still persist, please make note of it in this bug report.