Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 976063

Summary: attempt to re-authenticate on possible token expiry re-uses expired token
Product: Red Hat OpenStack Reporter: Eoghan Glynn <eglynn>
Component: python-novaclientAssignee: Jakub Ruzicka <jruzicka>
Status: CLOSED ERRATA QA Contact: Ami Jeain <ajeain>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 3.0CC: apevec, bperkins, dallan, eglynn, jruzicka, ndipanov, sclewis, sgordon, yeylon
Target Milestone: async   
Target Release: 3.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: python-novaclient-2.13.0-2.el6ost Doc Type: Bug Fix
Doc Text:
Attempts to re-authenticate on detecting possible token expiry actually re-used the expired token. As a result the novaclient instance would stop working once the original token had expired. By default tokens expire after 24 hours. An update has been made to ensure the old authentication token has been discarded before attempting re-authentication. As a result the novaclient instance continues to work following expiry of the original token.
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-07-10 15:41:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eoghan Glynn 2013-06-19 20:15:32 UTC
Description of problem:

The attempt to re-authenticate on possible token expiry actually re-uses the expired token, which is clearly bound to fail in the expired case.

As a result, unless the client explicitly handles the 401, a novaclient instance will stop working once the original token has expired (by default after 24 hours).


Version-Release number of selected component (if applicable):

python-novaclient-2.13.0-1.el6ost.noarch


How reproducible:

100%


Steps to Reproduce:
1. Set token.expiration to small value (e.g. 120s) in /etc/keystone/keystone.conf (default = 1 day)

2. sudo service openstack-keystone restart

3. any service that uses a long-lived novaclient will fail with 401 after a few minutes, e.g. if ceilometer installed:

  sudo service openstack-ceilometer-compute restart
  tail -f /var/log/ceilometer/compute.log


Actual results:
2013-06-19 15:56:31    ERROR [ceilometer.nova_client] Unauthorized (HTTP 401)


Expected results:
No error.


Additional info:

This issue may have been previously masked by the recently discovered & fixed vulnerability whereby signed tokens where not being properly checked for expiry:

    https://bugs.launchpad.net/python-keystoneclient/+bug/1179615

Comment 1 Eoghan Glynn 2013-06-19 20:17:12 UTC
Fix proposed upstream:

  https://review.openstack.org/33685

Comment 2 Alan Pevec 2013-06-28 18:17:47 UTC
Pull request for the novaclient "stable/grizzly"
https://github.com/redhat-openstack/python-novaclient/pull/1

Eoghan, do you want Jakub (*client maintainer) to take it from there?

Comment 4 Ami Jeain 2013-07-07 11:19:18 UTC
changed the expiration to 120 seconds, and ran some commands after 2 min, all with HTTP 401 error:
# nova keypair-add --pub_key ~/.ssh/id_rsa.pub userkey
ERROR: Unauthorized (HTTP 401)

Brought it back to 1 day, restarted all keystone/heat services, and ran:

curl https://raw.github.com/openstack/heat/master/tools/nova_create_flavors.sh |bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
106   106  106   106    0     0    140      0 --:--:-- --:--:-- --:--:--   530
WARNING: This script now make no modifications to the current flavors
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
| ID | Name      | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public | extra_specs |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
| 1  | m1.tiny   | 512       | 0    | 0         |      | 1     | 1.0         | True      | {}          |
| 2  | m1.small  | 2048      | 20   | 0         |      | 1     | 1.0         | True      | {}          |
| 3  | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         | True      | {}          |
| 4  | m1.large  | 8192      | 80   | 0         |      | 4     | 1.0         | True      | {}          |
| 5  | m1.xlarge | 16384     | 160  | 0         |      | 8     | 1.0         | True      | {}          |
+----+-----------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
[root@cougar14 init.d(keystone_admin)]# nova keypair-add --pub_key ~/.ssh/id_rsa.pub userkey
[root@cougar14 init.d(keystone_admin)]#

Comment 6 errata-xmlrpc 2013-07-10 15:41:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1020.html