Bug 1041560

Summary: keystone iptables rule installed by packstack is more restrictive than rules for API services requiring a keystone token
Product: Red Hat OpenStack Reporter: Eoghan Glynn <eglynn>
Component: openstack-packstackAssignee: Francesco Vollero <fvollero>
Status: CLOSED ERRATA QA Contact: Nir Magnezi <nmagnezi>
Severity: high Docs Contact:
Priority: high    
Version: 4.0CC: ajeain, aortega, breeler, derekh, ichavero, mmagr, sgordon, yeylon
Target Milestone: z1Keywords: Triaged, ZStream
Target Release: 4.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-packstack-2013.2.1-0.22.dev956.el6ost Doc Type: Known Issue
Doc Text:
Currently, PackStack does not allow all hosts to access keystone. As a result, remote callers of various API services are unable to obtain a new token, preventing use of these API services from remote hosts. As a workaround, execute the following commands on the controller host: $ INDEX=$(sudo iptables -L | grep -A 20 'INPUT.*policy ACCEPT' | grep -- -- | grep -n keystone | cut -f1 -d:) $ sudo iptables -I INPUT $INDEX -p tcp --dport 35357 -j ACCEPT $ sudo service iptables save After doing this the remote callers of API services work correctly.
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-01-23 14:23:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Eoghan Glynn 2013-12-12 17:20:03 UTC
Description of problem:

Packstack installs iptables rules for several API services with the source address set to "anywhere" (i.e. nova-api, horizon, ceilometer-api, swift-proxy).

These rules appear to allow those services to be callable from anywhere.

But in order for clients to actually interact with any of those services, they would have to go to keystone first to get a token.

And therein lies the problem ... the iptables rule installed for keystone is source-constrained to just the allinone host (or to the compute nodes, if additional nodes are added, IIUC).

Hence the firewall rules for the API services are confusingly "half open". They appear to allow those service be invoked from outside the controller host, as the service ports are open to traffic originating from anywhere. But since the keystone port is not open to the same extent, in practical terms the iptables rules for the API services do not have the effect intended/implied. 

So external clients can only invoke on the API services if they already have a cached token, but not otherwise, and the failure mode isn't obvious to someone new to openstack.


Version-Release number of selected component (if applicable):

openstack-packstack-2013.2.1-0.18.dev934.el6ost


How reproducible:

100%


Steps to Reproduce:

1. Install packstack --allinone in the usual way

2. Install the CLI for any on the services mentioned above (nova, ceilometer, swift) onto another host

3. Source the keystonerc_admin generated by packstack on the second host

4. Attempt a basic CLI command:

   $ nova --debug flavor-list
   $ ceilometer --debug meter-list

   ... etc


Actual results:

The CLI invocation on the "open" service fails, e.g:

  $ nova flavor-list
  ERROR: [Errno 113] No route to host
  $ ceilometer meter-list
  Authorization Failed: Unable to establish connection to http://192.168.122.130:35357/v2.0/tokens


Expected results:

The CLI invocation on the "open" service should succeed.


Additional info:

N/A

Comment 2 Alvaro Lopez Ortega 2014-01-10 17:13:05 UTC
Francesco has been working on it. It seems that he hit some sort of issue with the firewall Puppet module.

Comment 5 Nir Magnezi 2014-01-15 15:10:19 UTC
Verified NVR: openstack-packstack-2013.2.1-0.22.dev956.el6ost.noarch

Tested as follows (Followed Comment #0):
1. Installed openstack via packstack on a single node.
2. Used additional node and installed the following packages:
python-cinderclient
python-neutronclient
python-keystoneclient
python-glanceclient
python-swiftclient
python-novaclient
python-ceilometerclient
python-heatclient

In addition, Created a keystonerc file to source.

3. The following commands were tested OK:
nova list
ceilometer meter-list
cinder list
glance image-list
neutron net-list
keystone endpoint-list
heat list

Commands Used in Comment #0
nova --debug flavor-list
ceilometer --debug meter-list

Comment 9 Lon Hohberger 2014-02-04 17:20:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2014-0046.html