Bug 1129742 - [CISCO RHEL-OSP] Cannot connect to glance-api
Summary: [CISCO RHEL-OSP] Cannot connect to glance-api
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-packstack
Version: 5.0 (RHEL 7)
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: z1
: 5.0 (RHEL 7)
Assignee: Martin Magr
QA Contact: nlevinki
URL:
Whiteboard:
: 1141567 (view as bug list)
Depends On:
Blocks: 1154145 1154162 1154159
TreeView+ depends on / blocked
 
Reported: 2014-08-13 14:58 UTC by Britt Houser
Modified: 2016-04-27 02:27 UTC (History)
10 users (show)

Fixed In Version: openstack-packstack-2014.1.1-0.41.dev1251.el7ost
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-09-30 17:51:14 UTC
nlevinki: needinfo+


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2014:1324 normal SHIPPED_LIVE openstack-packstack and openstack-puppet-modules bug fix advisory 2014-09-30 21:49:21 UTC
OpenStack gerrit 100671 None None None Never
OpenStack gerrit 118594 None None None Never

Description Britt Houser 2014-08-13 14:58:21 UTC
Description of problem:

After packstack install, iptables is configured to only allow connections to glance-api from the compute nodes.  keystone and novaapi are allowed remote connections, but not glance-api.  Remote clients (e.g. tempest) error out b/c they are not allowed to upload files to glance.

Version-Release number of selected component (if applicable):


How reproducible:

Every time.

Steps to Reproduce:
1.  Install packstack
2.  Copy keystonerc_admin to non-compute node
3.  Source keystonerc_admin and try to run glance commands.

Actual results:

[root@cvf13-server-5 ~(keystone_admin)]# iptables -L INPUT
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
nova-api-INPUT  all  --  anywhere             anywhere
ACCEPT     tcp  --  cvf13-server-5.cvfdmz.sdu  anywhere             multiport dports amqps,amqp /* 001 amqp incoming amqp_192.168.131.5 */
ACCEPT     tcp  --  cvf13-server-6.cvfdmz.sdu  anywhere             multiport dports amqps,amqp /* 001 amqp incoming amqp_192.168.131.6 */
ACCEPT     tcp  --  cvf13-server-7.cvfdmz.sdu  anywhere             multiport dports amqps,amqp /* 001 amqp incoming amqp_192.168.131.7 */
ACCEPT     tcp  --  cvf13-server-8.cvfdmz.sdu  anywhere             multiport dports amqps,amqp /* 001 amqp incoming amqp_192.168.131.8 */
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 8777 /* 001 ceilometer-api incoming ceilometer_api */
ACCEPT     tcp  --  cvf13-server-7.cvfdmz.sdu  anywhere             multiport dports iscsi-target,8776 /* 001 cinder incoming cinder_192.168.131.7 */
ACCEPT     tcp  --  cvf13-server-8.cvfdmz.sdu  anywhere             multiport dports iscsi-target,8776 /* 001 cinder incoming cinder_192.168.131.8 */
ACCEPT     tcp  --  cvf13-server-7.cvfdmz.sdu  anywhere             multiport dports armtechdaemon /* 001 glance incoming glance_192.168.131.7 */
ACCEPT     tcp  --  cvf13-server-8.cvfdmz.sdu  anywhere             multiport dports armtechdaemon /* 001 glance incoming glance_192.168.131.8 */
ACCEPT     tcp  --  anywhere             anywhere             multiport dports http /* 001 horizon 80  incoming */
ACCEPT     tcp  --  anywhere             anywhere             multiport dports commplex-main,openstack-id /* 001 keystone incoming keystone */
ACCEPT     tcp  --  cvf13-server-5.cvfdmz.sdu  anywhere             multiport dports mysql /* 001 mysql incoming mysql_192.168.131.5 */
ACCEPT     tcp  --  cvf13-server-6.cvfdmz.sdu  anywhere             multiport dports mysql /* 001 mysql incoming mysql_192.168.131.6 */
ACCEPT     tcp  --  cvf13-server-7.cvfdmz.sdu  anywhere             multiport dports mysql /* 001 mysql incoming mysql_192.168.131.7 */
ACCEPT     tcp  --  cvf13-server-8.cvfdmz.sdu  anywhere             multiport dports mysql /* 001 mysql incoming mysql_192.168.131.8 */
ACCEPT     tcp  --  anywhere             anywhere             multiport dports http /* 001 nagios incoming */
ACCEPT     tcp  --  cvf13-server-5.cvfdmz.sdu  anywhere             multiport dports 5666 /* 001 nagios-nrpe incoming nagios_nrpe */
ACCEPT     tcp  --  cvf13-server-5.cvfdmz.sdu  anywhere             multiport dports 9696 /* 001 neutron server incoming neutron_server_192.168.131.5_192.168.131.5 */
ACCEPT     tcp  --  cvf13-server-6.cvfdmz.sdu  anywhere             multiport dports 9696 /* 001 neutron server incoming neutron_server_192.168.131.5_192.168.131.6 */
ACCEPT     tcp  --  cvf13-server-7.cvfdmz.sdu  anywhere             multiport dports 9696 /* 001 neutron server incoming neutron_server_192.168.131.5_192.168.131.7 */
ACCEPT     tcp  --  cvf13-server-8.cvfdmz.sdu  anywhere             multiport dports 9696 /* 001 neutron server incoming neutron_server_192.168.131.5_192.168.131.8 */
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 8773,8774,8775 /* 001 novaapi incoming */
ACCEPT     tcp  --  anywhere             anywhere             multiport dports 6080 /* 001 novncproxy incoming */
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
ACCEPT     icmp --  anywhere             anywhere
ACCEPT     all  --  anywhere             anywhere
ACCEPT     tcp  --  anywhere             anywhere             state NEW tcp dpt:ssh
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited
[root@cvf13-server-5 ~(keystone_admin)]#

Expected results:

I would have expected iptables rules which allow any source to connect instead of restricting to only compute nodes.

Additional info:

Perhaps neutron-api would fall into this category too?  I am not sure b/c in order to get tempest to run, we just opened up the firewall entirely.

Comment 3 Flavio Percoco 2014-08-19 07:53:59 UTC
Hi Britt,

Thanks for the report.

I believe your expectation may be true for some deployments. It really depends on whether you trust/want Glance as a public service. Just until recent releases, it was not suppose to be exposed as such.

Since I think it can be exposed as a public service, I'd expect the same thing you expected. Glance can still be hidden by binding it to a private network.

I'll move this bug to `packstack` so it can get attention from the right guys.

Comment 4 satya routray 2014-09-03 16:03:33 UTC
Please let us know in which release the fix is going in!

Comment 5 Flavio Percoco 2014-09-15 07:16:05 UTC
*** Bug 1141567 has been marked as a duplicate of this bug. ***

Comment 8 nlevinki 2014-09-23 08:18:04 UTC
checked with rpm -qa | grep openstack-packstack
openstack-packstack-2014.1.1-0.41.dev1251.el7ost.noarch
openstack-packstack-puppet-2014.1.1-0.41.dev1251.el7ost.noarch

it failed due to ipatables. when I disable the iptable the glance command is working from remote host.
[root@cougar09 audit(keystone_admin)]# iptables -L
Chain INPUT (policy DROP)                         
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:ssh /* 001 QA incoming SSH */
ACCEPT     icmp --  anywhere             anywhere             /* 002 QA incoming ICMP */           
ACCEPT     all  --  anywhere             anywhere             state RELATED,ESTABLISHED /* 003 QA incoming related session exist the host */
ACCEPT     all  --  anywhere             anywhere             /* 004 QA incoming loopback */                                                
ACCEPT     udp  --  anywhere             anywhere             udp spt:domain /* 005 QA incoming DNS */                                      
ACCEPT     udp  --  anywhere             anywhere             udp spt:ntp /* 006 QA incoming NTP */                                         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
[root@cougar09 audit(keystone_admin)]# systemctl stop iptables.service 
[root@cougar09 audit(keystone_admin)]# iptables -L
Chain INPUT (policy ACCEPT)                       
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         


[root@dhcp160-249 ~(keystone_admin)]# glance -d image-list
^C... terminating glance client

AFTER DISABLING IPTABLES
 
[root@dhcp160-249 ~(keystone_admin)]# glance -d image-list
curl -i -X GET -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'User-Agent: python-glanceclient' -H 'Connection: keep-alive' -H 'X-Auth-Token: {SHA1}7a95fd581af81937674bae0918c37561c2463fdc' -H 'Content-Type: application/octet-stream' http://10.35.160.139:9292/v1/images/detail?sort_key=name&sort_dir=asc&limit=20

HTTP/1.1 200 OK
date: Tue, 23 Sep 2014 08:12:41 GMT
connection: keep-alive
content-type: application/json; charset=UTF-8
content-length: 481
x-openstack-request-id: req-b622c929-eea9-4dcf-b517-a49b5fbd6ab8

{"images": [{"status": "active", "deleted_at": null, "name": "cirros", "deleted": false, "container_format": "bare", "created_at": "2014-09-23T07:05:28", "disk_format": "qcow2", "updated_at": "2014-09-23T07:05:37", "min_disk": 0, "protected": false, "id": "5c84ccb7-324a-4095-a68a-b81b6e055ebf", "min_ram": 0, "checksum": "d972013792949d0d3ba628fbe8685bce", "owner": "7b08bc799f2f4768bfe621d396f13c01", "is_public": true, "virtual_size": null, "properties": {}, "size": 13147648}]}

+--------------------------------------+--------+-------------+------------------+----------+--------+
| ID                                   | Name   | Disk Format | Container Format | Size     | Status |
+--------------------------------------+--------+-------------+------------------+----------+--------+
| 5c84ccb7-324a-4095-a68a-b81b6e055ebf | cirros | qcow2       | bare             | 13147648 | active |
+--------------------------------------+--------+-------------+------------------+----------+--------+

Comment 9 Martin Magr 2014-09-23 09:36:30 UTC
It seems that none of the iptables rules was applied. It is not clear from comment #8, what steps you did to get to this state. Can you please paste steps to reproduce?

Comment 10 Martin Magr 2014-09-24 08:22:51 UTC
Tried all-in-one and multi-host and both worked for me. Port 9292 is accessible publicly.

Comment 11 nlevinki 2014-09-25 14:19:48 UTC
here are the steps 
1)Install RHEL7 on cougar09 and  dhcp160-249 servers
2) install openstack all-in-one using packstack installer on server cougar09
3) Install in server dhcp160-249  python libs so it will recognize glance 
yum -y install libxslt-devel libxml2-devel gcc python-devel openssl-devel gmp-devel libffi libffi-devel wget gcc
easy_install 'pip<1.5'
4) scp keystone_admin file from cougar09 to dhcp160-249 and source it.
5) run the command "glance -d image-list", waited but there was no response. 
6) disable iptables on cougar09
7) repeat step 5 and it worked.

Comment 12 nlevinki 2014-09-28 07:30:59 UTC
I installed it again and it is working, port 9292 is open. found my error .
the end-point was listening on ip address 192.168.100.x and not on 10.35.160.y.

Comment 14 errata-xmlrpc 2014-09-30 17:51:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1324.html

Comment 15 Britt Houser 2014-10-08 16:23:03 UTC
I have verified the fix:

# rpm -qa | grep packstack
openstack-packstack-2014.1.1-0.41.dev1251.el7ost.noarch
openstack-packstack-puppet-2014.1.1-0.41.dev1251.el7ost.noarch

After packstack run, iptables has the following entry:

# iptables -L | grep glance
ACCEPT     tcp  --  anywhere             anywhere             multiport dports armtechdaemon /* 001 glance incoming glance_API */


Note You need to log in before you can comment on or make changes to this bug.