Bug 1001751 - Able to assign one floating IP to more instances.
Summary: Able to assign one floating IP to more instances.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-nova
Version: 3.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: z2
: 4.0
Assignee: Brent Eagles
QA Contact: yfried
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-08-27 16:23 UTC by Jaroslav Henner
Modified: 2019-09-09 15:31 UTC (History)
9 users (show)

Fixed In Version: openstack-nova-2013.2-0.25.rc1.el6ost
Doc Type: Bug Fix
Doc Text:
Prior to this update, the Networking API implementation for associating floating IP addresses would not validate previous assignments of the same address, nor remove the association if one was found. As a result, floating IP addresses would appear to be assigned to multiple instances until background tasks reconciled the change in assignment. This issue has been resolved with this update, and Networking now checks for previous assignments of a floating IP and immediately removes the association upon establishing the new association. Consequently, the floating IP address information available to Compute clients (python-novaclient, Dashboard) is immediately updated, and the apparent duplicate assignment no longer occurs under these conditions.
Clone Of:
Environment:
Last Closed: 2014-03-04 20:12:40 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Bugzilla 985954 0 high CLOSED Floating IPs are assigned one by one and it takes much of time. 2021-02-22 00:41:40 UTC
Red Hat Product Errata RHBA-2014:0213 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OpenStack Platform 4 Bug Fix and Enhancement Advisory 2014-03-05 01:11:55 UTC

Internal Links: 985954

Description Jaroslav Henner 2013-08-27 16:23:44 UTC
Description of problem:
It is possible to $subj. Note I am using quantum.

Version-Release number of selected component (if applicable):
openstack-quantum.noarch              2013.1.3-1.el6ost    @puddle              
openstack-nova-api.noarch             2013.1.3-3.el6ost    @puddle              


How reproducible:
always

Steps to Reproduce:
`--> nova boot --key-name jhenner --image cirros-0.3.1-x86_64-disk.img --flavor m1.tiny --nic=net-id=`quantum net-list | awk '/notrouted-shared/ { print $2 }'` ssh_test --hint force_hosts=master-02.rhos.rhev.lab.eng.brq.redhat.com                                                                 
`--> nova boot --key-name jhenner --image cirros-0.3.1-x86_64-disk.img --flavor m1.tiny --nic=net-id=`quantum net-list | awk '/notrouted-shared/ { print $2 }'` ssh_test2 --hint force_hosts=master-02.rhos.rhev.lab.eng.brq.redhat.com 
`--> nova add-floating-ip ssh_test2 1.2.3.422 


Actual results:
`--> nova floating-ip-list                      
+----------+--------------------------------------+-------------+----------+
| Ip       | Instance Id                          | Fixed Ip    | Pool     |
+----------+--------------------------------------+-------------+----------+
| 1.2.3.11 | None                                 | None        | external |
| 1.2.3.12 | None                                 | None        | external |
| 1.2.3.18 | None                                 | None        | external |
| 1.2.3.20 | None                                 | None        | external |
| 1.2.3.21 | None                                 | None        | external |
| 1.2.3.22 | 9b84142b-6252-44a0-a213-360ac2c65d84 | 172.16.0.16 | external |
| 1.2.3.23 | None                                 | None        | external |
| 1.2.3.28 | None                                 | None        | external |
+----------+--------------------------------------+-------------+----------+
`--> nova list            
+--------------------------------------+-----------+--------+----------------------------------------+
| ID                                   | Name      | Status | Networks                               |
+--------------------------------------+-----------+--------+----------------------------------------+
| bdb07760-8f14-4a04-99e9-e17c290a18df | ssh_test  | ACTIVE | notrouted-shared=172.16.0.11, 1.2.3.22 |
| 9b84142b-6252-44a0-a213-360ac2c65d84 | ssh_test2 | ACTIVE | notrouted-shared=172.16.0.16, 1.2.3.22 |
+--------------------------------------+-----------+--------+----------------------------------------+


Expected results:
ERROR on CLI side

Additional info:
I am not sure this is nova or quantum bug.

Comment 2 Jaroslav Henner 2013-10-03 21:47:08 UTC
I am now not sure but IIRC, the problem is nova caching the floating ips.

Comment 3 Brent Eagles 2014-01-27 15:13:28 UTC
In the original description of this issue, it indicates that this happens "always". Is that the case? Do you have a system that exhibits this behavior that I can access. To date, I have not had a system that behaves this way except under very rare circumstances and even then only once.

Comment 4 Brent Eagles 2014-02-06 21:01:04 UTC
see needinfo request on comment 3.

Comment 5 Brent Eagles 2014-02-10 18:01:15 UTC
First note that above instructions are (I assume) invalid. The IP is first assigned to one VM and then the other. With this in mind, the booting of the VMs themselves is not particularly relevant, simply assigning the floating IP to one VM then the other is enough to create the apparent behavior.

A few notes:

- Changing floating IP assignment, especially with neutron, is a massively asynchronous activity. In grizzly floating ip assignment updates was much worse because it was not immediately removed from nova's cache of information when the request was made from nova->neutron. This is improved in havana so this behavior should not occur.

- The duplicate assignment after a point is only apparent. Once neutron (in this case quantum) processes the assignement, the functional details of the assignment is "done". After a period of time, the cache is reconciled and the floating IP will appear in its proper place.

- The code that would affect and improve this behavior is actually in the nova module that implements nova functions in terms of the neutron API so this issue *is* allocated to the correct component.

The relevant code was committed to havana in July 2013, 
commit 49fdad5b6d1e0878438571c2e9c0421bc522cb2e.

Using the same procedure in this BZ with the 4.0 code produces a more satisfactory result. I'm moving to modified with the package that seems to have been the first to have contained this code.

Comment 9 yfried 2014-02-18 20:15:51 UTC
please disregard comment 8

version
Havana on RHEL6.5

[root@cougar16 ~(keystone_admin)]# rpm -qa | grep neutron
python-neutronclient-2.3.1-3.el6ost.noarch
python-neutron-2013.2.2-1.el6ost.noarch
openstack-neutron-2013.2.2-1.el6ost.noarch
openstack-neutron-openvswitch-2013.2.2-1.el6ost.noarch
[root@cougar16 ~(keystone_admin)]# rpm -qa | grep nova
openstack-nova-conductor-2013.2.2-2.el6ost.noarch
openstack-nova-cert-2013.2.2-2.el6ost.noarch
python-nova-2013.2.2-2.el6ost.noarch
openstack-nova-api-2013.2.2-2.el6ost.noarch
openstack-nova-compute-2013.2.2-2.el6ost.noarch
openstack-nova-scheduler-2013.2.2-2.el6ost.noarch
python-novaclient-2.15.0-2.el6ost.noarch
openstack-nova-common-2013.2.2-2.el6ost.noarch
openstack-nova-console-2013.2.2-2.el6ost.noarch
openstack-nova-novncproxy-2013.2.2-2.el6ost.noarch


Floating IP is moved from instance A to instance B


[root@cougar16 ~(keystone_admin)]# nova add-floating-ip server1 10.35.166.3
[root@cougar16 ~(keystone_admin)]# nova list
+--------------------------------------+---------+--------+------------+-------------+---------------------------------+
| ID                                   | Name    | Status | Task State | Power State | Networks                        |
+--------------------------------------+---------+--------+------------+-------------+---------------------------------+
| 13bc8aa1-8991-4ad8-8cd0-e609c28617db | server1 | ACTIVE | None       | Running     | private=172.20.0.2, 10.35.166.3 |
| 2fbfea5e-ac22-4671-a3e0-cf2d7e178d83 | server2 | ACTIVE | None       | Running     | private=172.20.0.5              |
+--------------------------------------+---------+--------+------------+-------------+---------------------------------+
[root@cougar16 ~(keystone_admin)]# nova add-floating-ip server2 10.35.166.3
[root@cougar16 ~(keystone_admin)]# nova list
+--------------------------------------+---------+--------+------------+-------------+---------------------------------+
| ID                                   | Name    | Status | Task State | Power State | Networks                        |
+--------------------------------------+---------+--------+------------+-------------+---------------------------------+
| 13bc8aa1-8991-4ad8-8cd0-e609c28617db | server1 | ACTIVE | None       | Running     | private=172.20.0.2              |
| 2fbfea5e-ac22-4671-a3e0-cf2d7e178d83 | server2 | ACTIVE | None       | Running     | private=172.20.0.5, 10.35.166.3 |
+--------------------------------------+---------+--------+------------+-------------+---------------------------------+

Comment 12 errata-xmlrpc 2014-03-04 20:12:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0213.html


Note You need to log in before you can comment on or make changes to this bug.