Bug 1349120 - heat stack-delete not removing floating ip and subsequently failing
Summary: heat stack-delete not removing floating ip and subsequently failing
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-heat
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 10.0 (Newton)
Assignee: Steve Baker
QA Contact: Amit Ugol
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-22 18:15 UTC by Harry Rybacki
Modified: 2016-12-14 15:41 UTC (History)
8 users (show)

Fixed In Version: openstack-heat-7.0.0-0
Doc Type: Bug Fix
Doc Text:
Prior to this update, Heat would occasionally consider a `FloatingIP` resource deleted while the deletion was in fact still in progress. Consequently, resources that the `FloatingIP` depended on would sometimes fail to be deleted because the `FloatingIP` still existed. With this update, Heat now checks that the `FloatingIP` can no longer be found before considering the resource deleted, and stack deletes should proceed normally.
Clone Of:
Environment:
Last Closed: 2016-12-14 15:41:15 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2948 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC
OpenStack gerrit 333055 None None None 2016-06-22 23:14:40 UTC
Launchpad 1595270 None None None 2016-06-22 18:15:16 UTC

Description Harry Rybacki 2016-06-22 18:15:17 UTC
Description of problem:

Discovered while running a tripleo-ci ping test script on top of a newton deployment,
which consists of an overcloud populated by a heat stack (which created an instance, floatingips, router...).

The heat stack-delete failed on Newton each time. [1]
By digging into the heat engine and neutron server logs, we can see that heat asks to delete the floatingips and then router interface directly after[2].
But if you look at the neutron server log [3], it shows that the request for deleting the floating is done few seconds after.
The remove_router_interface occurred at 14:12:52.594 and the request for the deleting the floating ip at 2016-06-22 14:12:54.510

How reproducible:

Easily

Steps to Reproduce:
1. Deploy OpenStack (Newton) with TripleO 
2. ssh into the undercloud as the stack user
3. Execute a pingtest/validate the overcloud[4]
  - creating a `tenant-stack` on the overcloud for testing and then attempting to delete it during cleanup
4. $ source overcloudrc
5. $ openstack stack list

Actual results:

+--------------------------------------+--------------+---------------+---------------------+---------------------+
| id                                   | stack_name   | stack_status  | creation_time       | updated_time        |
+--------------------------------------+--------------+---------------+---------------------+---------------------+
| 96d21224-f532-457f-be6f-e04d836a2b05 | tenant-stack | DELETE_FAILED | 2016-06-22T13:39:38 | 2016-06-22T13:41:11 |
+--------------------------------------+--------------+---------------+---------------------+---------------------+

Expected results:

+--------------------------------------+--------------+---------------+---------------------+---------------------+
| id                                   | stack_name   | stack_status  | creation_time       | updated_time        |
+--------------------------------------+--------------+---------------+---------------------+---------------------+
| 96d21224-f532-457f-be6f-e04d836a2b05 | tenant-stack | UPDATE_COMPLETE | 2016-06-22T13:39:38 | 2016-06-22T13:41:11 |
+--------------------------------------+--------------+---------------+---------------------+---------------------+

Additional info:


[1] - https://paste.fedoraproject.org/383244/66039931/
[2] - https://paste.fedoraproject.org/383279/66072051/
[3] - https://paste.fedoraproject.org/383325/10212146/
[4] - https://paste.fedoraproject.org/383404/61838514/

Comment 2 Steve Baker 2016-06-23 21:27:52 UTC
The fix for this has landed upstream

Comment 4 Jaromir Coufal 2016-10-13 18:44:28 UTC
Steve, can you please help moving the bug to modified with version of package this is in?

Comment 8 errata-xmlrpc 2016-12-14 15:41:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.