Bug 1349120

Summary:	heat stack-delete not removing floating ip and subsequently failing
Product:	Red Hat OpenStack	Reporter:	Harry Rybacki <hrybacki>
Component:	openstack-heat	Assignee:	Steve Baker <sbaker>
Status:	CLOSED ERRATA	QA Contact:	Amit Ugol <augol>
Severity:	high	Docs Contact:
Priority:	high
Version:	10.0 (Newton)	CC:	jcoufal, mburns, mlopes, rhel-osp-director-maint, sbaker, shardy, srevivo, zbitter
Target Milestone:	rc	Keywords:	Triaged
Target Release:	10.0 (Newton)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-heat-7.0.0-0	Doc Type:	Bug Fix
Doc Text:	Prior to this update, Heat would occasionally consider a `FloatingIP` resource deleted while the deletion was in fact still in progress. Consequently, resources that the `FloatingIP` depended on would sometimes fail to be deleted because the `FloatingIP` still existed. With this update, Heat now checks that the `FloatingIP` can no longer be found before considering the resource deleted, and stack deletes should proceed normally.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-12-14 15:41:15 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Harry Rybacki 2016-06-22 18:15:17 UTC

Description of problem:

Discovered while running a tripleo-ci ping test script on top of a newton deployment,
which consists of an overcloud populated by a heat stack (which created an instance, floatingips, router...).

The heat stack-delete failed on Newton each time. [1]
By digging into the heat engine and neutron server logs, we can see that heat asks to delete the floatingips and then router interface directly after[2].
But if you look at the neutron server log [3], it shows that the request for deleting the floating is done few seconds after.
The remove_router_interface occurred at 14:12:52.594 and the request for the deleting the floating ip at 2016-06-22 14:12:54.510

How reproducible:

Easily

Steps to Reproduce:
1. Deploy OpenStack (Newton) with TripleO 
2. ssh into the undercloud as the stack user
3. Execute a pingtest/validate the overcloud[4]
  - creating a `tenant-stack` on the overcloud for testing and then attempting to delete it during cleanup
4. $ source overcloudrc
5. $ openstack stack list

Actual results:

+--------------------------------------+--------------+---------------+---------------------+---------------------+
| id                                   | stack_name   | stack_status  | creation_time       | updated_time        |
+--------------------------------------+--------------+---------------+---------------------+---------------------+
| 96d21224-f532-457f-be6f-e04d836a2b05 | tenant-stack | DELETE_FAILED | 2016-06-22T13:39:38 | 2016-06-22T13:41:11 |
+--------------------------------------+--------------+---------------+---------------------+---------------------+

Expected results:

+--------------------------------------+--------------+---------------+---------------------+---------------------+
| id                                   | stack_name   | stack_status  | creation_time       | updated_time        |
+--------------------------------------+--------------+---------------+---------------------+---------------------+
| 96d21224-f532-457f-be6f-e04d836a2b05 | tenant-stack | UPDATE_COMPLETE | 2016-06-22T13:39:38 | 2016-06-22T13:41:11 |
+--------------------------------------+--------------+---------------+---------------------+---------------------+

Additional info:


[1] - https://paste.fedoraproject.org/383244/66039931/
[2] - https://paste.fedoraproject.org/383279/66072051/
[3] - https://paste.fedoraproject.org/383325/10212146/
[4] - https://paste.fedoraproject.org/383404/61838514/

Comment 2 Steve Baker 2016-06-23 21:27:52 UTC

The fix for this has landed upstream

Comment 4 Jaromir Coufal 2016-10-13 18:44:28 UTC

Steve, can you please help moving the bug to modified with version of package this is in?

Comment 8 errata-xmlrpc 2016-12-14 15:41:15 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html