Bug 1537268
| Summary: | After failed upgrade from 11-12, stack delete leaves networks in the undercloud and subsequent fresh deploy fails | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Ken Savich <ksavich> |
| Component: | openstack-heat | Assignee: | Zane Bitter <zbitter> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Ronnie Rasouli <rrasouli> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 12.0 (Pike) | CC: | ksavich, mburns, rhel-osp-director-maint, sbaker, shardy, srevivo, therve |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-04-11 12:31:38 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
That... shouldn't happen, I agree. Workaround would obviously be to remove the networks manually. Can you upload some heat-engine logs so we can try to see what might have happened? Yes, I did end up removing the networks manually. Unfortunately, however, I don't have the logs as this was a virtual machine with low disk space and I ended up cleaning /var/log pretty thoroughly. I'll do my best to recreate, if possible. Please reopen if that reappears. |
Description of problem: After a failed upgrade from 11.z to 12, `openstack stack delete overcloud` doesn't remove networks from the undercloud, which causes subsequent deployments to fail. Version-Release number of selected component (if applicable): openstack-heat-engine-9.0.1-3.el7ost.noarch How reproducible: Unsure Steps to Reproduce: 1.Upgrade OSP 11 undercloud to 11.z, with an existing overcloud. 2.Attempt to upgrade to OSP 12 3.Delete existing overcloud and attempt fresh deployment Actual results: Deleting the overcloud will look successful, but further inspection will show the following: $ source $HOME/stackrc $ openstack network list +--------------------------------------+--------------+--------------------------------------+ | ID | Name | Subnets | +--------------------------------------+--------------+--------------------------------------+ | 2a053388-b377-4d6a-bbee-81cbf6a937bf | tenant | d7fd90bd-6330-4dc1-9dba-7361d24bda06 | | 4771f43d-7db9-4ee3-96d2-f80fec7af632 | external | 074287e3-646a-4f98-8a48-5c904fe3b493 | | 4c830304-c4d5-4050-9b98-7ba8ad0b8e9b | ctlplane | 10d7c718-d689-44e3-a940-5e75b60f8b0f | | 72e07e0a-c0ec-472b-b865-870510f869c6 | private | 100167b9-2f87-4adf-b0f3-463b6951e02f | | ba72eebf-42a9-4e69-b17c-7b9e171007ae | internal_api | 77ddbb34-1572-4f09-94ef-dad97f2dd331 | | c20bf9f4-1383-40f6-aebb-fbd115893727 | storage_mgmt | 3f119b26-e1b6-4e28-bab6-f4067263efab | | cf02c22a-81e5-4384-9a07-ad5d166e4555 | storage | 12cf6cc8-3253-48d6-be7c-8d4f30d5c9b6 | +--------------------------------------+--------------+--------------------------------------+ A fresh deployment will fail, like so: Stack overcloud CREATE_FAILED overcloud.Networks.ManagementNetwork: resource_type: OS::TripleO::Network::Management physical_resource_id: 1e8fa700-29cd-4ba5-bf8a-c55684fddfd2 status: CREATE_FAILED status_reason: | CREATE aborted overcloud.Networks.InternalNetwork.InternalApiNetwork: resource_type: OS::Neutron::Net physical_resource_id: status: CREATE_FAILED status_reason: | Conflict: resources.InternalApiNetwork: Unable to create the flat network. Physical network internal_api is in use. Neutron server returns request_ids: ['req-e2796d9c-0ba2-45ee-a541-05202f5960cf'] overcloud.Networks.StorageMgmtNetwork.StorageMgmtNetwork: resource_type: OS::Neutron::Net physical_resource_id: status: CREATE_FAILED status_reason: | Conflict: resources.StorageMgmtNetwork: Unable to create the flat network. Physical network storage_mgmt is in use. Neutron server returns request_ids: ['req-591aaf87-de0d-455e-af24-f1b8bd163888'] overcloud.Networks.StorageNetwork.StorageNetwork: resource_type: OS::Neutron::Net physical_resource_id: status: CREATE_FAILED status_reason: | Conflict: resources.StorageNetwork: Unable to create the flat network. Physical network storage is in use. Neutron server returns request_ids: ['req-d209db44-3daa-4181-b4a8-ec188661ab3c'] overcloud.Networks.TenantNetwork.TenantNetwork: resource_type: OS::Neutron::Net physical_resource_id: status: CREATE_FAILED status_reason: | Conflict: resources.TenantNetwork: Unable to create the flat network. Physical network tenant is in use. Neutron server returns request_ids: ['req-69065bb1-97c3-4dae-a9f2-dd330d61dd3d'] overcloud.Networks.ExternalNetwork.ExternalNetwork: resource_type: OS::Neutron::Net physical_resource_id: status: CREATE_FAILED status_reason: | Conflict: resources.ExternalNetwork: Unable to create the flat network. Physical network external is in use. Neutron server returns request_ids: ['req-92ea601f-68cd-4bae-8b34-e4fa93cdf177'] Heat Stack create failed. Expected results: The undercloud should only contain the ctl-plane network after an `openstack stack delete overcloud` Additional info: