| Summary: | Stack deployment done twice with different network-environment files | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Robin Cernin <rcernin> | ||||
| Component: | openstack-heat | Assignee: | Steve Baker <sbaker> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Amit Ugol <augol> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 7.0 (Kilo) | CC: | cmedeiro, ggillies, mburns, mcornea, rhel-osp-director-maint, sbaker, shardy, skinjo, srevivo | ||||
| Target Milestone: | --- | ||||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2016-09-07 21:23:32 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Attachments: |
|
||||||
Created attachment 1196098 [details]
recovery.py script used to diagnose and fix this issue
|
Description of problem: We have two network-environment files, stage and prod. Originally the overcloud was deployed using the prod network-environment files. Then someone has run the overcloud deployment on top of prod environment with stage network-environment file. Which created new subnets and networks related to stage environment. Undercloud then see duplicated neutron networks from both stage and prod environment: $ neutron subnet-list +--------------------------------------+---------------------+-----------------+-----------------------------------------------------+ | id | name | cidr | allocation_pools | +--------------------------------------+---------------------+-----------------+-----------------------------------------------------+ | 22dc1807-5669-4a3b-9578-506bd3f7619a | storage_mgmt_subnet | 192.168.202.0/24 | {"start": "192.168.202.11", "end": "192.168.202.250"} | | 32f483ec-9306-4bb5-a5e6-037c09bfda91 | tenant_subnet | 192.168.201.0/24 | {"start": "192.168.201.11", "end": "192.168.201.250"} | | 41802666-6d63-4611-bef4-e5b364553565 | internal_api_subnet | 192.168.204.0/24 | {"start": "192.168.204.11", "end": "192.168.204.250"} | | 45876116-a72b-4f05-92f7-091a2fd68ef8 | storage_subnet | 192.168.103.0/24 | {"start": "192.168.103.11", "end": "192.168.103.250"} | | 47f66366-221d-4ca1-81d7-736c4dc62650 | internal_api_subnet | 192.168.104.0/24 | {"start": "192.168.104.11", "end": "192.168.104.250"} | | 4bd8c967-f99c-49a1-9ecb-a5559e0779d4 | | 10.1.38.0/24 | {"start": "10.1.38.20", "end": "10.1.38.120"} | | 6cc46f60-5a23-45b2-9d98-fb88075a886b | storage_mgmt_subnet | 192.168.102.0/24 | {"start": "192.168.102.11", "end": "192.168.102.250"} | | a7ce3bfe-0779-4b0d-9188-0d025e022042 | external_subnet | 10.2.188.0/24 | {"start": "10.2.188.11", "end": "10.2.188.253"} | | b2bdc779-c946-4782-9977-21e789fd2be9 | external_subnet | 10.2.188.0/24 | {"start": "10.2.188.11", "end": "10.2.188.253"} | | c83303c7-571e-4933-9d4b-eaf4f4cd519b | tenant_subnet | 192.168.101.0/24 | {"start": "192.168.101.11", "end": "192.168.101.250"} | +--------------------------------------+---------------------+-----------------+-----------------------------------------------------+ We have deleted all the duplicate subnets that matched the incorrect (staging) settings, and attempted a full stack update, unfortunately we hit the following error: Deploying templates in the directory /home/stack/centralci-openstack-templates/templates Stack failed with status: resources.Networks: resources.StorageNetwork: Conflict: resources.StorageSubnet: Unable to complete operation on subnet 45876116-a72b-4f05-92f7-091a2fd68ef8. One or more ports have an IP allocation from this subnet. Version-Release number of selected component (if applicable): openstack-heat-api-2015.1.4-1.el7ost.noarch openstack-heat-api-cfn-2015.1.4-1.el7ost.noarch openstack-heat-api-cloudwatch-2015.1.4-1.el7ost.noarch openstack-heat-common-2015.1.4-1.el7ost.noarch openstack-heat-engine-2015.1.4-1.el7ost.noarch openstack-heat-templates-0-0.8.20150605git.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-128.el7ost.noarch How reproducible: Deploy overcloud with two different environment files on top of each other. Steps to Reproduce: 1. See above. 2. 3. We need an fix to Heat DB to make sure when we run the update next time, it will not try to recreate the networks and will allow us to complete the update. Thank you.