Bug 1257717
| Summary: | Environment files must be passed anew on every stack update | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Zane Bitter <zbitter> |
| Component: | openstack-heat | Assignee: | Zane Bitter <zbitter> |
| Status: | CLOSED ERRATA | QA Contact: | Alexander Chuzhoy <sasha> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | high | ||
| Version: | 7.0 (Kilo) | CC: | ddomingo, jprovazn, jtrowbri, jwaterwo, mburns, ochalups, rhel-osp-director-maint, sasha, sbaker, shardy, yeylon |
| Target Milestone: | z2 | Keywords: | Triaged, ZStream |
| Target Release: | 7.0 (Kilo) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-heat-2015.1.1-1.el7ost | Doc Type: | Enhancement |
| Doc Text: |
On a PATCH update (using the "-x" flag in the 'heat stack-update' command), the existing environment is now retained unless explicitly overridden. This is because the Orchestration service now re-uses other parts of the environment, not just the parameters that were passed previously and not overriden.
This feature was added because in the most common stack update cases, users prefer to maintain the current environment (including resource mappings and the like). This will also prevent any unintended changes in complex deployments whenever users forget to include the required environment files at stack creation time.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-10-08 12:20:45 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1261921 | ||
| Bug Blocks: | 1255931, 1256538, 1265010 | ||
|
Description
Zane Bitter
2015-08-27 18:39:59 UTC
*** Bug 1264159 has been marked as a duplicate of this bug. *** FailedQA Environment: openstack-heat-common-2015.1.1-4.el7ost.noarch openstack-heat-api-cfn-2015.1.1-4.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-62.el7ost.noarch openstack-heat-templates-0-0.6.20150605git.el7ost.noarch openstack-heat-api-cloudwatch-2015.1.1-4.el7ost.noarch openstack-heat-api-2015.1.1-4.el7ost.noarch python-heatclient-0.6.0-1.el7ost.noarch openstack-heat-engine-2015.1.1-4.el7ost.noarch The deployment command used: openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 -e <yaml> --ntp-server x.x.x.x --timeout 90 --compute-flavor compute --control-flavor control --rhel-reg --reg-method portal --reg-org <orgid> --reg-activation-key '<key>' The update command (simply doesn't have the yaml file): openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --ntp-server x.x.x.x --timeout 90 --compute-flavor compute --control-flavor control --rhel-reg --reg-method portal --reg-org <orgid> --reg-activation-key '<key>' Results in: Deploying templates in the directory /usr/share/openstack-tripleo-heat-templates Stack failed with status: resources.Networks: resources.StorageMgmtNetwork: Conflict: resources.StorageMgmtSubnet: Unable to complete operation on subnet 521b0677-7689-4649-b81e-2ef5818e78f7. One or more ports have an IP allocation from this subnet. ERROR: openstack Heat Stack update failed. My best guess is that we're not using the (python API equivalent of) the -x CLI option (to use PATCH instead of PUT to update the stack) in all of the "openstack overcloud ..." commands. I was able to get a very similar error: $ openstack overcloud deploy --templates --debug -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml 2>&1 | tee netlog.create <stack created by OC service configuration failed - probably wrong network setup> $ openstack overcloud deploy --templates --debug 2>&1 | tee netlog.update ... Stack failed with status: resources.Networks: resources.ExternalNetwork: Conflict: resources.ExternalSubnet: Unable to complete operation on subnet ed6c6677-9eeb-4ea5-af36-5345bbbfde27. One or more ports have an IP allocation from this subnet. This seems to be caused by same issue as https://bugzilla.redhat.com/show_bug.cgi?id=1255931#c11 - an extra env file used on create overrides resource_registry for some resources, then when this registry env file is missing on update, resource_registry is changed which causes replacement of these resources: [stack@instack ~]$ heat resource-list -n 5 overcloud|grep FAIL| ExternalSubnet | ed6c6677-9eeb-4ea5-af36-5345bbbfde27 | OS::Neutron::Subnet | DELETE_FAILED | 2015-09-21T14:57:03Z | ExternalNetwork | | StorageSubnet | ff9a3495-2e2b-4469-b7d7-c6f1898b8d9c | OS::Neutron::Subnet | DELETE_FAILED | 2015-09-21T14:57:03Z | StorageNetwork | | InternalApiSubnet | e44f6beb-1bfd-499c-84d2-cea05cad6db7 | OS::Neutron::Subnet | DELETE_FAILED | 2015-09-21T14:57:04Z | InternalNetwork | | StorageMgmtSubnet | c0e5c0e9-a101-4546-8424-2813aa99b852 | OS::Neutron::Subnet | DELETE_FAILED | 2015-09-21T14:57:04Z | StorageMgmtNetwork | | TenantSubnet | 975a9582-fc12-4383-bc56-8f567f35d712 | OS::Neutron::Subnet | DELETE_FAILED | 2015-09-21T14:57:04Z | TenantNetwork | | Networks | 8d0f4329-a0de-4df0-84aa-d3c3bc59c314 | OS::TripleO::Network | UPDATE_FAILED | 2015-09-21T15:20:54Z | | | ExternalNetwork | df2f773a-6ee9-478f-81aa-8ae4d4d955bf | OS::TripleO::Network::External | UPDATE_FAILED | 2015-09-21T15:20:57Z | Networks | | TenantNetwork | 826dd7dd-a0ad-4e61-8a25-5af914a8f0bb | OS::TripleO::Network::Tenant | UPDATE_FAILED | 2015-09-21T15:20:57Z | Networks | | StorageMgmtNetwork | a42fae15-ff4e-48f2-af4c-133d63cc80f9 | OS::TripleO::Network::StorageMgmt | UPDATE_FAILED | 2015-09-21T15:20:58Z | Networks | | InternalNetwork | d8b37fba-c722-4e64-8633-23285eb03a24 | OS::TripleO::Network::InternalApi | UPDATE_FAILED | 2015-09-21T15:20:59Z | Networks | | StorageNetwork | 1e7eb6b1-6ea7-45d2-8eb6-86f5291ef653 | OS::TripleO::Network::Storage | UPDATE_FAILED | 2015-09-21T15:20:59Z | Networks | [stack@instack ~]$ cd /usr/share/openstack-tripleo-heat-templates/ [stack@instack openstack-tripleo-heat-templates]$ grep -r TenantSubnet network/tenant.yaml: TenantSubnetName: network/tenant.yaml: TenantSubnet: network/tenant.yaml: name: {get_param: TenantSubnetName} [stack@instack openstack-tripleo-heat-templates]$ grep -r tenant.yaml environments/network-isolation.yaml: OS::TripleO::Network::Tenant: ../network/tenant.yaml environments/network-isolation.yaml: OS::TripleO::Controller::Ports::TenantPort: ../network/ports/tenant.yaml environments/network-isolation.yaml: OS::TripleO::Compute::Ports::TenantPort: ../network/ports/tenant.yaml [stack@instack openstack-tripleo-heat-templates]$ grep -r OS::TripleO::Network::Tenant overcloud-resource-registry-puppet.yaml: OS::TripleO::Network::Tenant: network/noop.yaml network/networks.yaml: type: OS::TripleO::Network::Tenant environments/network-isolation.yaml: OS::TripleO::Network::Tenant: ../network/tenant.yaml overcloud-resource-registry.yaml: OS::TripleO::Network::Tenant: network/noop.yaml related rpms: [root@instack rdomanager_oscplugin]# rpm -qa|grep osc python-rdomanager-oscplugin-0.0.10-5.el7ost.noarch [root@instack rdomanager_oscplugin]# rpm -qa|grep heat openstack-heat-engine-2015.1.1-4.el7ost.noarch openstack-heat-api-2015.1.1-4.el7ost.noarch openstack-heat-templates-0-0.6.20150605git.el7ost.noarch python-heatclient-0.6.0-1.el7ost.noarch openstack-tripleo-heat-templates-0.8.6-64.el7ost.noarch openstack-heat-common-2015.1.1-4.el7ost.noarch openstack-heat-api-cfn-2015.1.1-4.el7ost.noarch openstack-heat-api-cloudwatch-2015.1.1-4.el7ost.noarch heat-cfntools-1.2.8-2.el7.noarch IF default environment file is not passed on update operation at all, then update passes. To not pass the default env file, this patch for python-tuskarclient was used: [root@instack rdomanager_oscplugin]# diff -r v1/overcloud_deploy.py.orig v1/overcloud_deploy.py.changes 508c508,511 < resource_registry_path = os.path.join(tht_root, RESOURCE_REGISTRY_NAME) --- > resource_registry_path = os.path.join(tht_root, RESOURCE_REGISTRY_NAME) > environments = [resource_registry_path, env_path] > else: > environments = [] 510d512 < environments = [resource_registry_path, env_path] That looks right to me, it explains everything that we're seeing. We'll also need to check other commands (like "openstack overcloud update") and make sure they don't pass the default environment files on updates any more either. Since the issue we're now encountering is in a different component, I created bug 1265010 to cover that. I'm moving this openstack-heat bug back to MODIFIED because there's no issue that we know of with the Heat feature, it's purely client-side issues with rdo-manager specifically. Verified: Environment: openstack-heat-engine-2015.1.1-5.el7ost.noarch Deployed the stack with: openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --ceph-storage-scale 0 -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-ceph-external.yaml -e /home/stack/network-environment.yaml --ntp-server X.X.X.X --timeout 90 --compute-flavor compute --control-flavor control -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml --neutron-bridge-mappings datacentre:br-ex --neutron-network-vlan-ranges datacentre:<range> --neutron-disable-tunneling --neutron-network-type vlan Updated the stack with: openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --ntp-server X.X.X.X --timeout 90 --compute-flavor compute --control-flavor control --neutron-bridge-mappings datacentre:br-ex --neutron-network-vlan-ranges datacentre:<range> --neutron-disable-tunneling --neutron-network-type vlan Completed successfully. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:1865 |