Bug 1572685 - Update of Overcloud failed during redeploy OSP 12
Summary: Update of Overcloud failed during redeploy OSP 12
Keywords:
Status: CLOSED DUPLICATE of bug 1572686
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 12.0 (Pike)
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: James Slagle
QA Contact: Arik Chernetsky
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-27 15:03 UTC by Stan Toporek
Modified: 2018-04-30 16:35 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-30 16:35:27 UTC
Target Upstream Version:


Attachments (Terms of Use)
Templates (898.10 KB, application/x-gzip)
2018-04-27 15:03 UTC, Stan Toporek
no flags Details

Description Stan Toporek 2018-04-27 15:03:19 UTC
Created attachment 1427723 [details]
Templates

Description of problem:

> according to ticket number ####, support suggested me to run an update commenting:
> #  - /home/stack/openstack-tripleo-heat-templates/environments/network-isolation.yaml
> First I tried to deploy commenting that line and the error ocurred. Now, even with this line uncommented, the problem occurs.

So if I understand the update history here, the latest two stack updates that were run was this command with these contents, first, having network-isolation.yaml commented out in answers.yaml?

~~~
openstack overcloud deploy -r /home/stack/templates/roles_data.yaml --answers-file ~/answers.yaml --ntp-server a.ntp.br,b.ntp.br,c.ntp.br,pool.ntp.br

answers.yaml:

templates: /home/stack/openstack-tripleo-heat-templates/
environments:
  - /home/stack/templates/node-info.yaml
  - /home/stack/templates/ports.yaml
  - /home/stack/templates/compute-hci.yaml
  - /home/stack/templates/overcloud_images.yaml
  - /home/stack/templates/environment-file-1.yaml
  - /home/stack/templates/cinder-dellps-config.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/cinder-backup.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-rgw.yaml
  - /home/stack/templates/storage-config.yaml
  - /home/stack/templates/ceph-config.yaml
  - /home/stack/templates/ceph-config-per_node.yaml
#  - /home/stack/openstack-tripleo-heat-templates/environments/network-isolation.yaml
  - /home/stack/templates/network-environment.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/tls-endpoints-public-dns.yaml
  - /home/stack/templates/rhel-registration/environment-rhel-registration.yaml
  - /home/stack/templates/rhel-registration/rhel-registration-resource-registry.yaml
  - /home/stack/templates/enable-tls.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml
  - /home/stack/templates/ironic.yaml
  - /home/stack/templates/fencing.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/services/octavia.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/auditd.yaml
  - /home/stack/templates/remove_manila.yaml
~~~


Then the stack failed, right? Then you tried re-running the same deploy command, but uncommented network-isolation in answers.yaml?


~~~
openstack overcloud deploy -r /home/stack/templates/roles_data.yaml --answers-file ~/answers.yaml --ntp-server a.ntp.br,b.ntp.br,c.ntp.br,pool.ntp.br

answers.yaml:

templates: /home/stack/openstack-tripleo-heat-templates/
environments:
  - /home/stack/templates/node-info.yaml
  - /home/stack/templates/ports.yaml
  - /home/stack/templates/compute-hci.yaml
  - /home/stack/templates/overcloud_images.yaml
  - /home/stack/templates/environment-file-1.yaml
  - /home/stack/templates/cinder-dellps-config.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/cinder-backup.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-rgw.yaml
  - /home/stack/templates/storage-config.yaml
  - /home/stack/templates/ceph-config.yaml
  - /home/stack/templates/ceph-config-per_node.yaml
  - /home/stack/openstack-tripleo-heat-templates/environments/network-isolation.yaml
  - /home/stack/templates/network-environment.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/tls-endpoints-public-dns.yaml
  - /home/stack/templates/rhel-registration/environment-rhel-registration.yaml
  - /home/stack/templates/rhel-registration/rhel-registration-resource-registry.yaml
  - /home/stack/templates/enable-tls.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml
  - /home/stack/templates/ironic.yaml
  - /home/stack/templates/fencing.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/services/octavia.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/auditd.yaml
  - /home/stack/templates/remove_manila.yaml
~~~


And that's where we are right now with regard to the stack deploy/update commands issued, and stack failure?


Then I assume, sometime previous to running these two latest deploy commands, that the most recent stack deploy/update command that was run SUCCESSFULLY, was actually this one below but before trying to comment out network-isolation?:


~~~
openstack overcloud deploy -r /home/stack/templates/roles_data.yaml --answers-file ~/answers.yaml --ntp-server a.ntp.br,b.ntp.br,c.ntp.br,pool.ntp.br

answers.yaml:

templates: /home/stack/openstack-tripleo-heat-templates/
environments:
  - /home/stack/templates/node-info.yaml
  - /home/stack/templates/ports.yaml
  - /home/stack/templates/compute-hci.yaml
  - /home/stack/templates/overcloud_images.yaml
  - /home/stack/templates/environment-file-1.yaml
  - /home/stack/templates/cinder-dellps-config.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/cinder-backup.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-ansible.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/ceph-ansible/ceph-rgw.yaml
  - /home/stack/templates/storage-config.yaml
  - /home/stack/templates/ceph-config.yaml
  - /home/stack/templates/ceph-config-per_node.yaml
  - /home/stack/openstack-tripleo-heat-templates/environments/network-isolation.yaml
  - /home/stack/templates/network-environment.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/tls-endpoints-public-dns.yaml
  - /home/stack/templates/rhel-registration/environment-rhel-registration.yaml
  - /home/stack/templates/rhel-registration/rhel-registration-resource-registry.yaml
  - /home/stack/templates/enable-tls.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/services/ironic.yaml
  - /home/stack/templates/ironic.yaml
  - /home/stack/templates/fencing.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/services/octavia.yaml
  - /usr/share/openstack-tripleo-heat-templates/environments/auditd.yaml
  - /home/stack/templates/remove_manila.yaml
~~~


Do I understand correctly the deploy command/template history here?   I ask these questions to make sure I'm not missing anything here. And also there's quite a few templates to parse through. Your knowledge of the history here might speed things up.


Thank you,
-Andrew

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:
Stack overcloud UPDATE_FAILED 

overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution:
  resource_type: OS::Mistral::ExternalResource
  physical_resource_id: 5e18c1b6-4f88-44c5-983c-9ca4c8df5387
  status: CREATE_FAILED
  status_reason: |
    resources.WorkflowTasks_Step2_Execution: ERROR
overcloud.Networks.InternalNetwork.InternalApiNetwork:
  resource_type: OS::Neutron::Net
  physical_resource_id: 
  status: CREATE_FAILED
  status_reason: |
    Conflict: resources.InternalApiNetwork: Unable to create the flat network. Physical network internal_api is in use.
    Neutron server returns request_ids: ['req-2966d94f-8ef8-47f8-b5c2-825fa312f774']
overcloud.Networks.StorageMgmtNetwork.StorageMgmtNetwork:
  resource_type: OS::Neutron::Net
  physical_resource_id: 
  status: CREATE_FAILED
  status_reason: |
    Conflict: resources.StorageMgmtNetwork: Unable to create the flat network. Physical network storage_mgmt is in use.
    Neutron server returns request_ids: ['req-ad21eb7e-acb1-4598-9fdb-5cb2dfb9755e']
overcloud.Networks.StorageNetwork.StorageNetwork:
  resource_type: OS::Neutron::Net
  physical_resource_id: 
  status: CREATE_FAILED
  status_reason: |
    Conflict: resources.StorageNetwork: Unable to create the flat network. Physical network storage is in use.
    Neutron server returns request_ids: ['req-4e86fd73-d963-4b8c-bedb-cae28b35b0bb']
overcloud.Networks.TenantNetwork.TenantNetwork:
  resource_type: OS::Neutron::Net
  physical_resource_id: 
  status: CREATE_FAILED
  status_reason: |
    Conflict: resources.TenantNetwork: Unable to create the flat network. Physical network tenant is in use.
    Neutron server returns request_ids: ['req-44b521e9-387f-4bbd-a8cf-d05a7ce583fc']
overcloud.Networks.ExternalNetwork.ExternalNetwork:
  resource_type: OS::Neutron::Net
  physical_resource_id: 
  status: CREATE_FAILED
  status_reason: |
    Conflict: resources.ExternalNetwork: Unable to create the flat network. Physical network external is in use.
    Neutron server returns request_ids: ['req-89b07cd7-8662-47b2-992b-6a0a3e7c861a']
Heat Stack update failed.
Heat Stack update failed.

Expected results:

Overcloud update.
Additional info:

Comment 1 Alex Schultz 2018-04-30 16:35:27 UTC

*** This bug has been marked as a duplicate of bug 1572686 ***


Note You need to log in before you can comment on or make changes to this bug.