Bug 1752653

Summary: brownfield upgrades fail on public VIP allocation
Product: Red Hat OpenStack Reporter: Jeremy Agee <jagee>
Component: openstack-neutronAssignee: Harald Jensås <hjensas>
Status: CLOSED ERRATA QA Contact: Eran Kuris <ekuris>
Severity: medium Docs Contact:
Priority: medium    
Version: 15.0 (Stein)CC: amuller, bcafarel, bfournie, chrisw, hjensas, njohnston, scohen
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openstack-neutron-14.0.3-0.20191009170456.074e183.el8ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1760122 1760124 (view as bug list) Environment:
Last Closed: 2020-03-05 11:53:49 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1760122, 1760124    

Description Jeremy Agee 2019-09-16 20:27:27 UTC
Description of problem:
During the overcloud upgrade adding tls everywhere using haproxy-internal-tls-certmonger.yaml or haproxy-public-tls-certmonger.yaml the upgrade fails on the public VIP step.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. upgrade the undercloud to enabled novajoin
2. upgrade the overcloud to enable tls everwhere

Actual results:

Initial overcloud was deployed the following.
[stack@undercloud-0 ~]$ cat virt/public_vip.yaml 
parameter_defaults:
    PublicVirtualFixedIPs: [{'ip_address':'10.0.0.101'}]

upgraded overcloud is configured with the following.
[stack@undercloud-0 ~]$ cat virt/public_vip.yaml 
parameter_defaults:
    ControlFixedIPs: [{'ip_address':'192.168.24.101'}]
    PublicVirtualFixedIPs: [{'ip_address':'10.0.0.101'}]
    InternalApiVirtualFixedIPs: [{'ip_address':'172.17.1.101'}]
    StorageVirtualFixedIPs: [{'ip_address':'172.17.3.101'}]
    StorageMgmtVirtualFixedIPs: [{'ip_address':'172.17.4.101'}]
    RedisVirtualFixedIPs: [{'ip_address':'172.17.1.102'}]


openstack overcloud deploy \
--timeout 100 \
--templates /usr/share/openstack-tripleo-heat-templates \
--stack overcloud \
--libvirt-type kvm \
--ntp-server 10.0.0.7 \
-e /home/stack/virt/config_lvm.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml \
-e /home/stack/virt/network/network-environment.yaml \
-e /home/stack/virt/network/dvr-override.yaml \
-e /home/stack/virt/enable-tls.yaml \
-e /home/stack/virt/inject-trust-anchor.yaml \
-e /home/stack/virt/public_vip.yaml \
-e /home/stack/virt/cloud-names.yaml \
-e /home/stack/virt/hostnames.yml \
-e /home/stack/virt/nodes_data.yaml \
-e ~/containers-prepare-parameter.yaml \
-e /home/stack/virt/extra_templates.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/tls-everywhere-endpoints-dns.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/services/haproxy-internal-tls-certmonger.yaml \
-e /usr/share/openstack-tripleo-heat-templates/environments/ssl/enable-internal-tls.yaml \
--log-file overcloud_deployment_30.log


2019-09-10 01:20:25Z [overcloud-Networks-eksflc66ulsa]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-09-10 01:20:26Z [Networks]: UPDATE_COMPLETE  state changed
2019-09-10 01:20:30Z [PublicVirtualIP]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:30Z [InternalApiVirtualIP]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:31Z [StorageVirtualIP]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:31Z [StorageMgmtVirtualIP]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:31Z [RedisVirtualIP]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:33Z [overcloud-PublicVirtualIP-yltntd7bgob5]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-09-10 01:20:33Z [overcloud-InternalApiVirtualIP-vm46zomvu7zp]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-09-10 01:20:33Z [overcloud-StorageVirtualIP-vieiumqfkbgy]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-09-10 01:20:33Z [overcloud-InternalApiVirtualIP-vm46zomvu7zp.InternalApiPort]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:33Z [overcloud-StorageVirtualIP-vieiumqfkbgy.StoragePort]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:33Z [overcloud-StorageMgmtVirtualIP-qmfsvldj7367]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-09-10 01:20:34Z [overcloud-StorageMgmtVirtualIP-qmfsvldj7367.StorageMgmtPort]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:34Z [overcloud-RedisVirtualIP-ajdd3axzfecj]: UPDATE_IN_PROGRESS  Stack UPDATE started
2019-09-10 01:20:34Z [overcloud-RedisVirtualIP-ajdd3axzfecj.VipPort]: UPDATE_IN_PROGRESS  state changed
2019-09-10 01:20:34Z [overcloud-PublicVirtualIP-yltntd7bgob5]: UPDATE_COMPLETE  Stack UPDATE completed successfully
2019-09-10 01:20:35Z [PublicVirtualIP]: UPDATE_COMPLETE  state changed
2019-09-10 01:20:35Z [overcloud-InternalApiVirtualIP-vm46zomvu7zp.InternalApiPort]: UPDATE_FAILED  BadRequest: resources.InternalApiPort: Invalid input for operation: IP allocation requires 
subnets for network.

Neutron server returns request_ids: ['req-05f1161a-fe29-42de-906e-159c610fc65f']
2019-09-10 01:20:36Z [overcloud-StorageVirtualIP-vieiumqfkbgy.StoragePort]: UPDATE_FAILED  BadRequest: resources.StoragePort: Invalid input for operation: IP allocation requires subnets for 
network.
Neutron server returns request_ids: ['req-11120b3d-711f-4074-8895-844195c6339e']
2019-09-10 01:20:36Z [overcloud-StorageVirtualIP-vieiumqfkbgy]: UPDATE_FAILED  Resource UPDATE failed: BadRequest: resources.StoragePort: Invalid input for operation: IP allocation requires subnets for network.
Neutron server returns request_ids: ['req-11120b3d-711f-4074-8895-844195c6339e']
2019-09-10 01:20:36Z [overcloud-StorageMgmtVirtualIP-qmfsvldj7367.StorageMgmtPort]: UPDATE_FAILED  BadRequest: resources.StorageMgmtPort: Invalid input for operation: IP allocation requires subnets for network.
Neutron server returns request_ids: ['req-434378a1-d96d-418c-b6af-a0544b45bf46']
2019-09-10 01:20:36Z [overcloud-StorageMgmtVirtualIP-qmfsvldj7367]: UPDATE_FAILED  Resource UPDATE failed: BadRequest: resources.StorageMgmtPort: Invalid input for operation: IP allocation requires subnets for network.
Neutron server returns request_ids: ['req-434378a1-d96d-418c-b6af-a0544b45bf46']
2019-09-10 01:20:37Z [overcloud.StorageMgmtVirtualIP]: UPDATE_FAILED  BadRequest: resources.StorageMgmtVirtualIP.resources.StorageMgmtPort: Invalid input for operation: IP allocation requires subnets for network.
Neutron server returns request_ids: ['req-434378a1-d96d-418c-b6af-a0544b45bf46']
2019-09-10 01:20:37Z [overcloud]: UPDATE_FAILED  Resource UPDATE failed: BadRequest: resources.StorageMgmtVirtualIP.resources.StorageMgmtPort: Invalid input for operation: IP allocation requires subnets for network.
Neutron server returns request_ids: ['req-434378a1-d96d-418c-b6af-a0544b45bf46']
2019-09-10 01:20:37Z [overcloud.RedisVirtualIP]: UPDATE_FAILED  BadRequest: resources.RedisVirtualIP.resources.VipPort: Invalid input for operation: IP allocation requires subnets for network.
Neutron server returns request_ids: ['req-2909f8e1-e015-4628-b0c2-0da149bbcd01']
2019-09-10 01:20:37Z [overcloud]: UPDATE_FAILED  Resource UPDATE failed: BadRequest: resources.RedisVirtualIP.resources.VipPort: Invalid input for operation: IP allocation requires subnets f
or network.
Neutron server returns request_ids: ['req-2909f8e1-e015-4628-b0c2-0da149bbcd01']                                                                                                              
2019-09-10 01:20:37Z [overcloud.StorageVirtualIP]: UPDATE_FAILED  BadRequest: resources.StorageVirtualIP.resources.StoragePort: Invalid input for operation: IP allocation requires subnets fo
r network.
Neutron server returns request_ids: ['req-11120b3d-711f-4074-8895-844195c6339e']                                                                                                              
2019-09-10 01:20:37Z [overcloud]: UPDATE_FAILED  Resource UPDATE failed: BadRequest: resources.StorageVirtualIP.resources.StoragePort: Invalid input for operation: IP allocation requires sub
nets for network.
Neutron server returns request_ids: ['req-11120b3d-711f-4074-8895-844195c6339e']

 Stack overcloud/7a628716-bd2c-417b-b096-bccc9910b4cd UPDATE_FAILED 

overcloud.RedisVirtualIP.VipPort:
  resource_type: OS::Neutron::Port
  physical_resource_id: 238d0e8a-fdf6-4613-837a-68abb5d799b8
  status: UPDATE_FAILED
  status_reason: |
    BadRequest: resources.VipPort: Invalid input for operation: IP allocation requires subnets for network.
    Neutron server returns request_ids: ['req-2909f8e1-e015-4628-b0c2-0da149bbcd01']
overcloud.StorageMgmtVirtualIP.StorageMgmtPort:
  resource_type: OS::Neutron::Port
  physical_resource_id: cece30d3-78ed-4a86-9904-410bc0b83cb3
  status: UPDATE_FAILED
  status_reason: |
    BadRequest: resources.StorageMgmtPort: Invalid input for operation: IP allocation requires subnets for network.
    Neutron server returns request_ids: ['req-434378a1-d96d-418c-b6af-a0544b45bf46']
overcloud.StorageVirtualIP.StoragePort:
  resource_type: OS::Neutron::Port
  physical_resource_id: ae058966-d31a-4f60-8e44-430a90b205a7
  status: UPDATE_FAILED
  status_reason: |
    BadRequest: resources.StoragePort: Invalid input for operation: IP allocation requires subnets for network.
    Neutron server returns request_ids: ['req-11120b3d-711f-4074-8895-844195c6339e']
overcloud.InternalApiVirtualIP.InternalApiPort:
  resource_type: OS::Neutron::Port
  physical_resource_id: da162a07-eb8c-4002-a33f-e6ab705a0a7f
  status: UPDATE_FAILED
  status_reason: |
    BadRequest: resources.InternalApiPort: Invalid input for operation: IP allocation requires subnets for network.
    Neutron server returns request_ids: ['req-05f1161a-fe29-42de-906e-159c610fc65f']
Heat Stack update failed.


Expected results:


Additional info:

Comment 4 errata-xmlrpc 2020-03-05 11:53:49 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:0709