Description of problem: During update from 7.0 to 7.1 on HA deployment(3 x ctrls + 1 x compute) with network isolation the overcloud instances lose connectivity via their floating IPs. Steps to Reproduce: 1. Deploy 7.0 openstack overcloud deploy --templates ~/templates-7.0/my-overcloud -e ~/templates-7.0/my-overcloud/environments/network-isolation.yaml -e ~/templates-7.0/network-environment.yaml --control-scale 3 --compute-scale 1 --ntp-server clock.redhat.com --libvirt-type qemu 2. Create external network, tenant network, router on the overcloud network. 3. Boot instance and assign it a floating IP on the external network 4. Update undercloud to 7.1 5. Update stack openstack overcloud update stack overcloud -i --templates ~/templates-7.1/my-overcloud -e ~/templates-7.1/my-overcloud/overcloud-resource-registry-puppet.yaml -e ~/templates-7.1/my-overcloud/environments/network-isolation.yaml -e ~/templates-7.1/network-environment.yaml -e ~/templates-7.1/update.yaml 6. Check connectivity to the instance floating IP Actual results: There's no connectivity during the update. Expected results: The instance keeps connectivity during the update. Additional info: I see the l3 agent ha_state in different states during update: active on 2 nodes, active on all of the nodes. At the end the update fails and there's a single alive l3 agent but it's got a standby ha_state. stack@instack:~>>> neutron l3-agent-list-hosting-router tenant-router +--------------------------------------+------------------------------------+----------------+-------+----------+ | id | host | admin_state_up | alive | ha_state | +--------------------------------------+------------------------------------+----------------+-------+----------+ | aef76e90-1157-46cf-a805-2059975639f8 | overcloud-controller-1.localdomain | True | :-) | active | | 2b195c24-65a1-4923-a506-5dba365d7d5c | overcloud-controller-0.localdomain | True | :-) | standby | | f8e2331e-fc16-4c7b-a8f3-4a51fd0e3406 | overcloud-controller-2.localdomain | True | :-) | active | +--------------------------------------+------------------------------------+----------------+-------+----------+ stack@instack:~>>> neutron l3-agent-list-hosting-router tenant-router +--------------------------------------+------------------------------------+----------------+-------+----------+ | id | host | admin_state_up | alive | ha_state | +--------------------------------------+------------------------------------+----------------+-------+----------+ | aef76e90-1157-46cf-a805-2059975639f8 | overcloud-controller-1.localdomain | True | :-) | active | | 2b195c24-65a1-4923-a506-5dba365d7d5c | overcloud-controller-0.localdomain | True | :-) | active | | f8e2331e-fc16-4c7b-a8f3-4a51fd0e3406 | overcloud-controller-2.localdomain | True | :-) | active | +--------------------------------------+------------------------------------+----------------+-------+----------+ stack@instack:~>>> neutron l3-agent-list-hosting-router tenant-router +--------------------------------------+------------------------------------+----------------+-------+----------+ | id | host | admin_state_up | alive | ha_state | +--------------------------------------+------------------------------------+----------------+-------+----------+ | aef76e90-1157-46cf-a805-2059975639f8 | overcloud-controller-1.localdomain | True | :-) | standby | | 2b195c24-65a1-4923-a506-5dba365d7d5c | overcloud-controller-0.localdomain | True | xxx | active | | f8e2331e-fc16-4c7b-a8f3-4a51fd0e3406 | overcloud-controller-2.localdomain | True | xxx | standby | +--------------------------------------+------------------------------------+----------------+-------+----------+
Created attachment 1092013 [details] controller0 sosreport
Created attachment 1092014 [details] controller1 sosreport
Created attachment 1092015 [details] controller2
Created attachment 1092019 [details] update.yaml
Results for a ping during update: --- 172.16.23.111 ping statistics --- 5409 packets transmitted, 5393 received, 0% packet loss, time 5413793ms rtt min/avg/max/mdev = 0.815/1.727/7.237/0.378 ms
hi, the doc text for this one would be the same as https://bugzilla.redhat.com/show_bug.cgi?id=1285079. I've copied here too as well.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:2651