Description of problem: ----------------------- Upgrade controllers from RHOS-13 to RHOS-14 failed while running 'Run deployment ControllerAllNodesValidationDeployment' step. It failed to reach IP address from ExternalNetwork/ "[2018-09-13 15:07:14,299] (heat-config) [INFO] deploy_signal_transport=NO_SIGNAL", ', "[2018-09-13 15:07:14,299] (heat-config) [DEBUG] Running /var/lib/heat-config/heat-config-script/db67c5d6-e65b-4e5a-ba84-8cafec50496d", ', "[2018-09-13 15:17:51,437] (heat-config) [INFO] Trying to ping 10.0.0.112 for local network 10.0.0.0/24.", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "Ping to 10.0.0.112 failed. Retrying...", ', "FAILURE", ', "[2018-09-13 15:17:51,437] (heat-config) [DEBUG] 10.0.0.112 is not pingable. Local Network: 10.0.0.0/24", ', "[2018-09-13 15:17:51,437] (heat-config) [ERROR] Error running /var/lib/heat-config/heat-config-script/db67c5d6-e65b-4e5a-ba84-8cafec50496d. [1]", ', Also other IPs on br-isolated bridge were not reachable between controllers. After reboot connectivity got restored. Version-Release number of selected component (if applicable): ------------------------------------------------------------- openstack-neutron-common-12.0.3-2.el7ost.noarch openstack-neutron-fwaas-12.0.1-0.20180409052822.384366b.el7ost.noarch openstack-neutron-openvswitch-12.0.3-2.el7ost.noarch puppet-neutron-12.4.1-1.3aa3109git.el7ost.noarch python-neutron-12.0.3-2.el7ost.noarch openstack-neutron-lbaas-12.0.1-0.20180424200349.cdbf25c.el7ost.noarch python-neutron-fwaas-12.0.1-0.20180409052822.384366b.el7ost.noarch openstack-neutron-12.0.3-2.el7ost.noarch openstack-neutron-ml2-12.0.3-2.el7ost.noarch python-neutron-lbaas-12.0.1-0.20180424200349.cdbf25c.el7ost.noarch python2-neutronclient-6.7.0-1.el7ost.noarch python2-neutron-lib-1.13.0-1.el7ost.noarch openstack-neutron-openvswitch-12.0.3-2.el7ost.noarch openvswitch-2.9.0-56.el7fdp.x86_64 openvswitch-selinux-extra-policy-1.0-5.el7fdp.noarch puppet-vswitch-8.4.0-0.20180329150818.fd94e91.el7ost.noarch python-openvswitch-2.9.0-56.el7fdp.noarch Steps to Reproduce: ------------------- 1. Upgrade RHOS-13 uc to RHOS-14 2. Prepare images/repos for RHOS-14 overcloud 3. Prepare upgrade playbooks 4. Run upgrade of controller nodes: openstack overcloud upgrade run \ --stack overcloud \ --roles Controller --playbook all Actual results: --------------- Post upgrade playbook failed Expected results: ----------------- Controllers upgrade succeeds Additional info: ---------------- Virtual setup: 3controllers + 2computes + 3ceph
Any initial troubleshooting?
nope
Hi Yuri, Can you reproduce this? If so can we login and look around once you do? It would be the easiest way to diagnose what the issue might be. Thanks.
Master patch merged, I created backport at https://review.openstack.org/#/c/617335/
Verified with openstack-tripleo-heat-templates-9.0.1-0.20181013060906.el7ost.noarch On overcloud: ------------- openvswitch-selinux-extra-policy-1.0-9.el7fdp.noarch openvswitch2.10-2.10.0-28.el7fdp.1.x86_64 python-rhosp-openvswitch-2.10-0.1.el7ost.noarch python-openvswitch2.10-2.10.0-28.el7fdp.1.x86_64 rhosp-openvswitch-2.10-0.1.el7ost.noarch openstack-neutron-openvswitch-13.0.2-0.20180929022427.c7f970c.el7ost.noarch [root@controller-0 ~]# docker ps | grep openvsw 629eacd0c45a 192.168.24.1:8787/rhosp14/openstack-neutron-openvswitch-agent:2018-12-14.1 "kolla_start" 2 hours ago Up 2 hours (healthy) neutron_ovs_agent
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045