Created attachment 1289621 [details] install-undercloud.log Description of problem: minor update for OSP11 was failed on overcloud upgrade stage: compute node http://pastebin.test.redhat.com/495720 compute node cannot reach external networks [heat-admin@compute-0 ~]$ ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. ^C --- 8.8.8.8 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms on undercloud node NAT table: [stack@undercloud-0 ~]$ sudo iptables -nL -t nat Chain PREROUTING (policy ACCEPT) target prot opt source destination nova-api-PREROUTING all -- 0.0.0.0/0 0.0.0.0/0 DOCKER all -- 0.0.0.0/0 0.0.0.0/0 ADDRTYPE match dst-type LOCAL Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination nova-api-OUTPUT all -- 0.0.0.0/0 0.0.0.0/0 DOCKER all -- 0.0.0.0/0 !127.0.0.0/8 ADDRTYPE match dst-type LOCAL Chain POSTROUTING (policy ACCEPT) target prot opt source destination nova-api-POSTROUTING all -- 0.0.0.0/0 0.0.0.0/0 nova-postrouting-bottom all -- 0.0.0.0/0 0.0.0.0/0 MASQUERADE all -- 172.17.0.0/16 0.0.0.0/0 Chain DOCKER (2 references) target prot opt source destination RETURN all -- 0.0.0.0/0 0.0.0.0/0 Chain nova-api-OUTPUT (1 references) target prot opt source destination Chain nova-api-POSTROUTING (1 references) target prot opt source destination Chain nova-api-PREROUTING (1 references) target prot opt source destination Chain nova-api-float-snat (1 references) target prot opt source destination Chain nova-api-snat (1 references) target prot opt source destination nova-api-float-snat all -- 0.0.0.0/0 0.0.0.0/0 Chain nova-postrouting-bottom (1 references) target prot opt source destination nova-api-snat all -- 0.0.0.0/0 0.0.0.0/0 Version-Release number of selected component (if applicable): OSP11 + rhel7.4 How reproducible: Steps to Reproduce: 1.deploy osp11 using infrared infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa --cleanup yes && infrared virsh -v --host-address 10.9.76.22 --host-key ~/.ssh/id_rsa --topology-nodes undercloud:1,controller:1,compute:1 -e override.controller.cpu=8 -e override.controller.memory=32768 -e override.undercloud.disks.disk1.size=100G && infrared tripleo-undercloud --version 11 --images-task=rpm && infrared tripleo-overcloud -v --introspect yes --tagging yes --post no --deployment-files virt --version 11 --deploy yes 2. run sudo rhos-release 11 -r 7.4 on undercloud && overcloud nodes 3.perform minor update: https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/11/html/upgrading_red_hat_openstack_platform/ Actual results: overcloud update failed Expected results: overcloud update passed Additional info:
This bugzilla has been removed from the release and needs to be reviewed and Triaged for another Target Release.
Related to https://bugzilla.redhat.com/show_bug.cgi?id=1463375 ?
maybe. I didn't checked nat rules after undercloud upgrade , because upgrade failed only on compute node, after controller node upgrade was finished
Hi, could we have the sos-report from the undercloud ?
reproduced
Created attachment 1291977 [details] sosreport from undercloud
Hi, the sos report looks a little empty. As a side note, this is particular to test environments, I don't think we've seen that before on non virtual test environment. But it would be nice to get to the bottom of it. As a workaround I run this after the upgrade steps: if ! /usr/sbin/ip a | grep vlan10; then sudo ifup ifcfg-vlan10 fi if ! sudo /usr/sbin/iptables -L BOOTSTACK_MASQ -nvx -t nat | grep 10.0.0.0; then sudo iptables -t nat -A BOOTSTACK_MASQ -o eth0 -s 10.0.0.0/24 -j MASQUERADE fi I think a similar (need to adjust to particular infrared setup) command should help you move forward.
yep, I have w/a for that issue and this isn't blocker.
Hi Artem, putting low priority on this one. We will look at it as time permit. Thanks for the report, if you have the complete sos-report that may be handy when we look back at it.
(In reply to Sofer Athlan-Guyot from comment #9) > Hi Artem, > > putting low priority on this one. We will look at it as time permit. > > Thanks for the report, if you have the complete sos-report that may be handy > when we look back at it. The root cause of this bz is the same as for https://bugzilla.redhat.com/show_bug.cgi?id=1460116
Hi, Noted, will add it to the see also section for reference while we trying to spot the root cause of this other one. Thanks,
*** This bug has been marked as a duplicate of bug 1460116 ***