Description of problem: Installation of the overcloud fails due to a misconfig of the masqueraded leaf in the undercloud. (The external access is required for NTP.) How reproducible: Always, with the configuration below: undercloud.conf: no DEFAULT/masquerade_network subnets = ctlplane-subnet,edge-subnet local_subnet = ctlplane-subnet enable_routed_networks = true [ctlplane-subnet] cidr = 172.16.0.0/24 dhcp_start = 172.16.0.20 dhcp_end = 172.16.0.120 inspection_iprange = 172.16.0.150,172.16.0.180 gateway = 172.16.0.1 masquerade = true [edge-subnet] cidr = 172.23.0.0/24 dhcp_start = 172.23.0.20 dhcp_end = 172.23.0.120 inspection_iprange = 172.23.0.150,172.23.0.180 gateway = 172.23.0.1 masquerade = true # iptables -L FORWARD Chain FORWARD (policy ACCEPT) target prot opt source destination neutron-filter-top all -- anywhere anywhere neutron-openvswi-FORWARD all -- anywhere anywhere ACCEPT all -- 10.0.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 10.0.0.0/24 ipv4 */ ACCEPT all -- 172.16.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 172.16.0.0/24 ipv4 */ ACCEPT all -- 172.23.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 172.23.0.0/24 ipv4 */ ACCEPT all -- 192.168.24.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 192.168.24.0/24 ipv4 */ ACCEPT all -- anywhere 10.0.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 10.0.0.0/24 ipv4 */ ACCEPT all -- anywhere 172.16.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 172.16.0.0/24 ipv4 */ ACCEPT all -- anywhere 172.16.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 172.23.0.0/24 ipv4 */ ACCEPT all -- anywhere 192.168.24.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 192.168.24.0/24 ipv4 */ REJECT all -- anywhere anywhere reject-with icmp-host-prohibited Actual results: We can see the following rule with a good comment, but an invalid CIDR. ACCEPT all -- anywhere 172.16.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 172.23.0.0/24 ipv4 */ Expected results: To have both leaves with the correct rules ACCEPT all -- anywhere 172.16.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 172.23.0.0/24 ipv4 */ ACCEPT all -- anywhere 172.23.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 172.23.0.0/24 ipv4 */ Additional info:
I added manually the rule to move forward, and I saw the same issue with the nat table: $ sudo iptables -t nat -L POSTROUTING Chain POSTROUTING (policy ACCEPT) target prot opt source destination RETURN all -- 192.168.124.0/24 base-address.mcast.net/24 RETURN all -- 192.168.124.0/24 255.255.255.255 MASQUERADE tcp -- 192.168.124.0/24 !192.168.124.0/24 masq ports: 1024-65535 MASQUERADE udp -- 192.168.124.0/24 !192.168.124.0/24 masq ports: 1024-65535 MASQUERADE all -- 192.168.124.0/24 !192.168.124.0/24 RETURN all -- 10.0.0.0/24 10.0.0.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return 10.0.0.0/24 ipv4 */ RETURN all -- 172.16.0.0/24 172.16.0.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return 172.16.0.0/24 ipv4 */ RETURN all -- 172.23.0.0/24 172.16.0.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return 172.23.0.0/24 ipv4 */ RETURN all -- 192.168.24.0/24 192.168.24.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return 192.168.24.0/24 ipv4 */ MASQUERADE all -- 10.0.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 10.0.0.0/24 ipv4 */ MASQUERADE all -- 172.16.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 172.16.0.0/24 ipv4 */ MASQUERADE all -- 172.23.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 172.23.0.0/24 ipv4 */ MASQUERADE all -- 192.168.24.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 192.168.24.0/24 ipv4 */ In addition to this rule: RETURN all -- 172.23.0.0/24 172.16.0.0/24 We have this one: RETURN all -- 172.16.0.0/24 172.16.0.0/24 which should be RETURN all -- 172.16.0.0/24 172.23.0.0/24
I don't think this is a neutron issue as the code adding these iptables rules lives in the openstack/puppet-tripleo repository. But I see it was moved from DF to Networking, wondering if that was just a mistake. Thanks.
I will pick this up. It's another regression in the move from instack-undercloud to containerized undercloud. There is also LP bug: https://bugs.launchpad.net/tripleo/+bug/1794729 that is kind of related. But this is in a worse state than I had realized.
Hi Christophe, Thank you for reporting this. Can you try it with this fix https://review.openstack.org/609858 applied on the undercloud? Thanks. // Harald
Hi Harald, Tested, the appropriate iptables rules are properly installed. Thanks !
Installed latest osp 14 14 -p 2018-11-13.1 Environment puppet-tripleo-9.3.1-0.20181010034746.157eaab.el7ost.noarch Steps 1) config undercloud.conf 2) openstack undercloud install (undercloud) [stack@undercloud-0 ~]$ cat undercloud.conf [DEFAULT] # Network interface on the Undercloud that will be handling the PXE # boots and DHCP for Overcloud instances. (string value) local_interface = eth0 local_ip = 192.168.24.1/24 undercloud_public_host = 192.168.24.2 undercloud_admin_host = 192.168.24.3 #TODO: use release >= 10 when RHBZ#1633193 is resolved undercloud_ntp_servers=clock.redhat.com container_images_file=/home/stack/containers-prepare-parameter.yaml docker_insecure_registries=docker-registry.engineering.redhat.com,brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888 undercloud_service_certificate = /etc/pki/instack-certs/undercloud.pem hieradata_override = /home/stack/hiera_override.yaml subnets = ctlplane-subnet,edge-subnet local_subnet = ctlplane-subnet enable_routed_networks = true [ctlplane-subnet] local_subnet = ctlplane-subnet cidr = 192.168.24.0/24 dhcp_start = 192.168.24.5 dhcp_end = 192.168.24.24 gateway = 192.168.24.1 inspection_iprange = 192.168.24.100,192.168.24.120 masquerade = true #TODO(skatlapa): add param to override masq [edge-subnet] cidr = 192.168.25.0/24 dhcp_start = 192.168.25.10 dhcp_end = 192.168.25.30 gateway = 192.168.25.254 inspection_iprange = 192.168.25.100,192.168.25.120 masquerade = true (undercloud) [stack@undercloud-0 ~]$ openstack subnet list +--------------------------------------+-----------------+--------------------------------------+-----------------+ | ID | Name | Network | Subnet | +--------------------------------------+-----------------+--------------------------------------+-----------------+ | 410f1dad-adc7-4055-9189-9dcb43f4371f | edge-subnet | 3d6d0a18-11f3-491e-95a4-ae369f3681f8 | 192.168.25.0/24 | | 76011857-989f-4884-8342-cbcd0eecba00 | ctlplane-subnet | 3d6d0a18-11f3-491e-95a4-ae369f3681f8 | 192.168.24.0/24 | +--------------------------------------+-----------------+--------------------------------------+-----------------+ (undercloud) [stack@undercloud-0 ~]$ sudo iptables -t nat -L POSTROUTING Chain POSTROUTING (policy ACCEPT) target prot opt source destination RETURN all -- 10.0.0.0/24 10.0.0.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return src 10.0.0.0/24 dest 10.0.0.0/24 ipv4 */ RETURN all -- 192.168.24.0/24 192.168.24.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return src 192.168.24.0/24 dest 192.168.24.0/24 ipv4 */ RETURN all -- 192.168.24.0/24 192.168.25.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return src 192.168.24.0/24 dest 192.168.25.0/24 ipv4 */ RETURN all -- 192.168.25.0/24 192.168.24.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return src 192.168.25.0/24 dest 192.168.24.0/24 ipv4 */ RETURN all -- 192.168.25.0/24 192.168.25.0/24 state NEW,RELATED,ESTABLISHED /* 137 routed_network return src 192.168.25.0/24 dest 192.168.25.0/24 ipv4 */ MASQUERADE all -- 10.0.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 10.0.0.0/24 ipv4 */ MASQUERADE all -- 192.168.24.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 192.168.24.0/24 ipv4 */ MASQUERADE all -- 192.168.25.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 138 routed_network masquerade 192.168.25.0/24 ipv4 */ (undercloud) [stack@undercloud-0 ~]$ sudo iptables -L FORWARD Chain FORWARD (policy ACCEPT) target prot opt source destination neutron-filter-top all -- anywhere anywhere neutron-openvswi-FORWARD all -- anywhere anywhere ACCEPT all -- 10.0.0.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 10.0.0.0/24 ipv4 */ ACCEPT all -- 192.168.24.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 192.168.24.0/24 ipv4 */ ACCEPT all -- 192.168.25.0/24 anywhere state NEW,RELATED,ESTABLISHED /* 139 routed_network forward source 192.168.25.0/24 ipv4 */ ACCEPT all -- anywhere 10.0.0.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 10.0.0.0/24 ipv4 */ ACCEPT all -- anywhere 192.168.24.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 192.168.24.0/24 ipv4 */ ACCEPT all -- anywhere 192.168.25.0/24 state NEW,RELATED,ESTABLISHED /* 140 routed_network forward destinations 192.168.25.0/24 ipv4 */ REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2019:0045