Bug 1805212

Summary: podman (1.6.4) rhel 8.1 no route to host from inside container
Product: Red Hat Enterprise Linux 8 Reporter: Mangirdas Judeikis <mjudeiki>
Component: podmanAssignee: Jindrich Novy <jnovy>
Status: CLOSED ERRATA QA Contact: atomic-bugs <atomic-bugs>
Severity: high Docs Contact:
Priority: urgent    
Version: 8.1CC: abrahm.scully, bbaude, cjeanner, ddarrah, dornelas, dwalsh, emacchi, ikke, jiji, jligon, jmaxwell, jnovy, klaas, lsm5, mcambria, mheon, pthomas, rrajaram, toneata, tsweeney, weshen, ypu
Target Milestone: rcKeywords: Regression, ZStream
Target Release: 8.2   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: podman-1.6.4-9.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1807603 (view as bug list) Environment:
Last Closed: 2020-04-28 15:53:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1806895, 1806896, 1806898, 1806899, 1806900, 1806901    
Bug Blocks: 1186913, 1734579    

Description Mangirdas Judeikis 2020-02-20 13:52:19 UTC
Description of problem:

No external connecitvity from within container on podman 1.6.4 on rhel 8.1


Version-Release number of selected component (if applicable):

sudo iptables --version
iptables v1.8.2 (nf_tables)



How reproducible:

podman 1.4.2 - works:
[root@test ~]# podman run -it --rm docker.io/fedora:27 bash 
[root@ad894a59ec5e /]# curl www.redhat.com

podman 1.6.4: 

Steps to Reproduce:
1. Install rhel 8.1
2. Upgrae podman to 1.6.4
[root@rp-000000 ~]# podman run -it --rm docker.io/fedora:27 bash                                                                                                                                                                                                                                                                                                                                                                                                                                       
[root@test ~]# podman run -it --rm docker.io/fedora:27 bash 
[root@9a4b54b20e41 /]# curl www.redhat.com
curl: (6) Could not resolve host: www.redhat.com

Actual results:

no route to host

Expected results:

curl works


Additional info:

https://github.com/containers/libpod/issues/3462

Comment 1 Mangirdas Judeikis 2020-02-20 14:17:00 UTC
Ignore github URL, but I suspect this might be related to the CNI, hence the url.

Comment 2 Jim Minter 2020-02-20 15:40:12 UTC
[root@rhel8 ~]# ifconfig 
cni-podman0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 10.88.0.1  netmask 255.255.0.0  broadcast 10.88.255.255
        inet6 fe80::94f5:fff:fef1:58f3  prefixlen 64  scopeid 0x20<link>
        ether 96:f5:0f:f1:58:f3  txqueuelen 1000  (Ethernet)
        RX packets 26  bytes 1640 (1.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 36  bytes 2962 (2.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.16.3.4  netmask 255.255.255.0  broadcast 172.16.3.255
        inet6 fe80::20d:3aff:fea6:147a  prefixlen 64  scopeid 0x20<link>
        ether 00:0d:3a:a6:14:7a  txqueuelen 1000  (Ethernet)
        RX packets 153696  bytes 180556922 (172.1 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 62739  bytes 11341152 (10.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 20  bytes 1692 (1.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 20  bytes 1692 (1.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[root@rhel8 ~]# podman run --rm -it fedora
[root@a4b52fc23235 /]# curl 23.2.168.38
curl: (7) Failed to connect to 23.2.168.38 port 80: No route to host
[root@a4b52fc23235 /]# curl www.google.com
curl: (6) Could not resolve host: www.google.com
[root@a4b52fc23235 /]# 

[root@rhel8 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 53632 packets, 174M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   20  1616 CNI-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* CNI firewall plugin rules */

Chain OUTPUT (policy ACCEPT 64154 packets, 11M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain CNI-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   20  1616 CNI-ADMIN  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* CNI firewall plugin rules */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.88.0.3            ctstate RELATED,ESTABLISHED
   10   808 ACCEPT     all  --  *      *       10.88.0.3            0.0.0.0/0           

Chain CNI-ADMIN (1 references)
 pkts bytes target     prot opt in     out     source               destination         
[root@rhel8 ~]# iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 39 packets, 4196 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain INPUT (policy ACCEPT 5 packets, 260 bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 10595 packets, 683K bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 CNI-5deeac696f2415fa92d1a5e2  all  --  *      *       10.88.0.3            0.0.0.0/0            /* name: "podman" id: "a4b52fc23235ac3d3259015a9d99da6637e9ffdf9fc756dd7ea5b8796cdea33c" */

Chain OUTPUT (policy ACCEPT 10597 packets, 683K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain CNI-5deeac696f2415fa92d1a5e2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.88.0.0/16         /* name: "podman" id: "a4b52fc23235ac3d3259015a9d99da6637e9ffdf9fc756dd7ea5b8796cdea33c" */
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0           !224.0.0.0/4          /* name: "podman" id: "a4b52fc23235ac3d3259015a9d99da6637e9ffdf9fc756dd7ea5b8796cdea33c" */

Comment 19 Ilkka Tengvall 2020-02-27 09:26:39 UTC
the proposed fix is not enough. It still can't send TCP outside the host. TCP masquerade rules won't get applied. See github issue: https://github.com/containers/libpod/issues/5335

Comment 31 errata-xmlrpc 2020-04-28 15:53:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1650