Bug 1915950
| Summary: | IPv6 errors after exiting podman | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Qian Cai <qcai> | |
| Component: | podman | Assignee: | Matthew Heon <mheon> | |
| Status: | CLOSED DUPLICATE | QA Contact: | atomic-bugs <atomic-bugs> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 8.4 | CC: | bbaude, dwalsh, gscrivan, jligon, jnovy, lsm5, mcambria, mheon, pthomas, redhat-bugzilla, robert.scheck, umohnani | |
| Target Milestone: | rc | Flags: | pm-rhel:
mirror+
|
|
| Target Release: | 8.4 | |||
| Hardware: | All | |||
| OS: | Linux | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | ||
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1918417 (view as bug list) | Environment: | ||
| Last Closed: | 2021-02-24 18:25:46 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1918417, 1932399, 1942665 | |||
Same thing with runc, i.e., # podman run --rm -it alpine sh Qian, Can you try this test with a more recent version of Podman? I think this may have been addressed in later versions of Podman. I don't have an environment set up for building the git master yet, so it might take a while. I'll look to see if there is a newer version in brew that can be installed on RHEL-8. Then, that would be easiest. Same thing with podman-3.0.0-6.module+el8.4.0+9088+ac8dc0c6.x86_64 ERRO[0016] Error deleting network: running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 2601:601:9f80:3c4f::2/64 -j CNI-2a33fd25b62f81040709553e -m comment --comment name: "podman" id: "dcaaf5d43d08a2a78c74e37c15cc4a398dfde2d8f8414f5f62ebc41977418721" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?). ERRO[0016] Error while removing pod from CNI network "podman": running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 2601:601:9f80:3c4f::2/64 -j CNI-2a33fd25b62f81040709553e -m comment --comment name: "podman" id: "dcaaf5d43d08a2a78c74e37c15cc4a398dfde2d8f8414f5f62ebc41977418721" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?). ERRO[0016] unable to cleanup network for container dcaaf5d43d08a2a78c74e37c15cc4a398dfde2d8f8414f5f62ebc41977418721: "error tearing down CNI namespace configuration for container dcaaf5d43d08a2a78c74e37c15cc4a398dfde2d8f8414f5f62ebc41977418721: running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 2601:601:9f80:3c4f::2/64 -j CNI-2a33fd25b62f81040709553e -m comment --comment name: \"podman\" id: \"dcaaf5d43d08a2a78c74e37c15cc4a398dfde2d8f8414f5f62ebc41977418721\" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?).\n" crictl looks like the same thing. # crictl stopp a382307204515 FATA[0000] stopping the pod sandbox "a382307204515": rpc error: code = Unknown desc = failed to destroy network for pod sandbox k8s_ubi8-sandbox_default_hdishd83djaidwnduwk28bcsb_1(a38230720451583ade7c0b84fa7d08663e3417b52ad2ffadca9b553fb36d82d3): running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 1100:200::5/24 -j CNI-7860ea922d1a46225b9c13c1 -m comment --comment name: "crio" id: "a38230720451583ade7c0b84fa7d08663e3417b52ad2ffadca9b553fb36d82d3" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?). cri-o-1.20.0-0.rhaos4.7.git845747f.el8.40.x86_64 cri-tools-1.20.0-1.el8.x86_64 Michael, any ideas?
Is the cni firewall plugin missing from /etc/cni/net.d/87-podman-bridge.conflist ?
I only see a diff.
Does the firewall plugin supply the right backend for the installed OS? IIRC this was needed at one point.
Is --runtime=kata sending CNI the firewall plugin?
Look at what I ran below (fedora 32) for differences in e.g. configuration. Suggest identifying diffs in config syntax etc. getting things to work without --runtime==kata.
[mcc@snark ebpf-stuff]$ sudo podman run --rm --network=ipv6test -it alpine sh
/ # ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
3: eth0@if25: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether aa:2d:81:7a:a5:9a brd ff:ff:ff:ff:ff:ff
inet 10.66.0.6/16 brd 10.66.255.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fd00::1:8:9/112 scope global
valid_lft forever preferred_lft forever
inet6 fe80::a82d:81ff:fe7a:a59a/64 scope link
valid_lft forever preferred_lft forever
/ # exit
[mcc@snark ebpf-stuff]$
[mcc@snark ebpf-stuff]$ sudo cat /etc/cni/net.d/666-podman-ipv6test.conflist
{
"cniVersion": "0.4.0",
"name": "ipv6test",
"plugins": [
{
"type": "bridge",
"bridge": "v6test0",
"isGateway": true,
"ipMasq": true,
"hairpinMode": true,
"ipam": {
"type": "host-local",
"routes": [
{
"dst": "0.0.0.0/0"
},
{
"dst": "::/0"
}
],
"ranges": [
[
{
"subnet": "10.66.0.0/16",
"gateway": "10.66.0.1"
}
],
[
{
"subnet": "fd00::1:8:0/112",
"gateway": "fd00::1:8:1"
}
]
]
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
},
{
"type": "firewall",
"backend": ""
},
{
"type": "tuning"
}
]
}
[mcc@snark ebpf-stuff]$
I believe this is the same CNI plugin issue. *** This bug has been marked as a duplicate of bug 1918417 *** *** Bug 1954844 has been marked as a duplicate of this bug. *** |
Description of problem: After setting up the ipv4/ipv6 dual-stack for podman using the diff below. # diff -u /etc/cni/net.d/87-podman-bridge.conflist.orig /etc/cni/net.d/87-podman-bridge.conflist --- /etc/cni/net.d/87-podman-bridge.conflist.orig 2021-01-13 10:44:39.787742958 -0500 +++ /etc/cni/net.d/87-podman-bridge.conflist 2021-01-13 10:59:12.012980806 -0500 @@ -7,16 +7,21 @@ "bridge": "cni-podman0", "isGateway": true, "ipMasq": true, - "hairpinMode": true, "ipam": { "type": "host-local", - "routes": [{ "dst": "0.0.0.0/0" }], + "routes": [{ "dst": "0.0.0.0/0" }, {"dst": "2000::/3" }], "ranges": [ [ { "subnet": "10.88.0.0/16", "gateway": "10.88.0.1" } + ], + [ + { + "subnet": "2601:601:9f80:3c4f::/64", + "gateway": "2601:601:9f80:3c4f::1" + } ] ] } # podman run --rm --runtime=kata -it alpine sh / # exit ERRO[0011] Error deleting network: running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 2601:601:9f80:3c4f::4/64 -j CNI-b0b16c6135b11805f014bb70 -m comment --comment name: "podman" id: "e4b7fa1550ac6c98dfbfb5a44e2928852d789bc172868645c80376142ea405df" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?). ERRO[0011] Error while removing pod from CNI network "podman": running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 2601:601:9f80:3c4f::4/64 -j CNI-b0b16c6135b11805f014bb70 -m comment --comment name: "podman" id: "e4b7fa1550ac6c98dfbfb5a44e2928852d789bc172868645c80376142ea405df" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?). ERRO[0011] unable to cleanup network for container e4b7fa1550ac6c98dfbfb5a44e2928852d789bc172868645c80376142ea405df: "error tearing down CNI namespace configuration for container e4b7fa1550ac6c98dfbfb5a44e2928852d789bc172868645c80376142ea405df: running [/usr/sbin/ip6tables -t nat -D POSTROUTING -s 2601:601:9f80:3c4f::4/64 -j CNI-b0b16c6135b11805f014bb70 -m comment --comment name: \"podman\" id: \"e4b7fa1550ac6c98dfbfb5a44e2928852d789bc172868645c80376142ea405df\" --wait]: exit status 1: iptables: Bad rule (does a matching rule exist in that chain?).\n" Version-Release number of selected component (if applicable): containernetworking-plugins-0.8.6-2.module+el8.3.0+8221+97165c3f.x86_64 podman-2.0.5-5.module+el8.3.0+8221+97165c3f.x86_64 iptables-1.8.4-15.el8_3.3.x86_64 How reproducible: always