Bug 1872128 - Can't run container with hostPort on ipv6 cluster
Summary: Can't run container with hostPort on ipv6 cluster
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.5
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.7.0
Assignee: Antonio Ojea
QA Contact: Weinan Liu
URL:
Whiteboard:
Depends On:
Blocks: 1897336 1900658
TreeView+ depends on / blocked
 
Reported: 2020-08-25 07:00 UTC by Nir
Modified: 2021-03-02 04:08 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: CRI-O only uses IPv4 iptables to implement containers Host Ports Consequence: HostPort doesn´t work for IPv6 Fix: Implement HostPort IPv6 support in CRI-O Result: CRI-O support HostPort for IPv6 and IPv4
Clone Of:
: 1900658 (view as bug list)
Environment:
Last Closed: 2021-02-24 15:16:08 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github cri-o cri-o pull 4116 0 None closed dual stack portmap support 2021-02-16 22:15:06 UTC
Github cri-o cri-o pull 4361 0 None closed Portmap support for dual-stack 2021-02-16 22:15:06 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:16:33 UTC

Description Nir 2020-08-25 07:00:20 UTC
Description of problem:

While trying to deploy daemonset which is using hostPort I get the following when describing one of the pods:
  Normal   AddedInterface          2m8s                   multus                                       Add eth0 [fd01::6:e41e:24ff:fe00:a8/64]
  Warning  FailedCreatePodSandBox  2m7s (x5304 over 22h)  kubelet, worker-2.ostest.test.metalkube.org  (combined from similar events): Failed to create pod sandbox: rpc error: code = Unknown desc = failed to add hostport mapping for sandbox k8s_poison-pill-ds-nls79_default_6933bfce-b094-45d4-926d-9134a3d1cc82_0(11e205eec09420a90f965a5a43c616dfb37c236aae152d3e99798b69e5fc8eed): HostPortManager IP family mismatch: fd01::6:e41e:24ff:fe00:a8, isIPv6 - true
  Normal   AddedInterface          115s                   multus                                       Add eth0 [fd01::6:e41e:24ff:fe00:a8/64]
(many of these "Add eth0 .... " )


Version-Release number of selected component (if applicable):
$ oc version
Client Version: 4.5.0-0.ci-2020-08-12-000552
Server Version: 4.5.0-0.ci-2020-08-12-000552
Kubernetes Version: v1.18.3


How reproducible:
always

Steps to Reproduce:
1.deploy container with hostPort on ipv6 cluster

Actual results:
container stuck on `ContainerCreating` state, decribe pod shows the error mentioned above

Expected results:
Container will be in running state, and listening on the specified hostPort

Comment 1 Antonio Ojea 2020-08-25 08:03:00 UTC
It seems that CRI-O is always using iptables IPv4 for managing the host port mapping

https://github.com/cri-o/cri-o/blob/81a23e1553271e747d177b863235a24337f41181/server/server.go#L338-L342

> 	iptInterface := utiliptables.New(utilexec.New(), utiliptables.ProtocolIPv4)

One solution to fix this bug would be to parse the network configuration and initializing the hostportManager with the proper ipFamily. However, this would not solve the dual stack problem, where we should run two hostPort managers in parallel, one for each ipFamily.

We solved this in kubernetes upstream https://github.com/kubernetes/kubernetes/pull/80854 using one portManager per IP family and leveraging the sandbox IPs to choose the right one.

I submitted a PR with the later approach https://github.com/cri-o/cri-o/pull/4116

Comment 2 Seth Jennings 2020-08-28 15:23:20 UTC
cri-o PR merged

Comment 16 Weinan Liu 2020-12-09 15:31:02 UTC
Blocked by [BM][IPI] Master deployment failed: No valid host was found. Reason: No conductor service registered which supports driver redfish for conductor group https://bugzilla.redhat.com/show_bug.cgi?id=1902653, we can not get an ipv6 cluster built

Comment 19 Antonio Ojea 2020-12-11 11:52:07 UTC
Have you curled/pinged/polled the exposed hostPort to verify that it acually forwards the traffic to the container?

Comment 20 Weinan Liu 2020-12-18 07:51:49 UTC
We only managed to get clusters of ipv6 & disconnected, no mirrored images available.
I checked pod creating with a Completed status.

Comment 23 errata-xmlrpc 2021-02-24 15:16:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633


Note You need to log in before you can comment on or make changes to this bug.