Bug 1798805

Description of problem:

bootkube fails on bare metal deployment with IPv6 control plane because it cannot resolve etcd* records.

Deployment is blocked at:

time="2020-02-06T03:30:59Z" level=info msg="Waiting up to 30m0s for the Kubernetes API at https://api.ocp-edge-cluster.qe.lab.redhat.com:6443..."
time="2020-02-06T03:30:59Z" level=debug msg="Still waiting for the Kubernetes API: the server could not find the requested resource"
time="2020-02-06T03:31:29Z" level=debug msg="Still waiting for the Kubernetes API: the server could not find the requested resource"

Version-Release number of selected component (if applicable):
4.3.0-0.nightly-2020-02-05-064223-ipv6.2

How reproducible:
always on 4.3.0-0.nightly-2020-02-05-064223-ipv6.2

Steps to Reproduce:
1. Deploy bare metal deployment with IPV6 control plane
2. Log in to the bootstrap VM
3. Check bootkube log

Actual results:

Feb 06 03:32:32 localhost bootkube.sh[2510]: {"level":"warn","ts":"2020-02-06T03:32:32.298Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-08495e38-eb0d-4102-8b80-78ccf178d430/etcd-0.ocp-edge-cluster.qe.lab.redhat.com:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest connection error: connection error: desc = \"transport: Error while dialing dial tcp: lookup etcd-0.ocp-edge-cluster.qe.lab.redhat.com on [fe80::5054:ff:fe3a:f406%ens3]:53: no such host\""}
Feb 06 03:32:32 localhost bootkube.sh[2510]: {"level":"warn","ts":"2020-02-06T03:32:32.298Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-1447de30-934a-4925-96cf-a6523b355f50/etcd-1.ocp-edge-cluster.qe.lab.redhat.com:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest connection error: connection error: desc = \"transport: Error while dialing dial tcp: lookup etcd-1.ocp-edge-cluster.qe.lab.redhat.com on [fe80::5054:ff:fe3a:f406%ens3]:53: no such host\""}
Feb 06 03:32:32 localhost bootkube.sh[2510]: {"level":"warn","ts":"2020-02-06T03:32:32.298Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-6fa7cbc2-0e26-4781-978b-5335e7e186d3/etcd-2.ocp-edge-cluster.qe.lab.redhat.com:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest connection error: connection error: desc = \"transport: Error while dialing dial tcp: lookup etcd-2.ocp-edge-cluster.qe.lab.redhat.com on [fe80::5054:ff:fe3a:f406%ens3]:53: no such host\""}

Checking resolv.conf on the boostrap VM we can see:

[root@localhost core]# cat /etc/resolv.conf 
# Generated by NetworkManager
nameserver fe80::5054:ff:fe3a:f406%ens3
nameserver fd2e:6f44:5dd8:c956::1


Expected results:
bootkube runs without errors

Additional info:

After adding the DNS VIP to resolv.conf on the bootstrap VM deployment proceeds:

[root@localhost core]# cat /etc/resolv.conf 
# Generated by NetworkManager
nameserver fd2e:6f44:5dd8:c956::2
nameserver fe80::5054:ff:fe3a:f406%ens3
nameserver fd2e:6f44:5dd8:c956::1

where fd2e:6f44:5dd8:c956::2 is the DNS VIP