| Summary: | Nodes can't resolved IPv4 address in dual stack configuration | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Rei <rhalle> |
| Component: | Networking | Assignee: | Yossi Boaron <yboaron> |
| Networking sub component: | runtime-cfg | QA Contact: | Rei <rhalle> |
| Status: | CLOSED ERRATA | Docs Contact: | |
| Severity: | medium | ||
| Priority: | unspecified | CC: | aos-bugs, yboaron |
| Version: | 4.8 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.9.0 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-10-18 17:47:26 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Bug Depends On: | |||
| Bug Blocks: | 1996573 | ||
This is regression from OCP-4.7 in 4.7 resolution is for IPv4 oc version Client Version: 4.7.0-0.nightly-2021-08-18-141355 Server Version: 4.7.0-0.nightly-2021-08-18-141355 Kubernetes Version: v1.20.0+4593a24 [root@seal23 ~]# oc get node NAME STATUS ROLES AGE VERSION master-0-0.ocp-edge-cluster-0.qe.lab.redhat.com Ready master 15m v1.20.0+4593a24 master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com Ready master 15m v1.20.0+4593a24 master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com Ready master 15m v1.20.0+4593a24 [root@seal23 ~]# watch oc get bmh -n openshift-machine-api [root@seal23 ~]# oc get node NAME STATUS ROLES AGE VERSION master-0-0.ocp-edge-cluster-0.qe.lab.redhat.com Ready master 26m v1.20.0+4593a24 master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com Ready master 26m v1.20.0+4593a24 master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com Ready master 26m v1.20.0+4593a24 worker-0-0.ocp-edge-cluster-0.qe.lab.redhat.com Ready worker 2m47s v1.20.0+4593a24 worker-0-1.ocp-edge-cluster-0.qe.lab.redhat.com Ready worker 98s v1.20.0+4593a24 [root@seal23 ~]# oc debug node/master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host dig master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com A' --insecure-skip-tls-verify Starting pod/master-0-1ocp-edge-cluster-0qelabredhatcom-debug ... To use host binaries, run `chroot /host` ; <<>> DiG 9.11.26-RedHat-9.11.26-4.el8_4 <<>> master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com A ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 48136 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: 78fcb8b250274b72 (echoed) ;; QUESTION SECTION: ;master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. IN A ;; ANSWER SECTION: master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. 30 IN A 192.168.123.145 ;; Query time: 0 msec ;; SERVER: 192.168.123.144#53(192.168.123.144) ;; WHEN: Thu Aug 19 08:54:54 UTC 2021 ;; MSG SIZE rcvd: 151 Removing debug pod ... [root@seal23 ~]# ^A^AAAA oc debug node/master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host dig master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com AAAA' --insecure-skip-tls-verify Starting pod/master-0-1ocp-edge-cluster-0qelabredhatcom-debug ... To use host binaries, run `chroot /host` ; <<>> DiG 9.11.26-RedHat-9.11.26-4.el8_4 <<>> master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com AAAA ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35570 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: 94a4e5668d1b0adb (echoed) ;; QUESTION SECTION: ;master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. IN AAAA ;; ANSWER SECTION: master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. 30 IN A 192.168.123.145 ;; Query time: 0 msec ;; SERVER: 192.168.123.144#53(192.168.123.144) ;; WHEN: Thu Aug 19 08:55:12 UTC 2021 ;; MSG SIZE rcvd: 151 Verify: [root@seal23 ~]# oc debug node/master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host ping master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com -6' --insecure-skip-tls-verify Starting pod/master-0-1ocp-edge-cluster-0qelabredhatcom-debug ... To use host binaries, run `chroot /host` PING master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com(master-0-2 (fd2e:6f44:5dd8::65)) 56 data bytes 64 bytes from master-0-2 (fd2e:6f44:5dd8::65): icmp_seq=1 ttl=64 time=2.43 ms 64 bytes from master-0-2 (fd2e:6f44:5dd8::65): icmp_seq=2 ttl=64 time=0.406 ms 64 bytes from master-0-2 (fd2e:6f44:5dd8::65): icmp_seq=3 ttl=64 time=0.367 ms 64 bytes from master-0-2 (fd2e:6f44:5dd8::65): icmp_seq=4 ttl=64 time=0.256 ms ^C Removing debug pod ... [root@seal23 ~]# oc debug node/master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host ping master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com -4' --insecure-skip-tls-verify Starting pod/master-0-1ocp-edge-cluster-0qelabredhatcom-debug ... To use host binaries, run `chroot /host` PING master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com (192.168.123.121) 56(84) bytes of data. 64 bytes from master-0-2 (192.168.123.121): icmp_seq=1 ttl=64 time=0.946 ms 64 bytes from master-0-2 (192.168.123.121): icmp_seq=2 ttl=64 time=0.259 ms 64 bytes from master-0-2 (192.168.123.121): icmp_seq=3 ttl=64 time=0.216 ms 64 bytes from master-0-2 (192.168.123.121): icmp_seq=4 ttl=64 time=0.402 ms ^C Removing debug pod ... [root@seal23 ~]# oc debug node/master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host dig master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com ' --insecure-skip-tls-verify Starting pod/master-0-1ocp-edge-cluster-0qelabredhatcom-debug ... To use host binaries, run `chroot /host` ; <<>> DiG 9.11.26-RedHat-9.11.26-4.el8_4 <<>> master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16163 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: 672e0ff3f32b86bc (echoed) ;; QUESTION SECTION: ;master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. IN A ;; ANSWER SECTION: master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. 13 IN A 192.168.123.121 ;; Query time: 0 msec ;; SERVER: 192.168.123.55#53(192.168.123.55) ;; WHEN: Sun Aug 22 12:38:42 UTC 2021 ;; MSG SIZE rcvd: 151 Removing debug pod ... [root@seal23 ~]# oc debug node/master-0-1.ocp-edge-cluster-0.qe.lab.redhat.com --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host dig master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com AAAA' --insecure-skip-tls-verify Starting pod/master-0-1ocp-edge-cluster-0qelabredhatcom-debug ... To use host binaries, run `chroot /host` ; <<>> DiG 9.11.26-RedHat-9.11.26-4.el8_4 <<>> master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com AAAA ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44879 ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: bf5dc7b3500906b2 (echoed) ;; QUESTION SECTION: ;master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. IN AAAA ;; ANSWER SECTION: master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. 30 IN AAAA fd2e:6f44:5dd8::65 ;; Query time: 0 msec ;; SERVER: 192.168.123.55#53(192.168.123.55) ;; WHEN: Sun Aug 22 12:38:54 UTC 2021 ;; MSG SIZE rcvd: 163 Removing debug pod ... [root@seal23 ~]# oc version Client Version: 4.9.0-0.nightly-2021-08-22-005117 Server Version: 4.9.0-0.nightly-2021-08-22-070405 Kubernetes Version: v1.22.0-rc.0+5c2f7cd Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.9.0 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:3759 |
Description of problem: In case you deployed dual stack BM setup nodes can't resolved IPv4 address OpenShift release version: [rhalle@ocp-edge36 ~]$ oc version Client Version: 4.8.0-0.nightly-2021-08-17-004424 Server Version: 4.8.0-0.nightly-2021-08-17-004424 Kubernetes Version: v1.21.1+9807387 Cluster Platform: BM How reproducible: Deploy ocp on BM Run: oc debug node/master-0-0 --image registry.ocp-edge-cluster-0.qe.lab.redhat.com:5000/rhel8/support-tools:latest -- bash -c 'chroot /host dig master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com A | grep -A 1 'ANSWER SECTION' | grep -v 'ANSWER SECTION' | awk '{print $NF}'' --insecure-skip-tls-verify Actual results: ;; flags: qr aa rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available ;; OPT PSEUDOSECTION: ; EDNS: version: 0, flags:; udp: 4096 ; COOKIE: 5a22c279593559ef (echoed) ;; QUESTION SECTION: ;master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. IN A Expected results: ;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 ;; WARNING: recursion requested but not available -- ;; ANSWER SECTION: master-0-2.ocp-edge-cluster-0.qe.lab.redhat.com. 30 IN A 192.168.123.58 Impact of the problem: Dual stack nodes can't communicate with IPv4 protocol Additional info: ** Please do not disregard the report template; filling the template out as much as possible will allow us to help you. Please consider attaching a must-gather archive (via `oc adm must-gather`). Please review must-gather contents for sensitive information before attaching any must-gathers to a bugzilla report. You may also mark the bug private if you wish.