*** Bug 1879085 has been marked as a duplicate of this bug. ***
Checked with 4.5.0-0.nightly-2021-01-12-113007, and /etc/NetworkManager/dispatcher.d/30-resolv-prepender do not contain the proxy info on the masters. Is this expected? $ oc get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME wj45ios114z-fv6bn-master-0 Ready master 71m v1.18.3+f561b20 192.168.1.165 <none> Red Hat Enterprise Linux CoreOS 45.82.202101120929-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios114z-fv6bn-master-1 Ready master 71m v1.18.3+f561b20 192.168.2.133 <none> Red Hat Enterprise Linux CoreOS 45.82.202101120929-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios114z-fv6bn-master-2 Ready master 70m v1.18.3+f561b20 192.168.2.193 <none> Red Hat Enterprise Linux CoreOS 45.82.202101120929-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios114z-fv6bn-worker-2z9xc Ready worker 45m v1.18.3+f561b20 192.168.2.32 <none> Red Hat Enterprise Linux CoreOS 45.82.202101120929-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios114z-fv6bn-worker-r9qkn Ready worker 45m v1.18.3+f561b20 192.168.0.86 <none> Red Hat Enterprise Linux CoreOS 45.82.202101120929-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios114z-fv6bn-worker-svt8r Ready worker 44m v1.18.3+f561b20 192.168.3.100 <none> Red Hat Enterprise Linux CoreOS 45.82.202101120929-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 $ oc get proxy cluster -o yaml |grep -i -A 5 "^spec" spec: httpProxy: http://proxy-user1:xxx@10.0.77.163:3128 httpsProxy: http://proxy-user1:xxx@10.0.77.163:3128 noProxy: rhos-d.infra.prod.upshift.rdu2.redhat.com,oauth-openshift.apps.wj45ios114z.0114-n-q.qe.rhcloud.com trustedCA: name: "" $ $ oc debug nodes/wj45ios114z-fv6bn-master-0 -- chroot /host /usr/bin/cat /etc/NetworkManager/dispatcher.d/30-resolv-prepender Creating debug namespace/openshift-debug-node-chmk7 ... Starting pod/wj45ios114z-fv6bn-master-0-debug ... To use host binaries, run `chroot /host` #!/bin/bash set -eo pipefail IFACE=$1 STATUS=$2 case "$STATUS" in up|down|dhcp4-change|dhcp6-change) logger -s "NM resolv-prepender triggered by ${1} ${2}." # Ensure resolv.conf exists before we try to run podman cp /var/run/NetworkManager/resolv.conf /etc/resolv.conf NAMESERVER_IP="192.168.0.6" DOMAIN="wj45ios114z.0114-n-q.qe.rhcloud.com" if [[ -n "$NAMESERVER_IP" ]]; then logger -s "NM resolv-prepender: Prepending 'nameserver $NAMESERVER_IP' to /etc/resolv.conf (other nameservers from /var/run/NetworkManager/resolv.conf)" sed -e "/^search/d" \ -e "/Generated by/c# Generated by OpenStack resolv prepender NM dispatcher script\nsearch $DOMAIN\nnameserver $NAMESERVER_IP" \ /var/run/NetworkManager/resolv.conf > /etc/resolv.tmp fi # Only leave the first 3 nameservers in /etc/resolv.conf sed -i ':a $!{N; ba}; s/\(^\|\n\)nameserver/\n# nameserver/4g' /etc/resolv.tmp mv -f /etc/resolv.tmp /etc/resolv.conf ;; *) ;; esac Removing debug pod ... Removing debug namespace/openshift-debug-node-chmk7 ... $ oc debug nodes/wj45ios114z-fv6bn-worker-r9qkn -- chroot /host /usr/bin/cat /etc/NetworkManager/dispatcher.d/30-resolv-prepender Creating debug namespace/openshift-debug-node-thnfc ... Starting pod/wj45ios114z-fv6bn-worker-2z9xc-debug ... To use host binaries, run `chroot /host` #!/bin/bash IFACE=$1 STATUS=$2 export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios114z.0114-n-q.qe.rhcloud.com,etcd-0.wj45ios114z.0114-n-q.qe.rhcloud.com,etcd-1.wj45ios114z.0114-n-q.qe.rhcloud.com,etcd-2.wj45ios114z.0114-n-q.qe.rhcloud.com,localhost,oauth-openshift.apps.wj45ios114z.0114-n-q.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com case "$STATUS" in up|down|dhcp4-change|dhcp6-change) logger -s "NM resolv-prepender triggered by ${1} ${2}." # Ensure resolv.conf exists before we try to run podman cp /var/run/NetworkManager/resolv.conf /etc/resolv.conf NAMESERVER_IP=$(/usr/bin/podman run --rm \ --authfile /var/lib/kubelet/config.json \ --net=host \ quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:879b9e7602a00bb1afdfa836bc3e52a0be0c8328db83e06ef1a3e15d498a8834 \ node-ip \ show \ "192.168.0.5" \ "192.168.0.6" \ "192.168.0.7") DOMAIN="wj45ios114z.0114-n-q.qe.rhcloud.com" if [[ -n "$NAMESERVER_IP" ]]; then logger -s "NM resolv-prepender: Prepending 'nameserver $NAMESERVER_IP' to /etc/resolv.conf (other nameservers from /var/run/NetworkManager/resolv.conf)" sed -e "/^search/d" \ -e "/Generated by/c# Generated by OpenStack resolv prepender NM dispatcher script\nsearch $DOMAIN\nnameserver $NAMESERVER_IP" \ /var/run/NetworkManager/resolv.conf > /etc/resolv.tmp fi # Only leave the first 3 nameservers in /etc/resolv.conf sed -i ':a $!{N; ba}; s/\(^\|\n\)nameserver/\n# nameserver/4g' /etc/resolv.tmp mv -f /etc/resolv.tmp /etc/resolv.conf ;; *) ;; esac Removing debug pod ... Removing debug namespace/openshift-debug-node-thnfc ...
Checked with 4.5.0-202011121956.p0-6082e94 and it's fixed now. ./openshift-install 4.5.0-0.nightly-2021-01-22-052210 built from commit b42b0475e12c6e5b8536b3aa53b5db3f6970e1de release image registry.ci.openshift.org/ocp/release@sha256:4317daf1d4e1d0d878f190f706c70175ac4f19713fddaeb52d01b39792386185 $ oc get clusterversion NAME VERSION AVAILABLE PROGRESSING SINCE STATUS version 4.5.0-0.nightly-2021-01-22-052210 True False 19m Cluster version is 4.5.0-0.nightly-2021-01-22-052210 $ oc get proxy cluster -o yaml | grep -A 5 "^spec" spec: httpProxy: http://proxy-user1:xxx@10.0.77.163:3128 httpsProxy: http://proxy-user1:xxx@10.0.77.163:3128 noProxy: rhos-d.infra.prod.upshift.rdu2.redhat.com,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com trustedCA: name: "" $ oc get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME wj45ios122py-nldsl-master-0 Ready master 52m v1.18.3+f561b20 192.168.0.124 <none> Red Hat Enterprise Linux CoreOS 45.82.202101191332-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios122py-nldsl-master-1 Ready master 52m v1.18.3+f561b20 192.168.3.188 <none> Red Hat Enterprise Linux CoreOS 45.82.202101191332-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios122py-nldsl-master-2 Ready master 52m v1.18.3+f561b20 192.168.0.27 <none> Red Hat Enterprise Linux CoreOS 45.82.202101191332-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios122py-nldsl-worker-6cx47 Ready worker 25m v1.18.3+f561b20 192.168.2.167 <none> Red Hat Enterprise Linux CoreOS 45.82.202101191332-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios122py-nldsl-worker-86lrr Ready worker 27m v1.18.3+f561b20 192.168.0.220 <none> Red Hat Enterprise Linux CoreOS 45.82.202101191332-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 wj45ios122py-nldsl-worker-b8vbv Ready worker 25m v1.18.3+f561b20 192.168.0.240 <none> Red Hat Enterprise Linux CoreOS 45.82.202101191332-0 (Ootpa) 4.18.0-193.40.1.el8_2.x86_64 cri-o://1.18.4-5.rhaos4.5.gitd655863.el8 $ oc get nodes -o name | xargs -n 1 -I {} oc debug {} -- chroot /host /usr/bin/cat /etc/NetworkManager/dispatcher.d/30-resolv-prepender | grep -i export 130 ↵ Starting pod/wj45ios122py-nldsl-master-0-debug ... To use host binaries, run `chroot /host` export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-0.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-1.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-2.wj45ios122py.0122-7fu.qe.rh cloud.com,localhost,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com Removing debug pod ... Starting pod/wj45ios122py-nldsl-master-1-debug ... To use host binaries, run `chroot /host` export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-0.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-1.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-2.wj45ios122py.0122-7fu.qe.rh cloud.com,localhost,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com Removing debug pod ... Starting pod/wj45ios122py-nldsl-master-2-debug ... To use host binaries, run `chroot /host` export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-0.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-1.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-2.wj45ios122py.0122-7fu.qe.r$ cloud.com,localhost,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com Removing debug pod ... Starting pod/wj45ios122py-nldsl-worker-6cx47-debug ... To use host binaries, run `chroot /host` export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-0.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-1.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-2.wj45ios122py.0122-7fu.qe.r$ cloud.com,localhost,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com Removing debug pod ... Starting pod/wj45ios122py-nldsl-worker-86lrr-debug ... To use host binaries, run `chroot /host` export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-0.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-1.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-2.wj45ios122py.0122-7fu.qe.r$ cloud.com,localhost,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com Removing debug pod ... Starting pod/wj45ios122py-nldsl-worker-b8vbv-debug ... To use host binaries, run `chroot /host` export HTTP_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export HTTPS_PROXY=http://proxy-user1:xxx@10.0.77.163:3128 export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-0.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-1.wj45ios122py.0122-7fu.qe.rhcloud.com,etcd-2.wj45ios122py.0122-7fu.qe.r$ cloud.com,localhost,oauth-openshift.apps.wj45ios122py.0122-7fu.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com Removing debug pod ...
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (OpenShift Container Platform 4.5.30 bug fix update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:0231