1904065 – [release 4.6] [Openstack] HTTP_PROXY setting for NetworkManager-resolv-prepender not working

Bug 1904065 - [release 4.6] [Openstack] HTTP_PROXY setting for NetworkManager-resolv-prepender not working

Summary: [release 4.6] [Openstack] HTTP_PROXY setting for NetworkManager-resolv-prepen...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	OpenShift Container Platform
Classification:	Red Hat
Component:	Machine Config Operator
Sub Component:
Version:	4.6.z
Hardware:	Unspecified
OS:	Unspecified
Priority:	urgent
Severity:	urgent
Target Milestone:	---
Target Release:	4.6.z
Assignee:	Emilien Macchi
QA Contact:	weiwei jiang
Docs Contact:
URL:
Whiteboard:
Duplicates (1):	1884821 (view as bug list)
Depends On:	1873556
Blocks:	1907636 1907637
TreeView+	depends on / blocked

Reported:	2020-12-03 13:06 UTC by Emilien Macchi
Modified:	2022-10-14 06:58 UTC (History)
CC List:	18 users (show)
Fixed In Version:
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:	1873556
Clones:	1907637 (view as bug list)
Environment:
Last Closed:	2020-12-21 13:23:59 UTC
Target Upstream Version:

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Github	openshift machine-config-operator pull 2279	0	None	closed	Bug 1904065: [on-prem] export proxy variables to be taken in account	2021-02-10 16:21:10 UTC
Red Hat Product Errata	RHSA-2020:5614	0	None	None	None	2020-12-21 13:24:24 UTC

Comment 1 Evgeny Slutsky 2020-12-09 13:26:10 UTC

this affects oVirt also

Comment 8 weiwei jiang 2020-12-18 06:08:02 UTC

Checked with 4.6.0-0.nightly-2020-12-17-202735, and it work well now.

$ oc get nodes -o wide 
NAME                                STATUS   ROLES    AGE    VERSION           INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                 CONTAINE
R-RUNTIME
wj46ios1218b-nbx79-master-0         Ready    master   105m   v1.19.0+7070803   192.168.1.73    <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://
1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-master-1         Ready    master   105m   v1.19.0+7070803   192.168.1.179   <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://
1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-master-2         Ready    master   105m   v1.19.0+7070803   192.168.0.86    <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://
1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-worker-0-2mf7w   Ready    worker   91m    v1.19.0+7070803   192.168.3.184   <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://
1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-worker-0-894rs   Ready    worker   92m    v1.19.0+7070803   192.168.1.62    <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://
1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-worker-0-9zdhk   Ready    worker   19m    v1.19.0+7070803   192.168.2.159   <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://
1.19.0-26.rhaos4.6.git8a05a29.el8

$ oc get proxy cluster -o yaml
apiVersion: config.openshift.io/v1
kind: Proxy
metadata:
  creationTimestamp: "2020-12-18T04:14:43Z"
  generation: 1
  ......
  name: cluster
  resourceVersion: "347"
  selfLink: /apis/config.openshift.io/v1/proxies/cluster
  uid: 293f543f-41fa-4b26-88f4-8d2d9a6816ca
spec:
  httpProxy: http://proxy-user1:HIDDEN@10.0.77.163.nip.io:3128
  httpsProxy: http://proxy-user1:HIDDEN@10.0.77.163.nip.io:3128
  noProxy: rhos-d.infra.prod.upshift.rdu2.redhat.com,oauth-openshift.apps.wj46ios1218b.1218-39w.qe.rhcloud.com,api.wj46ios1218b.1218-39w.qe.rhcloud.com,api-int.wj46ios1218b.1218-39w.qe.rhcloud.com
  trustedCA:
    name: ""
status:
  httpProxy: http://proxy-user1:HIDDEN@10.0.77.163.nip.io:3128
  httpsProxy: http://proxy-user1:HIDDEN@10.0.77.163.nip.io:3128
  noProxy: .cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj46ios1218b.1218-39w.qe.rhcloud.com,api.wj46ios1218b.1218-39w.qe.rhcloud.com,etcd-0.wj46ios1218b.1218-39w.qe.rhcloud.com,etcd-1.wj46ios1218b.1218-39w.qe.rhcloud.com,etcd-2.wj46ios1218b.1218-39w.qe.rhcloud.com,localhost,oauth-openshift.apps.wj46ios1218b.1218-39w.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com

$ oc get nodes -o wide
NAME                                STATUS   ROLES    AGE    VERSION           INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                                                       KERNEL-VERSION                 CONTAINER-RUNTIME
wj46ios1218b-nbx79-master-0         Ready    master   106m   v1.19.0+7070803   192.168.1.73    <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-master-1         Ready    master   107m   v1.19.0+7070803   192.168.1.179   <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-master-2         Ready    master   107m   v1.19.0+7070803   192.168.0.86    <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-worker-0-2mf7w   Ready    worker   93m    v1.19.0+7070803   192.168.3.184   <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-worker-0-894rs   Ready    worker   94m    v1.19.0+7070803   192.168.1.62    <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://1.19.0-26.rhaos4.6.git8a05a29.el8
wj46ios1218b-nbx79-worker-0-9zdhk   Ready    worker   21m    v1.19.0+7070803   192.168.2.159   <none>        Red Hat Enterprise Linux CoreOS 46.82.202012151054-0 (Ootpa)   4.18.0-193.37.1.el8_2.x86_64   cri-o://1.19.0-26.rhaos4.6.git8a05a29.el8


$ oc debug nodes/wj46ios1218b-nbx79-master-0 -- chroot /host /usr/bin/cat /etc/NetworkManager/dispatcher.d/30-resolv-prepender                                                                               1 ↵
Creating debug namespace/openshift-debug-node-mlklp ...
Starting pod/wj46ios1218b-nbx79-master-0-debug ...
To use host binaries, run `chroot /host`
#!/bin/bash
set -eo pipefail
IFACE=$1
STATUS=$2

export HTTP_PROXY=http://proxy-user1:HIDDEN@10.0.77.163.nip.io:3128
export HTTPS_PROXY=http://proxy-user1:HIDDEN@10.0.77.163.nip.io:3128
export NO_PROXY=.cluster.local,.svc,10.128.0.0/14,127.0.0.1,169.254.169.254,172.30.0.0/16,192.168.0.0/18,api-int.wj46ios1218b.1218-39w.qe.rhcloud.com,api.wj46ios1218b.1218-39w.qe.rhcloud.com,etcd-0.wj46ios1218b.1218-39w.qe.rhcloud.com,etcd-1.wj46ios1218b.1218-39w.qe.rhcloud.com,etcd-2.wj46ios1218b.1218-39w.qe.rhcloud.com,localhost,oauth-openshift.apps.wj46ios1218b.1218-39w.qe.rhcloud.com,rhos-d.infra.prod.upshift.rdu2.redhat.com
case "$STATUS" in
    up|down|dhcp4-change|dhcp6-change)
    logger -s "NM resolv-prepender triggered by ${1} ${2}."

    # Ensure resolv.conf exists before we try to run podman
    if [[ ! -e /etc/resolv.conf ]] || ! grep -q nameserver /etc/resolv.conf; then
        cp /var/run/NetworkManager/resolv.conf /etc/resolv.conf
    fi

    NAMESERVER_IP=$(/usr/bin/podman run --rm \
        --authfile /var/lib/kubelet/config.json \
        --net=host \
        quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a34f5bdb0cd71e99b446fbc4598594f71b45b149df3506ff4a4d229605f76ed \
        node-ip \
        show \
        "192.168.0.5" \
        "192.168.0.7")
    DOMAIN="wj46ios1218b.1218-39w.qe.rhcloud.com"
    if [[ -n "$NAMESERVER_IP" ]]; then
        logger -s "NM resolv-prepender: Prepending 'nameserver $NAMESERVER_IP' to /etc/resolv.conf (other nameservers from /var/run/NetworkManager/resolv.conf)"
        sed -e "/^search/d" \
            -e "/Generated by/c# Generated by OpenStack resolv prepender NM dispatcher script\nsearch $DOMAIN\nnameserver $NAMESERVER_IP" \
            /var/run/NetworkManager/resolv.conf > /etc/resolv.tmp
    fi
    # Only leave the first 3 nameservers in /etc/resolv.conf
    sed -i ':a $!{N; ba}; s/\(^\|\n\)nameserver/\n# nameserver/4g' /etc/resolv.tmp
    mv -f /etc/resolv.tmp /etc/resolv.conf
    ;;
    *)
    ;;
esac

Removing debug pod ...
Removing debug namespace/openshift-debug-node-mlklp ...

Comment 10 errata-xmlrpc 2020-12-21 13:23:59 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.6.9 security and bug fix update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5614

Comment 11 Ben Nemec 2021-02-12 21:16:26 UTC

*** Bug 1884821 has been marked as a duplicate of this bug. ***

Note You need to log in before you can comment on or make changes to this bug.