Bug 1242961

(In reply to Ryan Howe from comment #0)
> Or have the OpenShift master process know
> that you are using a proxy, and know that you have added a docker registry
> and know that you want to not proxy it then oenshift-master editing the
> files in /etc/sysconfig/master

Raised Bug 1253675 for this purpose.

> a way to either assign NO_PROXY an IP wildcards (172.30.) or CIDR notation value. 

This one still has its own value though, so keeping this open to track it.

NO_PROXY is not meant to take cidr ranges (although one example in the docs shows it).  https://github.com/docker/docker/issues/9145

I tested this out and confirmed that a push does not work with a range when added to /etc/sysconfig/docker but worked when I added the IP.

So, it seems that the way to do this would be to add the registry IP address to the docker config file and restart the daemon.  I assume that this would be a cluster wide operation and need to occur on N nodes.  It seems like more of an admin task that would need to be managed rather than something we could work into an OpenShift command.

I will get some second opinions and see if there is a good solution here.

Discussed with Clayton, this is more appropriate as a setup task in the installer.

We'll add support for this in ansible and the quick installer.

Andrew, the first part of this will be to add support in our ansible playbook for setting this proxy.

Hello Engineering team,

A customer provided a workaround for this issue. Could you please take a look and validate if it is an acceptable solution?

" it has a little bit of a chicken-and-the-egg scenario right now, but if it was included with the base openshift install it might work out of the box.


So the root cause of needing to specify things in no_proxy, is that the EXTERNAL proxy doesn't know about the SDN networks inside openshift, if it did it would route the traffic appropriately and no_proxy wouldn't be needed.

So this means we have two options:
1) get the external proxy to participate in the SDN as a unscheduled node
2) run a proxy inside openshift


1) has some issues clearly, the proxy has to be rhel7, has to have openshift installed, etc

for 2) the prereqs are pretty small, you need to know what networks openshift is using.  after that stand up a squid pod, setting those networks as always direct and setting the external proxy as the cache parent.   then configure all of openshift to use that pod as it's http(s)_proxy.

this way the squid pod shuttles traffic internally only and sends everything else up to the parent proxy"

Ricardo,

I think this is definitely an acceptable workaround but not something that we should advertise via the docs, etc due to the chicken/egg issues. I've setup the configuration you've described in #2 and it works great, however, if the squid pod were to go down and the squid image weren't pulled on the node the pod moved to then the docker http_proxy configuration would have to be remedied.

There's a drive to use dns for the registry and to add skydns to the node's resolver which would allow for a subdomain to be configured w/ no_proxy. In my opinion this would be the most maintainable solution moving forward.

GH issues related to that work:
https://github.com/openshift/origin/issues/2581
https://github.com/openshift/origin/issues/2540

This will be solved by https://trello.com/c/Q1oYRFax/21-configure-node-s-resolv-conf-to-use-skydns-from-master

There's three parts to fixing this, 1) bring cluster dns to the node, 2) configure HTTP_PROXY, HTTPS_PROXY, NO_PROXY, and BuildDefaults Admission controller, and 3) figure out how not to break masters connecting to pod ips. 

1) Almost ready to merge, under review.
https://github.com/openshift/openshift-ansible/pull/1588

2) Under review but cannot be pushed without addressing #3
https://github.com/openshift/openshift-ansible/pull/1385

3) Needs to be addressed before 2 is accepted
https://github.com/openshift/origin/issues/7694

3) was fixed by https://github.com/openshift/origin/pull/8016, which has merged.

How are 1 and 2 looking?

Moving this to installer.

1) DNS work is merged
2) 'Global Proxy Config' is being reviewed, hopefully merges tomorrow at the latest.

Need to ensure that when we know dnsmasq is in use we deploy the registry using dns rather than IP. reaching out to miminar to figure that out.

*** Bug 1340511 has been marked as a duplicate of this bug. ***

PM board card to track registry via DNS:

https://trello.com/c/BRPwto2S/896-8-registry-use-service-dns-for-the-internal-registry-instead-of-ip-address

(note that this should work now, this card will mostly be documentation and installer work).

.svc and .cluster.local should both be added to the NO_PROXY variable so for new installs of 3.6 this should be fixed. Moving to QA to verify

.svc and .cluster.local are added to NO_PROXY automatically in proxy environment.

NO_PROXY='.cluster.local,.svc,openshift-129.lab.sjc.redhat.com'

Now images are pushed to `docker-registry.default.svc:5000` directly and successfully.

This feature is already implemented since 3.6.

Retested version: openshift-ansible-3.6.173.0.5-3.git.0.522a92a.el7.noarch.rpm

This was fixed in the 3.6 release.