Cause: There was a race in the startup code that relied on the node setup setting a field that the userspace proxy needed. When the network plugin was not used (or if it was fast) the userspace proxy setup ran sooner and would read a nil value for the ip address of the node. Then when the proxy was enabled (or the unidler which uses it) and tried to use the nil value, it would crash.
Consequence: The userspace proxy would not work, and the unidler (which uses it) would fail, so would never unidle the service.
Fix: Add a retry loop to wait for the value to be set.
Result: The userspace proxy and unidler work as expected.
Description of problem:
When starting openshift with no network plugin, if you ever idle a pod, it will never unidle.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. hack/dind-cluster.sh start -r -n none -N 1
2. create a dc for a hello-openshift pod
3. oc expose dc hello-openshift
4. oc idle hello-openshift
5. curl ...
Nothing happens when you curl. If you look at the iptables rules on the node after step 4 you see that they are not correct... there are none pointing at the userspace proxy.
The curl should unidle the pod and it should return "Hello Openshift!"
It turns out that the problem is actually with the way the userspace proxy is set up. It probably happens when that is enabled and idling is out of the picture. The proxy setup races with the node registration, and the proxy tries to read the node record to retrieve the address, and fails if the registration has not happened yet, but proceeds despite the failure.
git clone latest origin build and verified in dind cluster openshift v3.9.0-alpha.3+4f709b4-198, the issue has been fixed.
the userspace proxy set up correctly after idling and pod can be unidled by curl as below:
[root@openshift-node-1 /]# iptables-save | grep unsecure
-A KUBE-PORTALS-CONTAINER -d 172.30.51.61/32 -p tcp -m comment --comment "default/service-unsecure:http" -m tcp --dport 27017 -j DNAT --to-destination 172.17.0.3:42459
-A KUBE-PORTALS-HOST -d 172.30.51.61/32 -p tcp -m comment --comment "default/service-unsecure:http" -m tcp --dport 27017 -j DNAT --to-destination 172.17.0.3:42459
[root@openshift-node-1 /]# curl 172.30.51.61:27017
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.