Bug 1519991

Summary:	The unidler fails when there is no network plugin
Product:	OpenShift Container Platform	Reporter:	Ben Bennett <bbennett>
Component:	Networking	Assignee:	Ben Bennett <bbennett>
Status:	CLOSED ERRATA	QA Contact:	Hongan Li <hongli>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.7.0	CC:	aos-bugs, bmeng, hongli, zzhao
Target Milestone:	---
Target Release:	3.9.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:	Cause: There was a race in the startup code that relied on the node setup setting a field that the userspace proxy needed. When the network plugin was not used (or if it was fast) the userspace proxy setup ran sooner and would read a nil value for the ip address of the node. Then when the proxy was enabled (or the unidler which uses it) and tried to use the nil value, it would crash. Consequence: The userspace proxy would not work, and the unidler (which uses it) would fail, so would never unidle the service. Fix: Add a retry loop to wait for the value to be set. Result: The userspace proxy and unidler work as expected.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2018-03-28 14:14:24 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1529473
Bug Blocks:

Description Ben Bennett 2017-12-01 21:05:43 UTC

Description of problem:

When starting openshift with no network plugin, if you ever idle a pod, it will never unidle.


Version-Release number of selected component (if applicable):

3.7.0


How reproducible:

100%


Steps to Reproduce:
1. hack/dind-cluster.sh start -r -n none -N 1
2. create a dc for a hello-openshift pod
3. oc expose dc hello-openshift
4. oc idle hello-openshift
5. curl ...


Actual results:

Nothing happens when you curl.  If you look at the iptables rules on the node after step 4 you see that they are not correct... there are none pointing at the userspace proxy.

Expected results:

The curl should unidle the pod and it should return "Hello Openshift!"


Additional info:

It turns out that the problem is actually with the way the userspace proxy is set up.  It probably happens when that is enabled and idling is out of the picture.  The proxy setup races with the node registration, and the proxy tries to read the node record to retrieve the address, and fails if the registration has not happened yet, but proceeds despite the failure.

Comment 2 Hongan Li 2018-01-29 07:53:28 UTC

git clone latest origin build and verified in dind cluster openshift v3.9.0-alpha.3+4f709b4-198, the issue has been fixed.

the userspace proxy set up correctly after idling and pod can be unidled by curl as below:
[root@openshift-node-1 /]# iptables-save | grep unsecure
-A KUBE-PORTALS-CONTAINER -d 172.30.51.61/32 -p tcp -m comment --comment "default/service-unsecure:http" -m tcp --dport 27017 -j DNAT --to-destination 172.17.0.3:42459
-A KUBE-PORTALS-HOST -d 172.30.51.61/32 -p tcp -m comment --comment "default/service-unsecure:http" -m tcp --dport 27017 -j DNAT --to-destination 172.17.0.3:42459


[root@openshift-node-1 /]# curl 172.30.51.61:27017
Hello-OpenShift-1 http-8080

Comment 5 errata-xmlrpc 2018-03-28 14:14:24 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489