Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1519991 - The unidler fails when there is no network plugin
The unidler fails when there is no network plugin
Status: CLOSED ERRATA
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking (Show other bugs)
3.7.0
Unspecified Unspecified
unspecified Severity unspecified
: ---
: 3.9.0
Assigned To: Ben Bennett
hongli
: NeedsTestCase
Depends On: 1529473
Blocks:
  Show dependency treegraph
 
Reported: 2017-12-01 16:05 EST by Ben Bennett
Modified: 2018-03-28 10:15 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Cause: There was a race in the startup code that relied on the node setup setting a field that the userspace proxy needed. When the network plugin was not used (or if it was fast) the userspace proxy setup ran sooner and would read a nil value for the ip address of the node. Then when the proxy was enabled (or the unidler which uses it) and tried to use the nil value, it would crash. Consequence: The userspace proxy would not work, and the unidler (which uses it) would fail, so would never unidle the service. Fix: Add a retry loop to wait for the value to be set. Result: The userspace proxy and unidler work as expected.
Story Points: ---
Clone Of:
Environment:
Last Closed: 2018-03-28 10:14:24 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Origin (Github) 17564 None None None 2017-12-01 16:10 EST
Red Hat Product Errata RHBA-2018:0489 None None None 2018-03-28 10:14 EDT

  None (edit)
Description Ben Bennett 2017-12-01 16:05:43 EST
Description of problem:

When starting openshift with no network plugin, if you ever idle a pod, it will never unidle.


Version-Release number of selected component (if applicable):

3.7.0


How reproducible:

100%


Steps to Reproduce:
1. hack/dind-cluster.sh start -r -n none -N 1
2. create a dc for a hello-openshift pod
3. oc expose dc hello-openshift
4. oc idle hello-openshift
5. curl ...


Actual results:

Nothing happens when you curl.  If you look at the iptables rules on the node after step 4 you see that they are not correct... there are none pointing at the userspace proxy.

Expected results:

The curl should unidle the pod and it should return "Hello Openshift!"


Additional info:

It turns out that the problem is actually with the way the userspace proxy is set up.  It probably happens when that is enabled and idling is out of the picture.  The proxy setup races with the node registration, and the proxy tries to read the node record to retrieve the address, and fails if the registration has not happened yet, but proceeds despite the failure.
Comment 2 hongli 2018-01-29 02:53:28 EST
git clone latest origin build and verified in dind cluster openshift v3.9.0-alpha.3+4f709b4-198, the issue has been fixed.

the userspace proxy set up correctly after idling and pod can be unidled by curl as below:
[root@openshift-node-1 /]# iptables-save | grep unsecure
-A KUBE-PORTALS-CONTAINER -d 172.30.51.61/32 -p tcp -m comment --comment "default/service-unsecure:http" -m tcp --dport 27017 -j DNAT --to-destination 172.17.0.3:42459
-A KUBE-PORTALS-HOST -d 172.30.51.61/32 -p tcp -m comment --comment "default/service-unsecure:http" -m tcp --dport 27017 -j DNAT --to-destination 172.17.0.3:42459


[root@openshift-node-1 /]# curl 172.30.51.61:27017
Hello-OpenShift-1 http-8080
Comment 5 errata-xmlrpc 2018-03-28 10:14:24 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0489

Note You need to log in before you can comment on or make changes to this bug.