Bug 1732182

Summary: [DOCS] crio.service does not start when there is no default route
Product: OpenShift Container Platform Reporter: Eric Rich <erich>
Component: DocumentationAssignee: Kathryn Alexander <kalexand>
Status: ASSIGNED --- QA Contact: Johnny Liu <jialiu>
Severity: medium Docs Contact: Vikram Goyal <vigoyal>
Priority: medium    
Version: 4.1.0CC: aos-bugs, gblomqui, jokerman, mmccomas, pthomas, scuppett
Target Milestone: ---   
Target Release: 4.2.z   
Hardware: Unspecified   
OS: Unspecified   
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Eric Rich 2019-07-22 22:32:58 UTC
Description of problem:

> Jul 22 20:35:42 localhost.localdomain systemd[1]: Starting Open Container Initiative Daemon...
> Jul 22 20:35:42 localhost.localdomain crio[10396]: time="2019-07-22 20:35:42.483075415Z" level=fatal msg="no default routes found in "/proc/net/route" or "/proc/net/ipv6_route""
> Jul 22 20:35:42 localhost.localdomain systemd[1]: crio.service: Main process exited, code=exited, status=1/FAILURE
> Jul 22 20:35:42 localhost.localdomain systemd[1]: crio.service: Failed with result 'exit-code'.
> Jul 22 20:35:42 localhost.localdomain systemd[1]: Failed to start Open Container Initiative Daemon.

Version-Release number of selected component (if applicable): 4.1.6
How reproducible: 100% 

Steps to Reproduce:
1. Create an isolated network 
2. Complete a bare metal install 

Actual results:

The bootstrap process fails; because cri-o never starts. 

Expected results:

Cri-o should be able to start on a system without a default route. 

Additional info:

Comment 4 Seth Jennings 2019-07-23 16:54:33 UTC
Currently, crio uses the default route to determine the "primary IP" for the node.  It uses the IP address in the same subnet as the default route to determine this.

However, I think the default route should really only be used as a hint to resolve abiguity in the case of multiple IPs.  It a single IP case i.e. common case, it is trivial to determine the primary IP.

Comment 7 Seth Jennings 2019-07-29 22:00:49 UTC
Just got a change to test; doesn't seem to work https://github.com/cri-o/cri-o/pull/2662#issuecomment-516178852

Comment 10 Seth Jennings 2019-08-20 20:17:20 UTC
While testing our fix for this, I discovered that the kube-apiserver also can't start if there is no default route.

This will not be fixed in time for 4.2.  In fact, I'm less convinced that it is something we should "fix" now that I see the upstream kube components have the same limitation.

Disconnected and proxy documentation should indicate that a default route is required on nodes, even if that route can not reach the internet.