Bug 1880463

Summary: [OCP 4.5] kube-apiserver unresponsive - context canceled errors
Product: OpenShift Container Platform Reporter: Mario Abajo <mabajodu>
Component: kube-apiserverAssignee: Lukasz Szaszkiewicz <lszaszki>
Status: CLOSED DUPLICATE QA Contact: Ke Wang <kewang>
Severity: high Docs Contact:
Priority: high    
Version: 4.5CC: aabhishe, aos-bugs, mfojtik, sttts, xxia
Target Milestone: ---   
Target Release: 4.7.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-10-08 11:00:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 6 Stefan Schimanski 2020-10-02 09:43:07 UTC 
The message
 
  killing connection/stream because serving request timed out and response had been started

means that we see timeouts of requests (after 60s) and the response has partly be written. This is not the typical timeout due to slow etcd as then the response is usually not started yet. This looks more like outgoing network issues, i.e. the apiserver cannot send back the data to the client in time. I see something about Istio is known to reconfigure networking quite a bit. The chance is high that the issue is on that side.

Note: we are working improving the error output "apiserver panic'd ...." as this is technically a panic, but a gracefully handled one. It is expected in this situation and is not critical for the apiserver, compare https://github.com/kubernetes/kubernetes/pull/95002.