Bug 1821493 - /readyz should start reporting failure on shutdown initiation
Summary: /readyz should start reporting failure on shutdown initiation
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: kube-apiserver
Version: 4.4
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.3.z
Assignee: Abu Kashem
QA Contact: Ke Wang
Depends On: 1811169 1811198
Blocks: 1811200 1821494 1821495
TreeView+ depends on / blocked
Reported: 2020-04-06 23:33 UTC by Abu Kashem
Modified: 2020-04-20 17:09 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1811198
: 1821494 (view as bug list)
Last Closed: 2020-04-20 17:08:56 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:1482 0 None None None 2020-04-20 17:09:17 UTC

Comment 5 Ke Wang 2020-04-10 09:50:03 UTC
Verified with OCP build 4.3.0-0.nightly-2020-04-10-021124, detail see below,

- In one terminal, enter into master 
$ master=$(oc get nodes | grep master | head -1 | cut -d " " -f1)
$ oc debug node/$master
sh-4.2# chroot /host
sh-4.4# while true; do curl -k --silent --show-error https://localhost:6443/readyz ; done

- In another terminal,
$ debug_pod=$(oc get pod -A | grep debug | awk '{print $2}')
$ oc rsh pod/$debug_pod
sh-4.2# chroot /host
sh-4.4# kill -INT `ps aux | grep " kube-apiserver " | grep -v grep | awk '{print $2}'`

- In the first terminal, check the output, after above kill, can immediately see:
[+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-discovery-available ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/openshift.io-clientCA-reload ok
[+]poststarthook/openshift.io-requestheader-reload ok
[+]poststarthook/quota.openshift.io-clusterquotamapping ok
[+]poststarthook/openshift.io-kubernetes-informers-synched ok
[+]poststarthook/openshift.io-startkubeinformers ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/apiservice-wait-for-first-sync ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[-]shutdown failed: reason withheld
healthz check failed

The endpoint of readyz will start returning failure as soon as kube-apiserver shutdown is initiated.

Comment 9 errata-xmlrpc 2020-04-20 17:08:56 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.