Bug 1999771

Summary: revert "force cert rotation every couple days for development" in 4.10
Product: OpenShift Container Platform Reporter: Stefan Schimanski <sttts>
Component: kube-apiserverAssignee: Abu Kashem <akashem>
Status: CLOSED ERRATA QA Contact: Rahul Gangwar <rgangwar>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 4.10CC: akashem, aos-bugs, cfergeau, kewang, mfojtik, nstielau, prkumar, rgangwar, slaznick, wlewis, xxia
Target Milestone: ---Keywords: Reopened
Target Release: 4.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1969404
: 2050407 (view as bug list) Environment:
Last Closed: 2022-03-10 16:06:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 2050407    
Bug Blocks:    

Comment 5 Rahul Gangwar 2022-02-05 08:03:07 UTC
Using below steps to verify this bug:
$ cat scripts/check_secret_expiry.sh
FILE="$1"
if [ ! -f "$1" ]; then
  echo "must provide \$1" && exit 0
fi
export IFS=$'\n'
for i in `cat "$FILE"`
do
  if `echo "$i" | grep "^#" > /dev/null`; then
    continue
  fi
  NS=`echo $i | cut -d ' ' -f 1`
  SECRET=`echo $i | cut -d ' ' -f 2`
  rm -f tls.crt; oc extract secret/$SECRET -n $NS --confirm > /dev/null
  echo "Check cert dates of $SECRET in project $NS:"
  openssl x509 -noout --date -in tls.crt; echo
done

$ cat certs.txt
openshift-kube-controller-manager-operator csr-signer-signer
openshift-kube-controller-manager-operator csr-signer
openshift-kube-controller-manager kube-controller-manager-client-cert-key
openshift-kube-apiserver-operator aggregator-client-signer
openshift-kube-apiserver aggregator-client
openshift-kube-apiserver external-loadbalancer-serving-certkey
openshift-kube-apiserver internal-loadbalancer-serving-certkey
openshift-kube-apiserver service-network-serving-certkey
openshift-config-managed kube-controller-manager-client-cert-key
openshift-config-managed kube-scheduler-client-cert-key
openshift-kube-scheduler kube-scheduler-client-cert-key


oc get clusterversion
NAME      VERSION                         AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.10.0-0.ci-2022-02-04-082923   True        False         19h     Cluster version is 4.10.0-0.ci-2022-02-04-082923

Checking the Certs,  they are with one day expiry times, this is as expected.

bash check_secret_expiry.sh certs.txt
Check cert dates of csr-signer-signer in project openshift-kube-controller-manager-operator:
notBefore=Feb  4 11:28:19 2022 GMT
notAfter=Feb  5 11:28:19 2022 GMT

Check cert dates of csr-signer in project openshift-kube-controller-manager-operator:
notBefore=Feb  4 11:44:31 2022 GMT
notAfter=Feb  5 11:28:19 2022 GMT

Check cert dates of kube-controller-manager-client-cert-key in project openshift-kube-controller-manager:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

Check cert dates of aggregator-client-signer in project openshift-kube-apiserver-operator:
notBefore=Feb  4 11:28:17 2022 GMT
notAfter=Feb  5 11:28:17 2022 GMT

Check cert dates of aggregator-client in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:36 2022 GMT
notAfter=Feb  5 11:28:17 2022 GMT

Check cert dates of external-loadbalancer-serving-certkey in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:34 2022 GMT
notAfter=Mar  6 11:44:35 2022 GMT

Check cert dates of internal-loadbalancer-serving-certkey in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:48 2022 GMT
notAfter=Mar  6 11:44:49 2022 GMT

Check cert dates of service-network-serving-certkey in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:35 2022 GMT
notAfter=Mar  6 11:44:36 2022 GMT

Check cert dates of kube-controller-manager-client-cert-key in project openshift-config-managed:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

Check cert dates of kube-scheduler-client-cert-key in project openshift-config-managed:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

Check cert dates of kube-scheduler-client-cert-key in project openshift-kube-scheduler:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT


After one day, checking the cert rotation again, 
bash check_secret_expiry.sh certs.txt
Check cert dates of csr-signer-signer in project openshift-kube-controller-manager-operator:
notBefore=Feb  5 06:40:44 2022 GMT
notAfter=Apr  6 06:40:45 2022 GMT

Check cert dates of csr-signer in project openshift-kube-controller-manager-operator:
notBefore=Feb  5 06:43:45 2022 GMT
notAfter=Mar  7 06:43:46 2022 GMT

Check cert dates of kube-controller-manager-client-cert-key in project openshift-kube-controller-manager:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

Check cert dates of aggregator-client-signer in project openshift-kube-apiserver-operator:
notBefore=Feb  5 06:40:52 2022 GMT
notAfter=Mar  7 06:40:53 2022 GMT

Check cert dates of aggregator-client in project openshift-kube-apiserver:
notBefore=Feb  5 06:43:53 2022 GMT
notAfter=Mar  7 06:40:53 2022 GMT

Check cert dates of external-loadbalancer-serving-certkey in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:34 2022 GMT
notAfter=Mar  6 11:44:35 2022 GMT

Check cert dates of internal-loadbalancer-serving-certkey in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:48 2022 GMT
notAfter=Mar  6 11:44:49 2022 GMT

Check cert dates of service-network-serving-certkey in project openshift-kube-apiserver:
notBefore=Feb  4 11:44:35 2022 GMT
notAfter=Mar  6 11:44:36 2022 GMT

Check cert dates of kube-controller-manager-client-cert-key in project openshift-config-managed:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

Check cert dates of kube-scheduler-client-cert-key in project openshift-config-managed:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

Check cert dates of kube-scheduler-client-cert-key in project openshift-kube-scheduler:
notBefore=Feb  4 11:44:52 2022 GMT
notAfter=Mar  6 11:44:53 2022 GMT

All certs are as expected expiry time 30 days, cert of csr-signer-signer in project openshift-kube-controller-manager-operator with 2 * 30.

Comment 7 errata-xmlrpc 2022-03-10 16:06:30 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.10.3 security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0056