Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1723928

Summary: Backport kubelet recovery script to 4.1.z
Product: OpenShift Container Platform Reporter: Ryan Phillips <rphillips>
Component: NodeAssignee: Ryan Phillips <rphillips>
Status: CLOSED ERRATA QA Contact: Sunil Choudhary <schoudha>
Severity: low Docs Contact:
Priority: low    
Version: 4.1.zCC: aos-bugs, jokerman, mmccomas, sponnaga, wsun
Target Milestone: ---   
Target Release: 4.1.z   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: 4.1.6
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-07-23 18:12:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ryan Phillips 2019-06-25 19:06:00 UTC
Description of problem:
The kubelet disaster recovery script can be backported to 4.1.z.


https://github.com/openshift/machine-config-operator/pull/792

Comment 2 Ryan Phillips 2019-06-27 15:01:24 UTC
QA tests were done via https://bugzilla.redhat.com/show_bug.cgi?id=1711910

Comment 4 Sunil Choudhary 2019-07-10 12:39:13 UTC
Verified on 4.1.0-0.nightly-2019-07-09-205457. Followed cert recover steps [1] and used /usr/local/bin/recover-kubeconfig.sh script on master host to generate kubeconfig

[1] https://docs.openshift.com/container-platform/4.1/disaster_recovery/scenario-3-expired-certs.html

Will open a doc bug to remove the script from above doc section.

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.1.0-0.nightly-2019-07-09-205457   True        False         79m     Cluster version is 4.1.0-0.nightly-2019-07-09-205457

$ oc get csr
NAME        AGE     REQUESTOR                                                                   CONDITION
csr-2dvtf   7m43s   system:node:ip-10-0-167-10.ap-south-1.compute.internal                      Approved,Issued
csr-4t9ml   92m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-55ghx   93m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-6dt86   6m50s   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-6xrtc   92m     system:node:ip-10-0-167-10.ap-south-1.compute.internal                      Approved,Issued
csr-85hp9   6m16s   system:node:ip-10-0-140-178.ap-south-1.compute.internal                     Approved,Issued
csr-8vf5d   86m     system:node:ip-10-0-132-36.ap-south-1.compute.internal                      Approved,Issued
csr-94tb5   86m     system:node:ip-10-0-152-186.ap-south-1.compute.internal                     Approved,Issued
csr-ccfzm   10m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-dzmdr   86m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-gt9zw   3m9s    system:node:ip-10-0-132-36.ap-south-1.compute.internal                      Approved,Issued
csr-k24zn   92m     system:node:ip-10-0-140-178.ap-south-1.compute.internal                     Approved,Issued
csr-nd94h   91s     system:node:ip-10-0-152-186.ap-south-1.compute.internal                     Approved,Issued
csr-pdng7   86m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-qg8hw   8m28s   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-s4m9k   3m44s   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-sf5v7   10m     system:node:ip-10-0-151-133.ap-south-1.compute.internal                     Approved,Issued
csr-tcnj6   92m     system:node:ip-10-0-151-133.ap-south-1.compute.internal                     Approved,Issued
csr-xng84   92m     system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued
csr-xxhf9   2m34s   system:serviceaccount:openshift-machine-config-operator:node-bootstrapper   Approved,Issued


# cat /usr/local/bin/recover-kubeconfig.sh 
#!/bin/bash

set -eou pipefail

# context
intapi=$(oc get infrastructures.config.openshift.io cluster -o "jsonpath={.status.apiServerURL}")
context="$(oc config current-context)"
# cluster
cluster="$(oc config view -o "jsonpath={.contexts[?(@.name==\"$context\")].context.cluster}")"
server="$(oc config view -o "jsonpath={.clusters[?(@.name==\"$cluster\")].cluster.server}")"
# token
ca_crt_data="$(oc get secret -n openshift-machine-config-operator node-bootstrapper-token -o "jsonpath={.data.ca\.crt}" | base64 --decode)"
namespace="$(oc get secret -n openshift-machine-config-operator node-bootstrapper-token  -o "jsonpath={.data.namespace}" | base64 --decode)"
token="$(oc get secret -n openshift-machine-config-operator node-bootstrapper-token -o "jsonpath={.data.token}" | base64 --decode)"

export KUBECONFIG="$(mktemp)"
kubectl config set-credentials "kubelet" --token="$token" >/dev/null
ca_crt="$(mktemp)"; echo "$ca_crt_data" > $ca_crt
kubectl config set-cluster $cluster --server="$intapi" --certificate-authority="$ca_crt" --embed-certs >/dev/null
kubectl config set-context kubelet --cluster="$cluster" --user="kubelet" >/dev/null
kubectl config use-context kubelet >/dev/null
cat "$KUBECONFIG"

Comment 6 errata-xmlrpc 2019-07-23 18:12:12 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:1766