Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1743908

Summary: Upgrade Produces Pending CSRs
Product: OpenShift Container Platform Reporter: Nick Schuetz <nschuetz>
Component: Cloud ComputeAssignee: Jan Chaloupka <jchaloup>
Status: CLOSED DUPLICATE QA Contact: Jianwei Hou <jhou>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.1.zCC: agarcial, aos-bugs, brad.ison, eparis, jokerman
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-08-28 09:51:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Nick Schuetz 2019-08-20 23:52:00 UTC
Description of problem:

After an upgrade from 4.1.9 to 4.1.11 there are pending CSRs.

Version-Release number of selected component (if applicable):

4.1.11

How reproducible:

Unknown

Steps to Reproduce:
1. Upgrade from 4.1.9 to 4.1.11
2.
3.

Actual results:

699 pending csr.

Expected results:

No pending CSRs. They should auto approve.

Additional info:

To approve the CSRs:
oc --config=/root/ocp4/auth/kubeconfig get csr -ojson | jq -r '.items[] | select(.status == {} ) | .metadata.name' | xargs oc adm certificate approve

Comment 1 Abhinav Dahiya 2019-08-21 20:16:18 UTC
1) Can you provide the oc adm must-gather
2) Can you provide details about your cluster

The CSR approval is done by cluster-machine-approver.

Comment 5 Brad Ison 2019-08-26 12:30:56 UTC
A particularly important piece of information: What platform is this?

Comment 6 Brad Ison 2019-08-27 15:22:20 UTC
Just a guess: If this is UPI, it is a requirement that the administrator approves serving certificate CSRs. See the docs here:

https://docs.openshift.com/container-platform/4.1/installing/installing_bare_metal/installing-bare-metal.html#installation-approve-csrs_installing-bare-metal

"Because the CSRs rotate automatically, approve your CSRs within an hour of adding the machines to the cluster. If you do not approve them within an hour, the certificates will rotate, and more than two certificates will be present for each node. You must approve all of these certificates. After you approve the initial CSRs, the subsequent node client CSRs are automatically approved by the cluster kube-controller-manager. You must implement a method of automatically approving the kubelet serving certificate requests."

If this isn't a UPI cluster, let us know. Otherwise, we'll mark this as a duplicate of the bug we have open on automating renewals for UPI.

Comment 7 Nick Schuetz 2019-08-27 22:59:30 UTC
(In reply to Brad Ison from comment #5)
> A particularly important piece of information: What platform is this?

This is a UPI on "Bare Metal" install.

Comment 8 Nick Schuetz 2019-08-27 23:07:52 UTC
(In reply to Abhinav Dahiya from comment #1)
> 1) Can you provide the oc adm must-gather
> 2) Can you provide details about your cluster
> 
> The CSR approval is done by cluster-machine-approver.

1. I'll attached the must-gather to this bug.
2. This UPI bare metal install consists of 3 masters and 3 workers. All hosts are running RHCOS 4.1. What other information do you need?

Comment 10 Nick Schuetz 2019-08-27 23:16:34 UTC
If UPI is expected to be manual approval please turn this bug into an RFE for automating the approval in this circumstance.

Comment 11 Brad Ison 2019-08-28 09:51:26 UTC
Marking as a duplicate on of 1737611, which is where we're exploring ways of allowing renewal on platforms that don't have the machine-api available.

*** This bug has been marked as a duplicate of bug 1737611 ***