Bug 2091170

Summary: Unable to deploy Hypershift operator on MCE hub using ManagedClusterAddOn without including s3 secret
Product: Red Hat Advanced Cluster Management for Kubernetes Reporter: Chad Crum <ccrum>
Component: HypershiftAssignee: Roke Jung <rjung>
Status: CLOSED ERRATA QA Contact: Chad Crum <ccrum>
Severity: high Docs Contact:
Priority: unspecified    
Version: rhacm-2.5CC: ccrum, njean, phwu
Target Milestone: ---Flags: bot-tracker-sync: rhacm-2.5.z+
Target Release: rhacm-2.5.1   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2100116 (view as bug list) Environment:
Last Closed: 2022-09-13 20:06:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2100116    

Description Chad Crum 2022-05-27 17:30:02 UTC
Description of problem:
In the case of using MCE/Hypershift on BM (As opposed to AWS provider) and using agent based hosted control planes, there is no need for s3 secret "hypershift-operator-oidc-provider-s3-credentials", yet the hypershift operator pod will not deploy without the s3 secret set.


Version-Release number of selected component (if applicable):
Latest MCE2.0 downstream snapshot
OCP 4.10.x hub cluster

How reproducible:
100%

Steps to Reproduce:
1. Ensure hub cluster has a managedcluster resource
2. Create managedclusteraddon, referencing hub managedcluster
oc apply -f - <<EOF
apiVersion: addon.open-cluster-management.io/v1alpha1
kind: ManagedClusterAddOn
metadata:
  name: hypershift-addon
  namespace: local-cluster   # name of managed cluster
spec:
  installNamespace: open-cluster-management-agent-addon
EOF

3. Check managedclusteraddon status in the local-cluster namespace

Actual results:
Hypershift addon remains unknown

oc get managedclusteraddon -n local-cluster
NAME               AVAILABLE   DEGRADED   PROGRESSING
hypershift-addon   Unknown                
work-manager       True           


Expected results:
AVAILABLE moves to true and hypershift operator deploys in hypershift operator name space

Additional info:

## If I do not create the S3 key: 


# hypershift-addon shows the following errors and stays unknown
 oc get managedclusteraddons.addon.open-cluster-management.io 
NAME               AVAILABLE   DEGRADED   PROGRESSING
hypershift-addon   Unknown                
work-manager       True       


    Last Transition Time:  2022-05-27T17:16:57Z
    Message:               The status of hypershift-addon add-on is unknown.
    Reason:                ManagedClusterAddOnLeaseNotFound
    Status:                Unknown
    Type:                  Available


# Hypershift addon gent pod tries to launch hypershift with the s3 secret, but the second is not created so hypershift never runs.

oc logs -f hypershift-addon-agent-5499c58967-cz8vk -n open-cluster-management-agent-addon                                                                                                                                                                   
I0527 17:24:17.124602       1 request.go:665] Waited for 1.028914334s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/operators.coreos.com/v1alpha2?timeout=32s                                                              
2022-05-27T17:24:23.698Z        INFO    agent.agent-reconciler  agent/hypershift.go:192 enter runHypershiftInstall                                                                                                                                                              
2022-05-27T17:24:23.715Z        ERROR   agent.agent-reconciler  agent/hypershift.go:206 failed to get bucket secret(local-cluster/hypershift-operator-oidc-provider-s3-credentials) from hub, will retry.       {"error": "secrets \"hypershift-operator-oidc-provider-s3-creden
tials\" not found"}                                                               [...]
2022-05-27T17:24:23.715Z        INFO    agent.agent-reconciler  agent/hypershift.go:207 exit runHypershiftInstall
2022-05-27T17:24:23.715Z        ERROR   agent.agent-reconciler  agent/hypershift.go:103 failed to run hypershift cmd    {"error": "secrets \"hypershift-operator-oidc-provider-s3-credentials\" not found"}

Comment 1 Chad Crum 2022-06-01 18:48:41 UTC
Work around to get hypershift operator deployed is to create a dummy s3 secret:

# Create file with dummy aws s3 credentials called s3-dummy-credentials and content below
[default]
aws_access_key_id     = ABCDEFGHIJKLMNOPQRST
aws_secret_access_key = ABCDEFGHIJKLMNOPQRSTUVWXYZabcd1234567890

# Create secret using previously created file (namespace should match the namespace used in add-on operator step)
oc create secret generic hypershift-operator-oidc-provider-s3-credentials --from-file=credentials=s3-dummy-credentials --from-literal=bucket=bucket --from-literal=region=us-east-1 -n local-cluster

Comment 3 Nelson Jean 2022-07-04 19:45:09 UTC
Hi @ccrum , can you please verify and close if no issues. Thanks!

Comment 4 phwu 2022-07-05 18:59:12 UTC
Same as this issue reported in 2.5.1: https://github.com/stolostron/backlog/issues/23868

Fix will be in 2.5.2

Comment 5 bot-tracker-sync 2022-07-05 20:26:05 UTC
G2Bsync 1175397613 comment 
 nelsonjean Tue, 05 Jul 2022 18:57:50 UTC 
 G2Bsync 

This issue was not fixed in MCE 2.0.1 / ACM 2.5.1. Moving it to 2.5.2.

Comment 7 Chad Crum 2022-07-13 21:40:11 UTC
Confirmed fixed with latest mce ds snapshot.

Comment 12 errata-xmlrpc 2022-09-13 20:06:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Critical: Red Hat Advanced Cluster Management 2.5.2 security fixes and bug fixes), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6507