Bug 2102495

Summary:	ManagedClusters in Pending import state after ACM hub migration
Product:	Red Hat Advanced Cluster Management for Kubernetes	Reporter:	Nikhil Gupta <ngupta>
Component:	Cluster Lifecycle	Assignee:	Le Yang <leyan>
Status:	CLOSED ERRATA	QA Contact:	Thuy Nguyen <thnguyen>
Severity:	high	Docs Contact:	Christopher Dawson <cdawson>
Priority:	unspecified
Version:	rhacm-2.4	CC:	dhuynh, huichen, zyin
Target Milestone:	---	Flags:	dhuynh: qe_test_coverage+ bot-tracker-sync: rhacm-2.4.z+
Target Release:	rhacm-2.4.6
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-09-26 14:52:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Nikhil Gupta 2022-06-30 04:31:19 UTC

Description of the problem:

A new hub cluster has been created and the ACM hub backup/restore operation has been carried out as described in the docs: 

https://access.redhat.com/documentation/en-us/red_hat_advanced_cluster_management_for_kubernetes/2.4/html/clusters/managing-your-clusters#hub-backup-and-restore

After the restore operation is completed some of the managed clusters have been imported successfully to the ACM hub. Several others (~half) are stuck in the  'Pending import/Unknown' state.


Release version:
advanced-cluster-management.v2.4.4

OCP version: 4.9.37

Steps to reproduce:
1. Deploy 15+ clusters through the ACM hub (Created via Hive)
2. Perform the Backup procedure on the old ACM hub
3. Create a new ACM hub
4. Shutdown the old ACM Hub 
5. Perform Restore operation on the new ACM hub

Actual results:
Some of the managed clusters have been imported successfully to the ACM hub. However, Several others (~half) are stuck in the  'Pending import/Unknown' state.

Expected results:
All managed clusters should be restored successfully as they all were created using ACM(Hive).

Additional info:
The failing clusters indicate it is still attempting to communicate with the Old ACM hub, not the new one.

Comment 2 bot-tracker-sync 2022-07-01 03:52:15 UTC

G2Bsync 1171868820 comment 
 zhiweiyin318 Fri, 01 Jul 2022 02:47:39 UTC 
 G2Bsync
discuss slack thread https://coreos.slack.com/archives/C0282HW2YHZ/p1656402856382809
the kubeconfig of `qasp-uat-cct-01`  hive cluster in `qasp-uat-cct-01-0-xnxvg-admin-kubeconfig `secret is incorrect.
we cannot access the cluster by this kubeconfig with error from hub cluster.
```
Unable to connect to the server: x509: certificate signed by unknown authority
```
need DR squad and hive team take a look to figure out why this kubeconfig is incorrect.

Comment 3 zyin@redhat.com 2022-09-13 06:57:44 UTC

added a known issue for this. 
1.2.8.7. Restored Hive managed clusters might be able to connect with the new hub cluster in  https://access.redhat.com/documentation/en-us/red_hat_advanced_cluster_management_for_kubernetes/2.6/html/release_notes/red-hat-advanced-cluster-management-for-kubernetes-release-notes#backup-known-issues

Comment 4 Thuy Nguyen 2022-09-13 15:26:23 UTC

Doc verified.

Comment 9 errata-xmlrpc 2022-09-26 14:52:40 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Critical: Red Hat Advanced Cluster Management 2.4.6 security update and bug fixes), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:6696