Bug 2173020

Summary:	Despite successful restore of capsule from the backup, Ui complains of "self signed certificate in certificate chain" for the capsule
Product:	Red Hat Satellite	Reporter:	Sayan Das <saydas>
Component:	Foreman Maintain	Assignee:	satellite6-bugs <satellite6-bugs>
Status:	NEW ---	QA Contact:	Satellite QE Team <sat-qe-bz-list>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	6.13.0	CC:	ehelms
Target Milestone:	Unspecified	Keywords:	Regression, Triaged
Target Release:	Unused	Flags:	bbuckingham: needinfo? (ehelms)
Hardware:	All
OS:	All
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Sayan Das 2023-02-23 17:26:01 UTC

Description of problem:

Even after successfully restoring a capsule server from backup, Visiting Satellite UI --> Infrastructure --> Capsules shows the external capsule in an error state and logs are filled with errors like 

SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)


Version-Release number of selected component (if applicable):

satellite-capsule-6.13.0-6.el8sat.noarch
rubygem-foreman_maintain-1.2.4-1.el8sat.noarch
satellite-maintain-0.0.1-1.el8sat.noarch



How reproducible:

Always

Steps to Reproduce:
1. Install a Satellite 6.13 and Capsule 6.13 and sync some stuff in both
2. Take offline backup ( without pulp ) of both and save them in a network storage
3. Rebuild the OS for both of the VMs with same hostname\IP\Networking as the old OS
4. Follow "14.1. Restoring from a Full Backup" from https://dxp-docp-prod.apps.ext-waf.spoke.prod.us-west-2.aws.paas.redhat.com/documentation/en-us/red_hat_satellite/6.13/html-single/administering_red_hat_satellite/index?lb_target=preview#Restoring_from_a_Full_Backup_admin to restore the backups on satellite and then capsule


Actual results:

--> Satellite successfully restored without any errors and operating fine
--> Capsule successfully restored without any errors but 

   ** Visiting Sat UI --> Infrastructure --> Capsules page shows the capsule in an Error state. 
   ** If we click open the capsule entry we get to see multiple errors like "SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)"


Expected results:


No such errors. 

It's either a flaw in the restore process or we are missing some intermediate steps in the documentation for the capsule part of the restoration.


Additional info:

NOTE: I tested with the both online and offline backups of capsules and the result remains the same.

And I cannot immediately confirm if it's a regression or not as I had last tested this process on 6.9 and it had worked but never had tested it afterwards ( until now ).

Comment 1 Sayan Das 2023-02-23 17:47:39 UTC

To fix this, I additionally had to do these steps :

* Force re-register the capsule


* On Satellite:

# capsule-certs-generate --foreman-proxy-fqdn capsule613.example.com --certs-tar /root/capsule613.example.com-certs.tar --certs-update-all

* And then

--> scp the /root/capsule613.example.com-certs.tar file on the capsule

--> Execute the instructions provided by capsule-certs-generate, on the capsule server

--> Refresh the features of capsule from satellite server. 

    # hammer capsule refresh-features --name capsule613.example.com 


Is it expected that, I would need to execute these steps to complete the restoration of the capsule server?

Comment 2 Brad Buckingham 2023-02-27 14:12:08 UTC

Hi Eric,

Can someone on platform answer the question in comment 1?

Thanks!