Bug 2277005

Summary: Upgrading a FIPS enabled Red Hat Satellite 6.14 to 6.15 face issues with Candlepin keystore
Product: Red Hat Satellite Reporter: Paul Dudley <pdudley>
Component: UpgradesAssignee: Evgeni Golov <egolov>
Status: ON_DEV --- QA Contact: Satellite QE Team <sat-qe-bz-list>
Severity: high Docs Contact:
Priority: high    
Version: 6.15.0CC: ahumbe, bmertens, egolov, ehelms, gsulliva, maford, msunil, osousa, pdwyer, rlavi, saydas, vijsingh, wderick
Target Milestone: streamKeywords: PrioBumpGSS, Triaged, Upgrades
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: foreman-installer-3.11.0-0.1.develop.20240429031107gitd83807b Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2279323 (view as bug list) Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Paul Dudley 2024-04-24 21:24:11 UTC 
Description

When upgrading to Satellite 6.15 we are seeing issues related to regenerating and reimporting the candlepin-ca; 
~~~
2024-04-24 11:22:55 [ERROR ] [configure] /Stage[main]/Certs::Candlepin/Truststore_certificate[/etc/candlepin/certs/truststore:candlepin-ca]/ensure: change from 'absent' to 'present' failed: Execution of '/bin/keytool -import -v -noprompt -storetype pkcs12 -keystore /etc/candlepin/certs/truststore -alias candlepin-ca -file /etc/candlepin/certs/candlepin-ca.crt -storepass:file /etc/pki/katello/truststore_password-file -J-Dcom.redhat.fips=false' returned 1: keytool error: java.io.IOException: keystore password was incorrect
2024-04-24 11:22:55 [ERROR ] [configure] java.io.IOException: keystore password was incorrect
2024-04-24 11:22:55 [ERROR ] [configure] at sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:2089)
2024-04-24 11:22:55 [ERROR ] [configure] at java.security.KeyStore.load(KeyStore.java:1445)
2024-04-24 11:22:55 [ERROR ] [configure] at sun.security.tools.keytool.Main.doCommands(Main.java:839)
2024-04-24 11:22:55 [ERROR ] [configure] at sun.security.tools.keytool.Main.run(Main.java:380)
2024-04-24 11:22:55 [ERROR ] [configure] at sun.security.tools.keytool.Main.main(Main.java:373)
2024-04-24 11:22:55 [ERROR ] [configure] Caused by: java.security.UnrecoverableKeyException: failed to decrypt safe contents entry: javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.
~~~

Steps to reproduce:

* Install RHEL 8.9 and enable FIPS
* Install a Satellite 6.14 on top of it.
* Try to upgrade the instance to Satellite 6.15.0

Actual Results:

Errors with candlepin keystore password as mentioned above

Expected Results:

No such errors and the upgrade should happen without any such issues.
Comment 4 Evgeni Golov 2024-04-25 08:00:33 UTC 
Created redmine issue https://projects.theforeman.org/issues/37384 from this bug
Comment 5 Evgeni Golov 2024-04-25 08:36:38 UTC 
Workaround: rm /etc/candlepin/certs/truststore /etc/candlepin/certs/keystore
These files will be regenerated by the installer.

Patch in the works: https://github.com/theforeman/puppet-certs/pull/444