Bug 2149344

Summary: Failures have been seen during non-CA replica installation, frequently when certmonger is trying to retrieve certificates, getting CA_REJECTED
Product: Red Hat Enterprise Linux 8 Reporter: cilmar <cilmar>
Component: ipaAssignee: Rob Crittenden <rcritten>
Status: CLOSED MIGRATED QA Contact: ipa-qe <ipa-qe>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 8.6CC: frenaud, rcritten, tscherf
Target Milestone: rcKeywords: MigratedToJIRA, Triaged
Target Release: ---   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-18 23:16:47 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description cilmar@redhat.com 2022-11-29 16:27:50 UTC
Description of problem:
 When attempting to install a RHEL 8 Replica from a RHEL 8 master, get an error when trying to issue GSS error stating that the credentials cache is empty and there's insufficient access to issue the certificate.

Version-Release number of selected component (if applicable):
 We could see it present in a migration process (rhel7 + Rhel8)
=================================================================

How reproducible:
The command that is being run is either ipa-replica-install --setup-ca --setup-dns --ssh-trust-dns --no-forwarders --auto-reverse --allow-zone-overlap or ipa-replica-install --setup-dns --ssh-trust-dns --no-forwarders --auto-reverse --allow-zone-overlap. 
=================================================================

Actual results:
[ERRORS]
2022-11-02T20:48:23Z DEBUG certmonger request is in state 'NEWLY_ADDED_READING_KEYINFO'
2022-11-02T20:48:23Z DEBUG certmonger request is in state 'GENERATING_KEY_PAIR'
2022-11-02T20:48:24Z DEBUG certmonger request is in state 'READING_KEYINFO'
2022-11-02T20:48:24Z DEBUG certmonger request is in state 'GENERATING_CSR'
2022-11-02T20:48:25Z DEBUG certmonger request is in state 'CA_REJECTED'
2022-11-02T20:48:25Z DEBUG Cert request 20221102204822 failed: CA_REJECTED (Server at https://<master-server>/ipa/json denied our request, giving up: 2100 (Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Credential cache is empty)).)
2022-11-02T20:48:25Z DEBUG Giving up on cert request 20221102204822
2022-11-02T20:48:25Z DEBUG Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py", line 635, in start_creation
    run_step(full_msg, method)
  File "/usr/lib/python3.6/site-packages/ipaserver/install/service.py", line 621, in run_step
    method()
  File "/usr/lib/python3.6/site-packages/ipaserver/install/dsinstance.py", line 867, in __enable_ssl
    resubmit_timeout=api.env.certmonger_wait_timeout
  File "/usr/lib/python3.6/site-packages/ipalib/install/certmonger.py", line 415, in request_and_wait_for_cert
    "Certificate issuance failed ({}: {})".format(state, ca_error)
RuntimeError: Certificate issuance failed (CA_REJECTED: Server at https://<master-server>/ipa/json denied our request, giving up: 2100 (Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Credential cache is empty)).)

2022-11-02T20:48:25Z DEBUG   [error] RuntimeError: Certificate issuance failed (CA_REJECTED: Server at https://<master-server>/ipa/json denied our request, giving up: 2100 (Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure.  Minor code may provide more information (Credential cache is empty)).)

Additional info:
 - The workaround was setup one specific host on sssd.conf using "ipa_server" parameter.
 - It's being also tracked using:
   https://pagure.io/freeipa/issue/9289

Comment 1 Rob Crittenden 2023-04-20 18:39:45 UTC
We believe this is related to racing replication. We will pick one server to be used throughout a server installation to prevent this.

Comment 2 Rob Crittenden 2023-09-01 17:14:33 UTC
Fixed upstream
master:
https://pagure.io/freeipa/c/45fa43540f6a436c94f0484cec6e252e99d06fa7

Comment 6 RHEL Program Management 2023-09-18 23:11:46 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 7 RHEL Program Management 2023-09-18 23:16:47 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.