RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1999142 - RHEL 8.5 IPA Replica setup fails against a RHEL 7.9 IPA server
Summary: RHEL 8.5 IPA Replica setup fails against a RHEL 7.9 IPA server
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: ipa
Version: 8.5
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Rob Crittenden
QA Contact: ipa-qe
URL:
Whiteboard:
Depends On:
Blocks: 2000917
TreeView+ depends on / blocked
 
Reported: 2021-08-30 13:53 UTC by Sumedh Sidhaye
Modified: 2021-11-10 00:02 UTC (History)
6 users (show)

Fixed In Version: idm-client-8050020210913151510.de73ecb2
Doc Type: Bug Fix
Doc Text:
Cause: The acmeIPAServerCert profile cannot be added against a RHEL 7.x server because those versions lack the sanToCNDefaultImpl capability. Consequence: An 8.5 replica cannot be added against a 7.x server. Fix: Catch the failed profile add during ipa-replica-install. Running ipa-server-upgrade post-install will add the missing profile so that ACME can work on the 8.5 server. Result: ACME certificates can be issued.
Clone Of:
: 2000917 (view as bug list)
Environment:
Last Closed: 2021-11-09 18:29:52 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker FREEIPA-6908 0 None None None 2021-09-23 14:14:08 UTC
Red Hat Issue Tracker RHELPLAN-95614 0 None None None 2021-08-30 19:49:03 UTC
Red Hat Product Errata RHBA-2021:4230 0 None None None 2021-11-09 18:30:16 UTC

Description Sumedh Sidhaye 2021-08-30 13:53:33 UTC
Created attachment 1819099 [details]
replica install log

Description of problem:
IPA replica setup is failing when server is RHEL7.9 and replica is RHEL 8.5

Version-Release number of selected component (if applicable):
RHEL 7.9 Server packages:
ipa-server-4.6.8-5.el7_9.7.x86_64
ipa-server-dns-4.6.8-5.el7_9.7.noarch
389-ds-base-1.3.10.2-12.el7_9.x86_64
pki-base-10.5.18-16.el7_9.noarch


RHEL 8.5 replica packages:
ipa-server-4.9.6-4.module+el8.5.0+11912+1b4496cf.x86_64
ipa-server-dns-4.9.6-4.module+el8.5.0+11912+1b4496cf.noarch
389-ds-base-1.4.3.23-10.module+el8.5.0+12398+47000435.x86_64
pki-base-10.11.0-2.module+el8.5.0+12220+9cc212a8.noarch


How reproducible:
Always

Steps to Reproduce:
1.Setup IPA server on a RHEL 7.9 system
2.Setup IPA replica on a RHEL 8.5 system
3.

Actual results:
RHEL 8.5 replica setup against RHEL 7.9 server fails 

Expected results:
Replica setup should be successful

Additional info:
https://pagure.io/freeipa/issue/8738

Comment 1 Florence Blanc-Renaud 2021-08-30 14:12:38 UTC
This seems to be a regression related to the fix for https://pagure.io/freeipa/issue/8738 / https://bugzilla.redhat.com/show_bug.cgi?id=1934991

When a replica is installed, the installer now tries to add missing profiles to the 1st server. With 8.4, it tries to add the acmeIPAServerCert profile which contains the following definition:
[...]
policyset.serverCertSet.9.default.class_id=sanToCNDefaultImpl
[...]

On a 7.9 server, this sanToCNDefaultImpl does not exist, and the profile import fails. We can see in the master ca debug log:
[30/Aug/2021:07:37:35][http-bio-8443-exec-15]: BasicProfile: createProfilePolicy:  Cannot find sanToCNDefaultImpl

and in the replica:
2021-08-30T11:37:35Z DEBUG response body (decoded): b'{"Attributes":{"Attribute":[]},"ClassName":"com.netscape.certsrv.base.BadRequestException","Code":400,"Message":"Invalid profile data"}'
2021-08-30T11:37:35Z DEBUG Error migrating 'acmeIPAServerCert': Request failed with status 400: Non-2xx response from CA REST API: 400. Invalid profile data

When the profile import fails, the replica assumes that the profile already exists and tries to disable it, then update it. Disable fails because the profile does not exist on the master:
(master ca debug log):
[30/Aug/2021:07:37:35][http-bio-8443-exec-16]: Trying to modify profile: acmeIPAServerCert.  Profile not found.

and on the replica:
2021-08-30T11:37:35Z DEBUG request POST https://gizmo.idmqe.lab.eng.bos.redhat.com:8443/ca/rest/profiles/acmeIPAServerCert?action=disable
2021-08-30T11:37:35Z DEBUG request body ''
2021-08-30T11:37:35Z DEBUG response status 500

Update also fails:
(master ca debug log):
[30/Aug/2021:07:37:35][http-bio-8443-exec-16]: Trying to modify profile: acmeIPAServerCert.  Profile not found.

(replica):
2021-08-30T11:37:36Z DEBUG The ipa-replica-install command failed, exception: HTTPRequestError: Request failed with status 500: Non-2xx response from CA REST API: 500.
2021-08-30T11:37:36Z ERROR Request failed with status 500: Non-2xx response from CA REST API: 500.
2021-08-30T11:37:36Z ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information

Comment 7 Rob Crittenden 2021-08-30 19:48:43 UTC
Flo's analysis is correct. sanToCNDefaultImpl is only available in newer dogtag and is code, not configuration. The installer communicates with the existing CA and LDAP so is limited to the features that they provide.

When it fails it returns a 500 error which isn't the best thing to catch and ignore:

Request failed with status 500: Non-2xx response from CA REST API: 500.

The most straightforward fix is to catch this and continue. It would be nicer if dogtag returned a 400 Bad Request instead but it is what it is.

ACME will not work after this but if the LDAP profile is deleted and ipa-server-upgrade is run this will fix it on the 8.5 server:

# ipa-acme-manage enable
# ldapdelete -x -D 'cn=directory manager' -W cn=acmeIPAServerCert,cn=certprofiles,cn=ca,dc=example,dc=test
# ipa-server-upgrade

The removal of the profile will allow the upgrade to happen against a RHEL 8 server which has the sanToCNDefaultImpl so the profile will be added correctly.

Comment 9 Rob Crittenden 2021-08-30 20:24:18 UTC
The workaround of pre-creating the ACME profile prior to installing the first EL8 instance is successful.

Instructions when starting from scratch. If there is already an EL7 deployment then the install step can be skipped.

On the EL7 server:

# ipa-server-install <options>

Add the temporary ACME profile before installing IdM on EL8

# cat acme.ldif 
dn: cn=acmeIPAServerCert,cn=certprofiles,cn=ca,dc=example,dc=test
objectClass: ipacertprofile
objectClass: top
cn: acmeIPAServerCert
description: Temporary ACME IPA service certificate profile
ipaCertProfileStoreIssued: FALSE

# ldapadd -x -D 'cn=directory manager' -W -f acme.ldif 
Enter LDAP Password: 
adding new entry "cn=acmeIPAServerCert,cn=certprofiles,cn=ca,dc=example,dc=test"

On the EL8 server:

# ipa-client-install <options>
# ipa-replica-install --setup-ca

Delete the temporary profile:

# ldapdelete -x -D 'cn=directory manager' -w password cn=acmeIPAServerCert,cn=certprofiles,cn=ca,dc=example,dc=test

Re-run upgrade which will add the missing profile:

# ipa-server-upgrade 

Remove the EL7 server from the ipa-ca A record (because it can't answer ACME requests):

# kinit admin
# ipa dnsrecord-del example.test. ipa-ca
No option to delete specific record provided.
Delete all? Yes/No (default No): 
Current DNS record contents:

A record: 192.168.122.141, 192.168.122.123

Delete A record '192.168.122.141'? Yes/No (default No): y
Delete A record '192.168.122.123'? Yes/No (default No): 
  Record name: ipa-ca
  A record: 192.168.122.123

On EL8 client install certbot:
# dnf -y install certbot --enablerepo=epel

Get an ACME certificate
# certbot --server https://ipa-ca.example.test/acme/directory register -m nobody --agree-tos --no-eff-email

# certbot --server https://ipa-ca.example.test/acme/directory certonly --domain `hostname` --standalone

Comment 10 Rob Crittenden 2021-08-30 20:26:33 UTC
Upstream ticket:
https://pagure.io/freeipa/issue/8974

Comment 11 Rob Crittenden 2021-08-30 20:50:50 UTC
Upstream PR https://github.com/freeipa/freeipa/pull/5995

Comment 12 Rob Crittenden 2021-09-01 17:13:07 UTC
This supersedes comment #9.

This change is: when adding a profile fails, the LDAP operation will be rolled back as well and a warning displayed. This will allow the profile to be added at a later time.

Running ipa-server-upgrade once the installation is finished will:

1. Re-issue the Apache web certificate so it has the appropriate SAN for ipa-ca
2. Add the acmeIPAServerCert profile to the local pki so that ACME certificates can be issued

To verify this:

1. install 7.9 server
2. install 8.5 replica (with this change)
3. run ipa-server-upgrade on the replica once the replica install is complete
4. ipa-acme-manage enable on replica
5. remove the 7.9 server from the ipa-ca A record
6. install a client and install certbot from epel
7. register certbot on the client: certbot --server https://ipa-ca.example.test/acme/directory register -m nobody --agree-tos --no-eff-email
8. request a cert on the client: certbot --server https://ipa-ca.example.test/acme/directory certonly --domain `hostname` --standalone

Comment 14 Rob Crittenden 2021-09-02 16:58:12 UTC
Fixed upstream
master:
https://pagure.io/freeipa/c/4414d50d2a5452477eea40bec58266b19f43065e

Comment 15 Florence Blanc-Renaud 2021-09-02 19:10:31 UTC
Fixed upstream
ipa-4-9:
https://pagure.io/freeipa/c/a6e708ab4006d6623c37de1692de5362fcdb5dd6

Comment 19 Michal Polovka 2021-09-24 12:47:39 UTC
Verified using RHEL8.5 and RHEL7.9 machine with latest nightly compose 

7.9: ipa-server-4.6.8-5.el7_9.7.x86_64
8.5: ipa-server-4.9.6-6.module+el8.5.0+12660+88e16a2c.x86_64

7.9: ipa-server-install --domain ...--realm ...
(success)

8.5:  ipa-client-install ... --server ...

Client configuration complete.
The ipa-client-install command was successful

8.5 ipa-replica-install ...
...
The ipa-replica-install command was successful

Therefore marking as verified.

Comment 21 errata-xmlrpc 2021-11-09 18:29:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (ipa bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:4230


Note You need to log in before you can comment on or make changes to this bug.