Red Hat Bugzilla – Bug 1398600
IPA replica install fails with dirsrv errors.
Last modified: 2017-08-01 05:42:02 EDT
Description of problem: Fresh install of IPA server, when trying to make a replia of it, instllation fails at stage retrieving DS certificates from master server Version-Release number of selected component (if applicable): Rhel 7.3, IPA 4.4 How reproducible: Steps to Reproduce: On server01 IdM Server installed ok (master) cmdline: ==== ipa-server-install --mkhomedir --no-ntp --no_hbac_allow --no-ssh --no-sshd --hostname=server01.example.com --domain=example.com --realm=EXAMPLE.COM --ds-password="XXXXX" --admin-password="YYYYY" -U Created and defined user -hostregister- to enroll hosts ipa domainlevel-get command gets 1 2. On client02 IdM Client installed ok, cmdline: ==== ipa-client-install --mkhomedir --no-krb5-offline-passwords --no-nisdomain --no-dns-sshfp --no-ssh --no-sshd --no-ntp --domain=example --server=client01.example.com --realm=EXAMPLE.COM --principal=hostregister --password=hostregister -U --force-join 3. On client02 Promote installed IdM Client to Replica server, cmdline: ==== kinit admin ipa-replica-install --mkhomedir --no-ssh --no-sshd --no-ntp 4. Error on step [29/44] due to [28/44], ipa-replica-install logs: ==== [root@ulpldp02 ~]# kinit admin Password for admin@APP10.OSAKIDETZA.EUS: [root@ulpldp02 ~]# ipa-replica-install --mkhomedir --no-ssh --no-sshd --no-ntp Run connection check to master Connection check OK Configuring directory server (dirsrv). Estimated time: 1 minute [1/44]: creating directory server user [2/44]: creating directory server instance [3/44]: updating configuration in dse.ldif [4/44]: restarting directory server [5/44]: adding default schema [6/44]: enabling memberof plugin [7/44]: enabling winsync plugin [8/44]: configuring replication version plugin [9/44]: enabling IPA enrollment plugin [10/44]: enabling ldapi [11/44]: configuring uniqueness plugin [12/44]: configuring uuid plugin [13/44]: configuring modrdn plugin [14/44]: configuring DNS plugin [15/44]: enabling entryUSN plugin [16/44]: configuring lockout plugin [17/44]: configuring topology plugin [18/44]: creating indices [19/44]: enabling referential integrity plugin [20/44]: configuring certmap.conf [21/44]: configure autobind for root [22/44]: configure new location for managed entries [23/44]: configure dirsrv ccache [24/44]: enabling SASL mapping fallback [25/44]: restarting directory server [26/44]: creating DS keytab [27/44]: retrieving DS Certificate [28/44]: restarting directory server ipa : CRITICAL Failed to restart the directory server (Command '/bin/systemctl restart dirsrv@EXAMPLE.service' returned non-zero exit status 1). See the installation log for details. [29/44]: setting up initial replication [error] error: [Errno 111] Connection refused Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up. ipa.ipapython.install.cli.install_tool(Replica): ERROR [Errno 111] Connection refused ipa.ipapython.install.cli.install_tool(Replica): ERROR The ipa-replica-install command failed. See /var/log/ipareplica-install.log for more information Actual results: Replica install incomplete Expected results: Replica install complete Additional info:
Hello can we get directory server error log and journalctl -xe why restart failed?
Upstream ticket: https://fedorahosted.org/freeipa/ticket/6514
JFTR after IRC discussion: it looks that certmonger received CA_UNREACHABLE, ticket above was opened to improve error reporting. Waiting for more logs.
Better error reporting has been fixed upstream, this doesn't fix the root cause master: https://fedorahosted.org/freeipa/changeset/dbb98765d73519289ee22f3de1a5ccde140f6f5d
pki-tomcatd@pki-tomcat.service should be listening on port ::1:8009. The /etc/pki/pki-tomcat/server.xml configuration seems correct. Use netstat/netcat to confirm that the port is open when Dogtag is running. You can also restart Dogtag and look in /var/log/pki/pki-tomcat/catalina.<date>.log for a line like: 14-Dec-2016 12:26:30.618 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-0:0:0:0:0:0:0:1-8009"] I'm interested if Flo's comments in comment #27 are applicable (IPv6 disabled).
Does it make sense to check for presence of: /etc/hosts ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 Before replica install?
In reply to comment #36 : the issue happens on the replica when IPv6 is disabled on the *master*, not sure whether we can check this remotely.
True, sounds like another use case for http://www.freeipa.org/page/V4/Diagnostics_Tool
Upstream ticket: https://fedorahosted.org/freeipa/ticket/6575
I'm not sure if this is pure misconfiguration from admin side or a more issue in some component. Either way it seems quite common. And therefore some fix would be good. IPA shouldn't probably touch the file on update but PKI can at least report that it in more usable manner. Or ideally a component which "owns" this part of /etc/hosts should update it. I don't know which on it is. Moving to pki-core to consider reporting e.g. in self-test.
Petr, As discussed over IRC, it might be better if we could use a generic "localhost" address instead of "127.0.0.1" or "::1" so we don't have to change the /etc/hosts. We need someone to confirm that "localhost" would work in both IPv4 and IPv6 environments. Suppose that works, we can fix the default AJP hostname configuration in PKI with this ticket: https://fedorahosted.org/pki/ticket/2570. Once that is fixed, IPA probably should revert the change made in bug #1081561. However, these changes will fix new installations only. Existing installations will still have the old "127.0.0.1" or "::1". Is that OK, or do we need to replace them with "localhost" automatically? If so, should that be done by PKI or IPA?
Endi, Flo tested setting 'localhost' and it solves the issue for new installs. Flo will create IPA patch which changes for new installs (similar as in https://git.fedorahosted.org/cgit/freeipa.git/commit/?id=fa3b3193fabcaa37c2ba9865089fcfc06939c77f ) Whether to change the value also on upgrade is a question. If we would decide to change it on upgrade and given that the configuration value comes from IPA, it would make sense to request the change on IPA update. Endi, is there a "nice" way which IPA could you to amend PKI config?
Petr, Flo: There is no "nice" way for IPA to amend the PKI config. You'll have to search for the `address="::1"` in /etc/pki/pki-tomcat/server.xml and replace with `address="localhost"`. It could be done via a proper XML lib, but simple string search/replace will probably suffice.
Here's a sample script to replace an attribute in server.xml: https://git.fedorahosted.org/cgit/pki.git/tree/base/server/upgrade/10.2.2/02-EnableWebApplicationAutoDeploy
Petr, I think there are 2 options for IPA: 1. Override pki_ajp_host with "localhost" instead of "::1". 2. Wait until PKI ticket #2570 (bug #1413136) is fixed, then remove the pki_ajp_host override. With option #1 IPA will be responsible to fix existing server.xml during upgrade. With option #2 it will be PKI's responsibility. Alternatively, we can leave server.xml unchanged and let the admin fix it manually if needed. Which one would you prefer? I'm returning this bug to IPA since the fix requires changes in IPA either way.
I'd prefer fixing on PKI side and removing this very specific config from IPA. But it depends on timing. Is bug #1413136 something to be fixed in 7.3 batch update? If yes, then fix it on PKI side including updates, remove custom config from IPA. If not, then we might go with the IPA fix for 7.3, including update. And then for 7.4 we can remove the config from IPA installer and let it be completely a responsibility of PKI.
Bug #1413136 is already fixed for 7.4 and proposed for 7.3. The included upgrade script will automatically replace "127.0.0.1" and "::1" to "localhost" in existing servers, so all IPA needs to do is remove the pki_ajp_host override.
Created attachment 1244203 [details] IPA patch The submitted patch fixes the IPA-side configuration. The patch will be accepted upstream at a later point, when the new version of PKI is released.
Moving to POST According to comment 54
JFTR upstream fix master: https://fedorahosted.org/freeipa/changeset/eaa87c75b9f57500265b2dc9480b996b2b92e1e3
Fix pushed to ipa-4-4 branch ipa-4-4: https://fedorahosted.org/freeipa/changeset/4a30e9d53475d60fb76242a098f1d969d6b19f75
Created attachment 1279956 [details] output Verified on ipa-server-4.5.0-9.el7.x86_64
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:2304