RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1232819 - testing ipa-restore on fresh system install fails
Summary: testing ipa-restore on fresh system install fails
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.1
Hardware: Unspecified
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: IPA Maintainers
QA Contact: Namita Soman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-17 14:46 UTC by jas
Modified: 2015-11-19 12:04 UTC (History)
7 users (show)

Fixed In Version: ipa-4.2.0-5.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-19 12:04:03 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
console output with verification steps (8.93 KB, text/plain)
2015-10-01 11:45 UTC, Kaleem
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:2362 0 normal SHIPPED_LIVE ipa bug fix and enhancement update 2015-11-19 10:40:46 UTC

Description jas 2015-06-17 14:46:25 UTC
Description of problem:

I did an ipa-server-install, added a few users, then did a full ipa-backup.  I then did an ipa-server-install --uninstall followed by an ipa-restore.  It worked.  I re-kickstarted the system ensuring that the ipa-server package was installed.  I then ran the ipa-restore again.  This time, the restore failed because /var/log/dirsrv did not exist. In fact, just to point out that directly after kickstart, the dirsrv user and dirsrv group didn't exist yet.  They seem to get added when running ipa-restore.   I created /var/log/dirsrv, and made it owned by dirsrv:dirsrv but then there was an issue because /var/log/dirsrv/slapd-EECS-YORKU-CA (domain dir) didn't exist.  I created it, but then there was an additional issue because the server had a permission denied error when writing to /var/log/dirsrv/slapd-EECS-YORKU-CA (even though they were all owned by dirsrv:dirsrv).  This is most certainly an SELinux issue. I rekickstarted the system with SELinux permissive.  Again after running ipa-restore, there was the error about /var/log/dirsrv not existing.  Again, I created it, and the internal slapd and everything worked.  I would certainly expect that ipa-restore is able to complete the restore process without assistance running SELinux in permissive or enforcing mode. 

(It's surprising because I did notice some SELinux fixes made not that long ago.  I would have thought that with the fixes, everything would work.)

How reproducible:

ipa-server-install
ipa-backup
re-kickstart
ipa-restore

Comment 2 jas 2015-06-17 18:16:34 UTC
Let me add that after the ipa-restore completed "successfully", I can login from any account on any IPA client with one exception.  I cannot login from any account TO the IPA server, and I could do that before.  The IPA server denies the request.  In /var/log/secure it says "invalid password".  The password is not incorrect, and resetting the password in the GUI doesn't solve the problem either.

Comment 3 jas 2015-06-17 19:33:44 UTC
Sigh.  In addition to the previous problems, the ipa-backup does not include /etc/tmpfiles.d/dirsrv-EECS-YORKU-CA.conf... so after a kickstart, and an ipa-restore, and a reboot, /var/run/dirsrv doesn't get created, and hence dirsrv fails to start.

Comment 4 jas 2015-06-17 20:32:11 UTC
In addition, it seems that the IPA accounts don't work locally after an ipa-restore because ipa-server-install makes modifications to /etc/pam.d to insert "pam_sss.so" into the chain.  However, ipa-backup doesn't backup those changes, and ipa-restore doesn't restore them, or re-create them.

It's not even as if you can run an "ipa-server-install" followed by an uninstall and it will leave those entries around... because uninstall gets rid of them.

[root@ipa tmp]# diff -r pam.d.orig pam.d.new
diff -r pam.d.orig/fingerprint-auth pam.d.new/fingerprint-auth
10a11
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
19a21
> session     optional      pam_sss.so
diff -r pam.d.orig/fingerprint-auth-ac pam.d.new/fingerprint-auth-ac
10a11
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
19a21
> session     optional      pam_sss.so
diff -r pam.d.orig/password-auth pam.d.new/password-auth
6a7
> auth        sufficient    pam_sss.so use_first_pass
11a13
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
15a18
> password    sufficient    pam_sss.so use_authtok
22a26
> session     optional      pam_sss.so
diff -r pam.d.orig/password-auth-ac pam.d.new/password-auth-ac
6a7
> auth        sufficient    pam_sss.so use_first_pass
11a13
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
15a18
> password    sufficient    pam_sss.so use_authtok
22a26
> session     optional      pam_sss.so
diff -r pam.d.orig/smartcard-auth pam.d.new/smartcard-auth
10a11
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
19a21
> session     optional      pam_sss.so
diff -r pam.d.orig/smartcard-auth-ac pam.d.new/smartcard-auth-ac
10a11
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
19a21
> session     optional      pam_sss.so
diff -r pam.d.orig/system-auth pam.d.new/system-auth
6a7
> auth        sufficient    pam_sss.so use_first_pass
11a13
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
15a18
> password    sufficient    pam_sss.so use_authtok
22a26
> session     optional      pam_sss.so
diff -r pam.d.orig/system-auth-ac pam.d.new/system-auth-ac
6a7
> auth        sufficient    pam_sss.so use_first_pass
11a13
> account     [default=bad success=ok user_unknown=ignore] pam_sss.so
15a18
> password    sufficient    pam_sss.so use_authtok
22a26
> session     optional      pam_sss.so

Comment 5 Petr Vobornik 2015-06-18 08:16:57 UTC
Thank you for the bug report. This is an issue and I'll clone the bug upstream to be fixed. 

For restore it's essential to use the same version of rpms. What is the ipa-server version? Is it ipa-4.1.0-18.el7?

As a workaround you can try:

ipa-server-install
ipa-backup
re-kickstart
ipa-server-install (with the same options)
ipa-restore

Restore will override the data set by the second install but it won't miss the things which are otherwise missing.

Comment 6 Petr Vobornik 2015-06-18 08:27:26 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/5071

Comment 7 jas 2015-06-18 10:42:04 UTC
On this set of machines I'm running CentOS 7.1 and the version is 4.1.0-18.el7.centos.3. The backup and restore are done with the same RPM versions.

The reinstall then uninstall then restore would potentially work except for PAM rules to allow logins into the IPA server because those changes are reverted back when you do an uninstall but not applied during a restore. For that I guess I could apply a patch file to patch all the files in /etc/pam.d manually...

Comment 8 Rob Crittenden 2015-06-18 12:56:36 UTC
(In reply to jas from comment #7)

I think you misunderstood his suggestion. If you do a fresh install on the box using the same options are the initial install, the do a full restore on top of that, then pam should remain configured.

Comment 9 jas 2015-06-18 14:16:42 UTC
Thanks, Rob.  You are absolutely correct.  I misread Petr's note.  I vaguely at least thought I remember reading somewhere that before an ipa-restore, you had to do an uninstall, so I never considered doing the ipa-restore right after the install.  That being said, I rekickstarted the system using SELinux enforcing, and followed the steps.  It worked great.  In the future, when this bug is fixed, I won't need to do the install anymore.

(PS: The ipa server install process seems to be pretty good about showing what it's doing, and where it is in the process.  I like that!  However, I noticed that the /etc/pam.d files get updated right after "Restarting the web server" and before "Setup complete".  It looks like there are some steps forgotten in the printed output, which is of course totally unrelated to the issue above.)

Comment 10 Martin Kosek 2015-06-19 06:44:15 UTC
This happens when IPA Client is being installed, right after the Web Server restart:

~~~~
    # Restart httpd to pick up the new IPA configuration
    service.print_msg("Restarting the web server")
    http.restart()

    if setup_kra:
        kra.install(api, None, options)

    # Set the admin user kerberos password
    ds.change_admin_password(admin_password)

    # Call client install script
    try:
        args = [paths.IPA_CLIENT_INSTALL, "--on-master", "--unattended",
                "--domain", domain_name, "--server", host_name,
                "--realm", realm_name, "--hostname", host_name]
~~~~

I agree it would be better to print a message about it (nudge for Petr/Rob).

Comment 11 Adriaan De Lange 2015-07-27 11:27:05 UTC
(In reply to jas from comment #3)
> Sigh.  In addition to the previous problems, the ipa-backup does not include
> /etc/tmpfiles.d/dirsrv-EECS-YORKU-CA.conf... so after a kickstart, and an
> ipa-restore, and a reboot, /var/run/dirsrv doesn't get created, and hence
> dirsrv fails to start.

I have exactly the same problem
However hen I do a restore (ex. ipa-restore /var/lib/ipa/backup/ipa-full-2015-07-27-11-29-13/) I noticed that the dirsvr services did start when I then run a systemctl start ipa.service ipa doesstart. (CA and all)

Thus when I reboot the IPA/IDM server I need run the restore command check that the dirsrv is started (which always works) and then start ipa.

Comment 14 Kaleem 2015-10-01 11:45:19 UTC
Created attachment 1079056 [details]
console output with verification steps

Verified.

Please find the attached verification logs

Comment 15 David Jones 2015-10-09 16:45:22 UTC
The IPA restore documentation does say to do an uninstall first. It's highlighted as "IMPORTANT". So either the documentation should be changed or the ipa-restore script fixed. 

I found that I had to first install IPA on the server and then uninstall, or the restore would fail. But just installing and then restoring makes more sense.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/restore.html

Comment 16 David Jones 2015-10-09 17:15:27 UTC
The the proposed solution doesn't work for me. The restore fails. 

Directory Manager (existing master) password: 

Preparing restore from /var/lib/ipa/backup/ipa-full-2015-10-09-09-58-14 on ipa.example.com
Performing FULL restore from FULL backup
Restoring data will overwrite existing live data. Continue to restore? [no]: yes
Each master will individually need to be re-initialized or
re-created from this one. The replication agreements on
masters running IPA 3.1 or earlier will need to be manually
re-enabled. See the man page for details.
Disabling all replication.
Unable to get connection, skipping disabling agreements: Unable to bind to LDAP server: [Errno 111] Connection refused
Stopping IPA services
Stopping IPA failed: Job for dirsrv failed. See 'systemctl status dirsrv' and 'journalctl -xn' for details.
Failed to read data from Directory Service: Command ''/bin/systemctl' 'start' 'dirsrv'' returned non-zero exit status 1
Shutting down

Restoring files
Systemwide CA database updated.
Restoring from userRoot in EXAMPLE-COM
ldif2db failed: 
Restoring from ipaca in EXAMPLE-COM
ldif2db failed: 
Starting IPA services
Command ''ipactl' 'start'' returned non-zero exit status 1

Comment 17 errata-xmlrpc 2015-11-19 12:04:03 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2362.html


Note You need to log in before you can comment on or make changes to this bug.