Bug 1081626 - When certmonger is still tracking cert in ipa, uninstall fails but error does not indicate this
Summary: When certmonger is still tracking cert in ipa, uninstall fails but error does...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipa
Version: 7.0
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: ---
Assignee: Martin Kosek
QA Contact: Namita Soman
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-27 17:50 UTC by Namita Soman
Modified: 2015-03-05 10:10 UTC (History)
7 users (show)

Fixed In Version: ipa-4.0.3-1.el7
Doc Type: Known Issue
Doc Text:
An IPA master is uninstalled while SSL certificates for services other than IPA servers are tracked by the certmonger service. Consequently, an unexpected error can occur, and the uninstallation process fails. To work around this problem, start certmonger, and run the ipa-getcert command to list the tracked certificates. Then run the "ipa-getcert stop-tracking -i <Request ID>" command to stop certmonger from tracking the certificates, and run the IPA uninstall script again.
Clone Of:
Environment:
Last Closed: 2015-03-05 10:10:46 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:0442 normal SHIPPED_LIVE Moderate: ipa security, bug fix, and enhancement update 2015-03-05 14:50:39 UTC

Description Namita Soman 2014-03-27 17:50:23 UTC
Description of problem:
When running test for bz817080, uninstall failed with 
IOError: [Errno 2] No such file or directory: '/var/lib/certmonger/requests//20140327164857.tmp'

In beta, had gotten error:
ipa         : ERROR    Some certificates may still be tracked by certmonger.
This will cause re-installation to fail.
Start the certmonger service and list the certificates being tracked
 # getcert list
These may be untracked by executing
 # getcert stop-tracking -i <request_id>
for each id in: 20131203201604

# ls -l /var/lib/certmonger/requests/20140327164857*
-rw-------. 1 root root 482 Mar 27 12:50 /var/lib/certmonger/requests/20140327164857

Can uninstall successfully after running  getcert stop-tracking on the certs


Version-Release number of selected component (if applicable):
ipa-server-3.3.3-25.el7.x86_64, certmonger-0.70-2.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. Follow steps as listed in bz817080


Actual results:
Shutting down all IPA services
Removing IPA client configuration
Unconfiguring ntpd
Unconfiguring CA
Unconfiguring named
Unconfiguring web server
Unexpected error - see /var/log/ipaserver-uninstall.log for details:
IOError: [Errno 2] No such file or directory: '/var/lib/certmonger/requests//20140327164857.tmp'



Expected results:

Shutting down all IPA services
Removing IPA client configuration
Unconfiguring ntpd
Unconfiguring CA
Unconfiguring named
Unconfiguring web server
Unconfiguring krb5kdc
Unconfiguring kadmin
Unconfiguring directory server
Unconfiguring ipa_memcached
Unconfiguring ipa-otpd
ipa         : ERROR    Some certificates may still be tracked by certmonger.
This will cause re-installation to fail.
Start the certmonger service and list the certificates being tracked
 # getcert list
These may be untracked by executing
 # getcert stop-tracking -i <request_id>
for each id in: 20131203201604

Additional info:

from /var/log/ipaserver-uninstall.log:
2014-03-27T16:50:54Z DEBUG stderr=
2014-03-27T16:51:02Z DEBUG   File "/usr/lib/python2.7/site-packages/ipaserver/install/installutils.py", line 638, in run_script
    return_value = main_function()

  File "/usr/sbin/ipa-server-install", line 674, in main
    return uninstall()

  File "/usr/sbin/ipa-server-install", line 496, in uninstall
    httpinstance.HTTPInstance(fstore).uninstall()

  File "/usr/lib/python2.7/site-packages/ipaserver/install/httpinstance.py", line 423, in uninstall
    self.stop_tracking_certificates()

  File "/usr/lib/python2.7/site-packages/ipaserver/install/httpinstance.py", line 453, in stop_tracking_certificates
    db.untrack_server_cert(self.cert_nickname)
    
  File "/usr/lib/python2.7/site-packages/ipaserver/install/certs.py", line 548, in untrack_server_cert
    certmonger.stop_tracking(self.secdir, nickname=nickname)
    
  File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line 256, in stop_tracking
    request_id = get_request_id(criteria)

  File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line 104, in get_request_id
    rv = find_request_value('%s/%s' % (REQUEST_DIR, file), key)

  File "/usr/lib/python2.7/site-packages/ipapython/certmonger.py", line 51, in find_request_value
    fp = open(filename, 'r')

2014-03-27T16:51:02Z DEBUG The ipa-server-install command failed, exception: IOError: [Errno 2] No such file or directory: '/var/lib/certmonger/requests//20140327164857.tmp'

Comment 1 Rob Crittenden 2014-03-27 17:56:30 UTC
Just to confirm, you added two additional certificates to be tracked prior to uninstallation, one in a temporary database and one in /etc/httpd/alias, correct?

Comment 3 Namita Soman 2014-03-27 18:20:44 UTC
Steps taken:
# ipa-server-install --setup-dns --forwarder=$DNSFORWARD --hostname=$HOSTNAME -r $RELM -p $ADMINPW -P $ADMINPW -a $ADMINPW -U

# kinit admin
         
# ipa host-add bz817080.$DOMAIN --force"
# mkdir -p /tmp/certdb
# echo Secret123 > /tmp/certdb/passwd1
# certutil -f /tmp/certdb/passwd1 -N -d /tmp/certdb
# certutil -f /tmp/certdb/passwd1 -R -s 'cn=bz817080.testrelm.com,o=testrelm.com' -d /tmp/certdb -z /etc/group -a > /tmp/certdb/bz817080.csr
# ipa cert-request --add --principal bz817080/bz817080.testrelm.com /tmp/certdb/bz817080.csr > /tmp/certdb/bz817080.crt

Serial number was 11

# ipa cert-show --out=/tmp/certdb/bz817080.crt 11

And to answer the needinfo ques....yes - as below:

:: [   PASS   ] :: Running 'certutil -f /tmp/certdb/passwd1 -A -n bz817080 -d /tmp/certdb -t u,u,u -a < /tmp/certdb/bz817080.crt' (Expected 0, got 0)
New tracking request "20140327164857" added.
:: [   PASS   ] :: Running 'ipa-getcert start-tracking -d /tmp/certdb -n bz817080' (Expected 0, got 0)
Notice: Trust flag u is set automatically if the private key is present.
:: [   PASS   ] :: Running 'certutil -f /tmp/certdb/passwd1 -A -n bz817080 -d /etc/httpd/alias -t u,u,u -a < /tmp/certdb/bz817080.crt' (Expected 0, got 0)
New tracking request "20140327164858" added.
:: [   PASS   ] :: Running 'ipa-getcert start-tracking -d /etc/httpd/alias -n bz817080' (Expected 0, got 0)

Comment 4 Martin Kosek 2014-03-28 07:28:09 UTC
*** Bug 1080844 has been marked as a duplicate of this bug. ***

Comment 5 Martin Kosek 2014-03-28 08:06:03 UTC
Rob, is this purely a certmonger issue then (Nalin CCed) or do you see a possible improvement in IPA as well?

Comment 6 Rob Crittenden 2014-03-28 12:16:46 UTC
I don't know what the tmp file contains. I don't think this is a "bug" in certmonger though, as we are really poking our noses into its private data.

IMHO assuming there are no plans to change the certmonger request filename format, it may even be worthwhile to add a regex of filenames to look for and ignore the rest.

At a minimum IPA should have a try/except around the open to fail gracefully. In this case it probably shouldn't give up completely, just move to the next file in the list, and log somewhere.

Comment 7 Martin Kosek 2014-03-28 12:28:45 UTC
Ok, makes sense. I will clone an upstream ticket.

Comment 8 Martin Kosek 2014-03-28 12:29:21 UTC
Upstream ticket:
https://fedorahosted.org/freeipa/ticket/4280

Comment 9 Martin Kosek 2014-09-05 08:58:08 UTC
Fixed upstream
master:
https://fedorahosted.org/freeipa/changeset/6d94cdf250c470bf77a0e769ea30a90fa5815b81

ipa-4-1:
https://fedorahosted.org/freeipa/changeset/78b2a7abbb33f9b880b2920812c443e86a7d3c06

ipa-4-0:
https://fedorahosted.org/freeipa/changeset/ff6e43cc14b846531aac37a0250eb079db9aac6e


IPA now uses proper certmonger dbus API and does not touch internal config files. So bugs like this one should no longer appear.

Comment 11 Kaleem 2014-12-15 08:07:31 UTC
Verified.

IPA version:
============
ipa-server-4.1.0-12.el7.x86_64

Snip from automation log:
=========================
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: ipaserverinstall_bz817080 - ipa-server-install --uninstall doesn't clear certmonger dirs, which leads to install failing bz817080
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

:: [   PASS   ] :: Command 'ipa-server-install --setup-dns --forwarder=10.10.160.2 --hostname=idm-qe-05.testrelm.test -r TESTRELM.TEST -p xxxxxxxx -P xxxxxxxx -a xxxxxxxx -U' (Expected 0, got 0)
:: [   PASS   ] :: Command 'ipa host-add bz817080.testrelm.test --force' (Expected 0, got 0)
:: [   PASS   ] :: Command 'mkdir -p /tmp/certdb' (Expected 0, got 0)
:: [   PASS   ] :: Changing context to cert_t so that certs can be generated (Expected 0, got 0)
:: [   PASS   ] :: Command 'echo xxxxxxxx > /tmp/certdb/passwd1' (Expected 0, got 0)
:: [   PASS   ] :: Command 'certutil -f /tmp/certdb/passwd1 -N -d /tmp/certdb' (Expected 0, got 0)
:: [   PASS   ] :: Command 'certutil -f /tmp/certdb/passwd1 -R -s 'cn=bz817080.testrelm.test,o=testrelm.test' -d /tmp/certdb -z /etc/group -a > /tmp/certdb/bz817080.csr' (Expected 0, got 0)
:: [   PASS   ] :: Command 'ipa cert-request --add --principal bz817080/bz817080.testrelm.test /tmp/certdb/bz817080.csr > /tmp/certdb/bz817080.crt' (Expected 0, got 0)
:: [   PASS   ] :: Command 'ipa cert-show --out=/tmp/certdb/bz817080.crt 11' (Expected 0, got 0)
:: [   PASS   ] :: Command 'certutil -f /tmp/certdb/passwd1 -A -n bz817080 -d /tmp/certdb -t u,u,u -a < /tmp/certdb/bz817080.crt' (Expected 0, got 0)
:: [   PASS   ] :: Command 'ipa-getcert start-tracking -d /tmp/certdb -n bz817080' (Expected 0, got 0)
:: [   PASS   ] :: Command 'certutil -f /tmp/certdb/passwd1 -A -n bz817080 -d /etc/httpd/alias -t u,u,u -a < /tmp/certdb/bz817080.crt' (Expected 0, got 0)
:: [   PASS   ] :: Command 'ipa-getcert start-tracking -d /etc/httpd/alias -n bz817080' (Expected 0, got 0)
:: [   PASS   ] :: Command 'ipa-server-install --uninstall -U > /tmp/tmp.Tlb92cB1Fu/ipaserverinstall_bz817080.out 2>&1' (Expected 0, got 0)
:: [   PASS   ] :: Command 'cat /tmp/tmp.Tlb92cB1Fu/ipaserverinstall_bz817080.out' (Expected 0, got 0)
:: [   PASS   ] :: File '/tmp/tmp.Tlb92cB1Fu/ipaserverinstall_bz817080.out' should contain 'ipa.*ERROR.*Some certificates may still be tracked by certmonger.' 
:: [   PASS   ] :: File '/tmp/tmp.Tlb92cB1Fu/ipaserverinstall_bz817080.out' should contain 'This will cause re-installation to fail.' 
:: [   PASS   ] :: File '/tmp/tmp.Tlb92cB1Fu/ipaserverinstall_bz817080.out' should contain 'Start the certmonger service and list the certificates being tracked' 
:: [   PASS   ] :: ipa-server-install --uninstall clears certmonger dirs 
:: [   PASS   ] :: Re-install to verify BZ 817080 not found (Expected 0, got 0)
:: [   PASS   ] :: Command 'getcert stop-tracking -d /tmp/certdb -n bz817080' (Expected 0, got 0)
:: [   PASS   ] :: Command 'rm -rf /tmp/certdb' (Expected 0, got 0)
:: [   PASS   ] :: Command 'getcert stop-tracking -d /etc/httpd/alias -n bz817080' (Expected 0, got 0)
:: [   LOG    ] :: Uninstall for next test
:: [   PASS   ] :: Uninstalling ipa server for next test (Expected 0, got 0)
:: [   PASS   ] :: Making sure that /etc/sssd/sssd.conf does not exist. BZ 819982 (Expected 2, got 2)
:: [   LOG    ] :: Duration: 8m 38s
:: [   LOG    ] :: Assertions: 25 good, 0 bad
:: [   PASS   ] :: RESULT: ipaserverinstall_bz817080 - ipa-server-install --uninstall doesn't clear certmonger dirs, which leads to install failing bz817080

Comment 13 errata-xmlrpc 2015-03-05 10:10:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-0442.html


Note You need to log in before you can comment on or make changes to this bug.