Created attachment 1123467 [details] mail text received for crash Description of problem: Following crash seen with pki-common pkg during IPA server install backtrace: :__init__.py:55:load:IndexError: list index out of range : :Traceback (most recent call last): : File "/usr/sbin/pki-server", line 89, in <module> : cli.execute(sys.argv) : File "/usr/sbin/pki-server", line 84, in execute : super(PKIServerCLI, self).execute(args) : File "/usr/lib/python2.6/site-packages/pki/cli.py", line 195, in execute : module.execute(module_args) : File "/usr/lib/python2.6/site-packages/pki/server/cli/upgrade.py", line 79, in execute : subsystem = pki.server.PKISubsystem(subsystem_dir) : File "/usr/lib/python2.6/site-packages/pki/server/__init__.py", line 44, in __init__ : self.load() : File "/usr/lib/python2.6/site-packages/pki/server/__init__.py", line 55, in load : value = parts[1] :IndexError: list index out of range Version-Release number of selected component (if applicable): pki-common.noarch 0:9.0.3-47.el6 How reproducible: Always Steps to Reproduce: 1. IPA install Actual results: Crash seen Expected results: No crash expected Additional info: Please find the attached files for more reference.
Created attachment 1123469 [details] crash dump
Was this a fresh IPA install, or an upgrade? Is there anything special with regards to how you are installing IPA?
(In reply to Nathan Kinder from comment #3) > Was this a fresh IPA install, or an upgrade? Is there anything special with > regards to how you are installing IPA? It was a fresh IPA install. We are doing normal IPA install by using following in automated execution /usr/sbin/ipa-server-install --setup-dns --forwarder=10.16.36.29 --hostname=apollo.testrelm.test -r TESTRELM.TEST -n testrelm.test -p xxxxxxxx -a xxxxxxxx --ip-address=10.19.34.16 -U Do let me know if any other info required.
Hi, I was not able to reproduce the problem using the same build. In the bug description the code was trying to parse the properties in CS.cfg and some how failed because one of the lines doesn't contain the property value. Could you post the CS.cfg? I couldn't find it in the attachment. Thanks.
Created attachment 1129165 [details] CS.cfg Please find the attached CS.cfg.
Thanks. I don't see anything wrong with the attached CS.cfg. Could you add some print statements to the loop in /usr/lib/python2.6/site-packages/pki/server/__init__.py:52 as follows? print 'Parsing %s:' % self.cs_conf for line in lines: print '* %s' % line parts = line.split('=', 1) name = parts[0] value = parts[1] self.config[name] = value Then run the installation again. The output should appear in /var/log/ipaserver-install.log. Please post the line from CS.cfg that's causing the parsing error. Thanks.
Created attachment 1132990 [details] ipaserver-install.log with extra prints
Endi, I ran an install with the extra prints you indicated. I couldn't see anything other than the extra data printed in the install log. I included the whole log though in case I missed something. As far as I've seen, ipa-server-install is completing as expected when this crash occurs. Let Kaleem or myself know if you need us to try something else. Thanks, Scott
The only thing I can think is happening here is that this is a timing issue. The system is calling the upgrade scripts and reading in the CS.cfg file before the filesystem has finished writing to it. In the case where you added more debug statements above, the crash did not occur - presumably because the timing had changed. Can you try the following test? 1. Remove the debug code Endi asked you to add above. 2. In the file /usr/share/pki/scripts/functions, change the line that says --> pki-server upgrade --subsystem "$PKI_INSTANCE_PATH" --silent to: sync;sync pki-server upgrade --subsystem "$PKI_INSTANCE_PATH" --silent 3. See if the issue recurs.
Tom ... How do we get this on the RHEL 6.8 blocker list? Who can provide "Blocker" flag "+"?
The root cause of this is actually quite straightforward. It turns out that the CS.cfg file which is being parsed contains some comment lines before configuration, and these lines cause the exception to be raised and the alert to be issued. This is a trivial case which can be resolved by making the parsing code ignore comment and blank lines.
Fix checked into rhel 6.8 branch .. Counting objects: 62, done. Compressing objects: 100% (31/31), done. Writing objects: 100% (32/32), 5.18 KiB, done. Total 32 (delta 18), reused 0 (delta 0) To ssh://git.app.eng.bos.redhat.com/srv/git/pki.git 3637b1f..dd5039f DOGTAG_9_0_RHEL_BRANCH -> DOGTAG_9_0_RHEL_BRANCH Matt will be creating builds.
I would change the existing doc text to something like this: The ipa-server-install utility sometimes terminates unexpectedly An attempt to install an Identity Management server can fail due to a problem with the *pki-common* package. The fail occurs because the `CS.cfg` certificate authority (CA) configuration file which is being parsed contains unexpected comment lines before configuration. This issue has been resolved by making the parsing code ignore comment and blank lines.
Verified. Version :: pki-ca-9.0.3-48.el6.noarch Results :: :: [ BEGIN ] :: Running ' /usr/sbin/ipa-server-install --setup-dns --forwarder=$FORWARDER --hostname=cisco-c240m3-01.testrelm.test -r TESTRELM.TEST -n testrelm.test -p Secret123 -a Secret123 --ip-address=$IPADDRESS -U' The log file for this installation can be found in /var/log/ipaserver-install.log ============================================================================== This program will set up the IPA Server. This includes: * Configure a stand-alone CA (dogtag) for certificate management * Configure the Network Time Daemon (ntpd) * Create and configure an instance of Directory Server * Create and configure a Kerberos Key Distribution Center (KDC) * Configure Apache (httpd) * Configure DNS (bind) To accept the default shown in brackets, press the Enter key. Warning: skipping DNS resolution of host cisco-c240m3-01.testrelm.test Using reverse zone 70.16.10.in-addr.arpa. The IPA Master Server will be configured with: Hostname: cisco-c240m3-01.testrelm.test IP address: $IPADDRESS Domain name: testrelm.test Realm name: TESTRELM.TEST BIND DNS server will be configured to serve IPA domain with: Forwarders: $FORWARDER Reverse zone: $REVNET.in-addr.arpa. Configuring NTP daemon (ntpd) [1/4]: stopping ntpd [2/4]: writing configuration [3/4]: configuring ntpd to start on boot [4/4]: starting ntpd Done configuring NTP daemon (ntpd). Configuring directory server for the CA (pkids): Estimated time 30 seconds [1/3]: creating directory server user [2/3]: creating directory server instance [3/3]: restarting directory server Done configuring directory server for the CA (pkids). Configuring certificate server (pki-cad): Estimated time 3 minutes 30 seconds [1/21]: creating certificate server user [2/21]: creating pki-ca instance [3/21]: configuring certificate server instance [4/21]: disabling nonces [5/21]: creating CA agent PKCS#12 file in /root [6/21]: creating RA agent certificate database [7/21]: importing CA chain to RA certificate database [8/21]: fixing RA database permissions [9/21]: setting up signing cert profile [10/21]: set up CRL publishing [11/21]: set certificate subject base [12/21]: enabling Subject Key Identifier [13/21]: setting audit signing renewal to 2 years [14/21]: configuring certificate server to start on boot [15/21]: restarting certificate server [16/21]: requesting RA certificate from CA [17/21]: issuing RA agent certificate [18/21]: adding RA agent as a trusted user [19/21]: configure certificate renewals [20/21]: configure Server-Cert certificate renewal [21/21]: Configure HTTP to proxy connections Done configuring certificate server (pki-cad). Configuring directory server (dirsrv): Estimated time 1 minute [1/38]: creating directory server user [2/38]: creating directory server instance [3/38]: adding default schema [4/38]: enabling memberof plugin [5/38]: enabling winsync plugin [6/38]: configuring replication version plugin [7/38]: enabling IPA enrollment plugin [8/38]: enabling ldapi [9/38]: disabling betxn plugins [10/38]: configuring uniqueness plugin [11/38]: configuring uuid plugin [12/38]: configuring modrdn plugin [13/38]: enabling entryUSN plugin [14/38]: configuring lockout plugin [15/38]: creating indices [16/38]: enabling referential integrity plugin [17/38]: configuring ssl for ds instance [18/38]: configuring certmap.conf [19/38]: configure autobind for root [20/38]: configure new location for managed entries [21/38]: restarting directory server [22/38]: adding default layout [23/38]: adding delegation layout [24/38]: adding replication acis [25/38]: creating container for managed entries [26/38]: configuring user private groups [27/38]: configuring netgroups from hostgroups [28/38]: creating default Sudo bind user [29/38]: creating default Auto Member layout [30/38]: adding range check plugin [31/38]: creating default HBAC rule allow_all [32/38]: Upload CA cert to the directory [33/38]: initializing group membership [34/38]: adding master entry [35/38]: configuring Posix uid/gid generation [36/38]: enabling compatibility plugin [37/38]: tuning directory server [38/38]: configuring directory to start on boot Done configuring directory MARK-LWD-LOOP -- 2016-03-13 13:40:05 -- server (dirsrv). Configuring Kerberos KDC (krb5kdc): Estimated time 30 seconds [1/10]: adding sasl mappings to the directory [2/10]: adding kerberos container to the directory [3/10]: configuring KDC [4/10]: initialize kerberos container [5/10]: adding default ACIs [6/10]: creating a keytab for the directory [7/10]: creating a keytab for the machine [8/10]: adding the password extension to the directory [9/10]: starting the KDC [10/10]: configuring KDC to start on boot Done configuring Kerberos KDC (krb5kdc). Configuring kadmin [1/2]: starting kadmin [2/2]: configuring kadmin to start on boot Done configuring kadmin. Configuring ipa_memcached [1/2]: starting ipa_memcached [2/2]: configuring ipa_memcached to start on boot Done configuring ipa_memcached. Configuring the web interface (httpd): Estimated time 1 minute [1/14]: setting mod_nss port to 443 [2/14]: setting mod_nss protocol list to TLSv1.0 - TLSv1.2 [3/14]: setting mod_nss password file [4/14]: enabling mod_nss renegotiate [5/14]: adding URL rewriting rules [6/14]: configuring httpd [7/14]: setting up ssl [8/14]: setting up browser autoconfig [9/14]: publish CA cert [10/14]: creating a keytab for httpd [11/14]: clean up any existing httpd ccache [12/14]: configuring SELinux for httpd [13/14]: restarting httpd [14/14]: configuring httpd to start on boot Done configuring the web interface (httpd). Applying LDAP updates Restarting the directory server Restarting the KDC Configuring DNS (named) [1/9]: adding DNS container [2/9]: setting up our zone [3/9]: setting up reverse zone [4/9]: setting up our own record [5/9]: setting up kerberos principal [6/9]: setting up named.conf [7/9]: restarting named [8/9]: configuring named to start on boot [9/9]: changing resolv.conf to point to ourselves Done configuring DNS (named). Global DNS configuration in LDAP server is empty You can use 'dnsconfig-mod' command to set global DNS options that would override settings in local named.conf files Restarting the web server ============================================================================== Setup complete Next steps: 1. You must make sure these network ports are open: TCP Ports: * 80, 443: HTTP/HTTPS * 389, 636: LDAP/LDAPS * 88, 464: kerberos * 53: bind UDP Ports: * 88, 464: kerberos * 53: bind * 123: ntp 2. You can now obtain a kerberos ticket using the command: 'kinit admin' This ticket will allow you to use the IPA tools (e.g., ipa user-add) and the web user interface. Be sure to back up the CA certificate stored in /root/cacert.p12 This file is required to create replicas. The password for this file is the Directory Manager password :: [ PASS ] :: Command ' /usr/sbin/ipa-server-install --setup-dns --forwarder=$FORWARDER --hostname=cisco-c240m3-01.testrelm.test -r TESTRELM.TEST -n testrelm.test -p Secret123 -a Secret123 --ip-address=$IPADDRESS -U' (Expected 0, got 0) No ABRT crash dump seen from this run.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-0909.html