RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 688934 - Wait for Directory Service ports to open
Summary: Wait for Directory Service ports to open
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: ipa
Version: 6.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Rob Crittenden
QA Contact: Chandrasekar Kannan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-03-18 14:56 UTC by Rob Crittenden
Modified: 2015-01-04 23:47 UTC (History)
7 users (show)

Fixed In Version: ipa-2.0.0-17.el6
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2011-05-19 13:44:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2011:0631 0 normal SHIPPED_LIVE new package: ipa 2011-05-18 17:55:55 UTC

Description Rob Crittenden 2011-03-18 14:56:12 UTC
Description of problem:

Sometimes there is a timing issue in IPA installation. When Directory Server instance is restarted and then there are LDAP operations started, the LDAP listening ports may not be open yet. This leads to errors like the following one:

$ sudo ipa-server-install
...
  [30/32]: enabling compatibility plugin
  [31/32]: tuning directory server
root        : CRITICAL Failed to load ds-nfiles.ldif: Command '/usr/bin/ldapmodify -h vm-077.idm.lab.bos.redhat.com -v -f /tmp/tmpyhvNAE -x -D cn=Directory Manager -y /tmp/tmpgRvDSm' returned non-zero exit status 255
  [32/32]: configuring directory to start on boot
done configuring dirsrv.
Unexpected error - see ipaserver-install.log for details:
 {'desc': "Can't contact LDAP server"}

This could be solved by implementing a wait for open listening ports to function DsInstance::__restart_instance (like it is in CaInstance::__restart_instance). A timeout would be convenient. It may be set to some safe value, for example 5 mins.

A common function implementing the open-port wait used in both CaInstance and DsInstance would be convenient.

Comment 1 Dmitri Pal 2011-03-22 01:34:25 UTC
https://fedorahosted.org/freeipa/ticket/1076

Comment 2 Rob Crittenden 2011-03-22 19:00:18 UTC
master: 18542cd165d047cba69ed2b3ac12e59993bf2fb0

Comment 4 Jenny Severance 2011-04-08 17:24:14 UTC
Please add steps to reproduce this issue. Thanks!

Comment 5 Rob Crittenden 2011-04-11 14:27:24 UTC
This was an intermittent problem seen while trying to install IPA.

It was more common on busy and/or low-powered machines. So you can try limiting memory to cause swapping in a VM.

Comment 7 Martin Kosek 2011-04-15 09:05:07 UTC
This issue is difficult to reproduce, but I suggest using this approach:

1. Edit /etc/init.d/dirsrv and temporarily block port 389 used by ldapmodify to access the Directory Server:

...
restart() {
    stop
    (iptables -I INPUT -p tcp --dport 389 -j REJECT; sleep 10; iptables -F) &
    start
}
...

2. Start the standard IPA installation

Unpatched version should fail with a lot of ldapmodify error messages like the one above. Pathed version should wait until the port 389 is open (10 seconds) and then continue with the installation.

Comment 8 Namita Soman 2011-04-18 17:09:37 UTC
Having trouble reproducing the issue on an older version of ipa install.

Using: ipa-server-2.0.0-16.el6.x86_64
First edited /etc/init.d/dirsrv as suggested above.

Installed successfully, no errors in install log.

Next tried few attempts changing sleep to 60, 100, 300.
Also tried with --selfsign

Still was able to successfully install ipa server.

Was in touch with mkosek to get steps to see the problem.
He suggested editing it - so that now my section looks like below:
restart() {
    stop
    iptables -I INPUT -p tcp --dport 389 -j REJECT
    start
}
And then when the install hangs, to run iptables -F

Saw the install fail, with no hangs and the errors when using ipa-server-2.0.0-16.el6.x86_64. With no hangs - didn't run iptables -F

Assuming - this reproduced the issue...moved to another machine using ipa-server-2.0.0-21.el6.x86_64, where this is fixed.

Tried this thrice with the new install, but install never hung. So didn't see when would be the right time to run iptables -F to unblock the port.

Comment 9 Martin Kosek 2011-04-19 11:41:06 UTC
There are issues with the approach I posted above. Additionally, the patched IPA installer does not wait for open ports after *every* dirsrv restart, but only in few cases (including the reported case). This shall be fixed in the IPA upstream.

You can use the following updated /etc/init.d/dirsrv to simulate the reported issue:
restart() {
    stop
    # BZ 688934 reproducer START
    IPA_PRE_TUNING_PHASE=`grep limits.conf /var/lib/ipa/sysrestore/sysrestore.index`
    IPA_POST_TUNING_PHASE=`grep ldappwd /var/lib/ipa/sysrestore/sysrestore.index`

    # IPA installator is now in step [31/32]: tuning directory server
    if [ -n "$IPA_PRE_TUNING_PHASE" -a -z "$IPA_POST_TUNING_PHASE" ]; then
        # block DS port
        iptables -I INPUT -p tcp --dport 389 -j REJECT
    fi
    # BZ 688934 reproducer END
    start
}

This will block the DS port only for the reported case, i.e. step "[31/32]: tuning directory server" in the IPA installation procedure.

Reproducing steps:
1) Update /etc/init.d/dirsrv as written above
2) Start IPA installation (--selfsign option may be used for faster test)
3) When the installer reaches step "[31/32]: tuning directory server", the IPA should:
  a) For unpatched version: print the reported error message, installation fails
  b) Patched version should get into a waiting loop until the port is unblocked. Use "iptables -F" to unblock the port before 300 seconds timeout. Installation should succeed then.

Comment 10 Namita Soman 2011-04-19 12:49:19 UTC
Followed the steps above, and saw the install fail in old server.
Then did the same - saw the install hang with new server, and after unblocking ports, install continued. Also, tried install...leaving tghe port blocked, and install failed as expected.

But this fix addresses the port not being available at step [31/32]. When the port is not available at other times of install, it still doesn't wait for the port to open, and install fails.

Comment 12 Namita Soman 2011-04-19 14:42:49 UTC
Opened bug 697878 to address the other cases when the installs fails, and doesn't wait for port to open. Marking this verified, since the step this was reported for, has been fixed.

Comment 14 errata-xmlrpc 2011-05-19 13:44:44 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHEA-2011-0631.html


Note You need to log in before you can comment on or make changes to this bug.