Bug 840381

Summary: Plugin crashes if initial connection times out
Product: Red Hat Enterprise Linux 6 Reporter: Petr Spacek <pspacek>
Component: bind-dyndb-ldapAssignee: Adam Tkac <atkac>
Status: CLOSED ERRATA QA Contact: Namita Soman <nsoman>
Severity: unspecified Docs Contact:
Priority: high    
Version: 6.4CC: fjayalat, gbrinkle, jgalipea, msauton, ovasik, pspacek
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: The bug in bind-dyndb-ldap caused that the plugin could crashed named process when connection to LDAP timed out. Consequence: When connection to LDAP timed out (or failed), named process was sometimes aborted and DNS service was unavailable. Fix: The plugin was fixed Result: Plugin now handles situations when connection to LDAP fails gracefully
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-02-21 08:58:19 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
/var/log/messages - as resquested none

Description Petr Spacek 2012-07-16 08:15:21 UTC
This bug is created as a clone of upstream ticket:
https://fedorahosted.org/bind-dyndb-ldap/ticket/84

A copy from IPA trac https://fedorahosted.org/freeipa/ticket/2924 (originaly reported by shanks):

Version: freeipa-server-2.99.0-0.20120710T1130Zgit0d11b8b.fc17.x86_64
bind-9.9.1-2.P1.fc17.x86_64
bind-dyndb-ldap-1.1.0-0.20120618T1354Zgita7cd8ae.fc17.x86_64

{{{
[root@dhcp201-193 ~]# ipactl start
Starting Directory Service
Starting KDC Service
Starting KPASSWD Service
Starting DNS Service
Job failed. See system journal and 'systemctl status' for details.
Failed to start DNS Service
Shutting down
Aborting ipactl
[root@dhcp201-193 ~]#
}}}

/var/log/messages: 
{{{
Jul 12 08:31:51 dhcp201-193 named[8040]: set up managed keys zone for view _default, file 'managed-keys.bind'
Jul 12 08:32:01 dhcp201-193 named[8040]: bind to LDAP server failed: Timed out
Jul 12 08:32:01 dhcp201-193 kernel: [258419.211587] named[8041] general protection ip:7f0a3e674e7b sp:7f0a40cdaa20 error:0 in libldap-2.4.so.2.8.3[7f0a3e65b000+4c000]
Jul 12 08:32:01 dhcp201-193 abrt[8045]: /var/named/core.8040 fd(-1) is not a regular file with link count 1: Permission denied
Jul 12 08:32:02 dhcp201-193 abrt[8045]: Saved core dump of pid 8040 (/usr/sbin/named) to /var/spool/abrt/ccpp-2012-07-12-08:32:01-8040 (42422272 bytes)
Jul 12 08:32:02 dhcp201-193 abrtd: Directory 'ccpp-2012-07-12-08:32:01-8040' creation detected
Jul 12 08:32:02 dhcp201-193 systemd[1]: named.service: control process exited, code=exited status=1
Jul 12 08:32:02 dhcp201-193 systemd[1]: Unit named.service entered failed state.
}}}

Further information is part of upstream ticket.

Steps to reproduce: https://fedorahosted.org/bind-dyndb-ldap/ticket/84#comment:1

Comment 2 Grant Brinkley 2012-09-09 23:25:59 UTC
Per the attached SFDC ticket - applying bind-dyndb-ldap-1.1.0-0.9.b1.el6_3.2.x86_64.rpm does not fix the issue on the client system.

LDAP still fails to recover automatically on reboot.

Comment 3 Petr Spacek 2012-09-11 11:08:41 UTC
I need more information. Lines from /var/log/messages related to this problem would help. Thanks.

Comment 4 Grant Brinkley 2012-09-12 00:48:45 UTC
Created attachment 611961 [details]
/var/log/messages - as resquested

Comment 5 Petr Spacek 2012-09-12 08:35:22 UTC
Unfortunatelly attachment 611961 [details] is quite old (3 months!). I need logs from machine with bind-dyndb-ldap-1.1.0-0.9.b1.el6_3.2.x86_64.rpm installed.

I analyzed last two BIND starts from logs you provided:
Jun 13 16:30:20 vuwunicoipam001 named[28281]: bind to LDAP server failed: Timed out
Jun 13 16:30:20 vuwunicoipam001 named[28281]: loading configuration: failure
Jun 13 16:30:20 vuwunicoipam001 named[28281]: exiting (due to fatal error)

BIND failed to start, because Directory Server not responded in time limit. This problem should be solved by bind-dyndb-ldap-1.1.0-0.9.b1.el6_3.2.x86_64.rpm. The plugin will reconnect periodically.

Last attempt to start BIND was sucessfull:
Jun 13 16:30:42 vuwunicoipam001 named[28715]: running
Jun 13 16:30:42 vuwunicoipam001 named[28715]: zone ods.vuw.ac.nz/IN: sending notifies (serial 2016)
Jun 13 16:30:42 vuwunicoipam001 named[28715]: zone 0.0.195.130.in-addr.arpa/IN: sending notifies (serial 2012270301)
Jun 13 16:30:42 vuwunicoipam001 named[28715]: zone 0.7.70.10.in-addr.arpa/IN: sending notifies (serial 2012280301)
Jun 13 16:30:42 vuwunicoipam001 named[28715]: zone 0.33.80.10.in-addr.arpa/IN: sending notifies (serial 2012180401)
Jun 13 16:30:42 vuwunicoipam001 named[28715]: zone 3.70.10.in-addr.arpa/IN: sending notifies (serial 2012130601)
Jun 13 16:30:42 vuwunicoipam001 named[28715]: zone 0.3.70.10.in-addr.arpa/IN: sending notifies (serial 2012120601)

(Zones ods.vuw.ac.nz are stored in LDAP, I suppose. Correct me if I'm wrong.)

Next log record from BIND was logged 16 hours later (Jun 14 09:09:18).
Jun 14 09:09:18 vuwunicoipam001 named[28715]: LDAP query timed out. Try to adjust "timeout" parameter

This indicates a DS problem, not a BIND one. This error message doesn't repeat in logs, so it was intermittent problem which disappeared.

Comment 8 Jenny Severance 2012-09-25 16:13:07 UTC
regression test automated in ipa-ctl test suite

Comment 14 Namita Soman 2012-11-27 04:06:57 UTC
verified using ipa-server-3.0.0-8.el6.x86_64

Results of automated test:
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: [   LOG    ] :: ipa-ctl bz840381 At times ipactl fails to start DNS service and a crash is detected.
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

Stopping pki-ca: [  OK  ]
Stopping httpd: [  OK  ]
Stopping ipa_memcached: [  OK  ]
Stopping named: .[  OK  ]
Stopping Kerberos 5 Admin Server: [  OK  ]
Stopping Kerberos 5 KDC: [  OK  ]
Shutting down dirsrv: 
    PKI-IPA...[  OK  ]
    TESTRELM-COM...[  OK  ]
Stopping CA Service
Stopping HTTP Service
Stopping MEMCACHE Service
Stopping DNS Service
Stopping KPASSWD Service
Stopping KDC Service
Stopping Directory Service
:: [   PASS   ] :: Stop all ipa services
:: [   PASS   ] :: Start ipa services, direct output to /dev/shm/bz840381.txt
:: [   PASS   ] :: Ensure that a DNS failure is not in the output file BZ 840381
:: [   PASS   ] :: Make sure that bind has not crashed. BZ 840381
'f8c05b6c-b7bd-42b8-9162-9271b3447f90'
ipa-ctl-bz840381-At-times-ipactl-fails-to-start-DNS-service-and-a-crash-is-detected- result: PASS

Comment 15 Namita Soman 2012-11-28 12:37:09 UTC
verified as above

Comment 18 errata-xmlrpc 2013-02-21 08:58:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0359.html