Hide Forgot
Description of problem: A customer has a large number of DNS zones configured in IPA. The bind ldap backend re-synchonises / updates all objectclass types idnsConfigObject, idnsZone, idnsForwardZone, idnsRecord, when the logrotate script calls 'rndc reload' as part of 'systemctl reload named'. It takes over 45 sec to MOD 1099 records. This causes DNS outages in the customer environment. The customer has presently worked around this issue by disabling log rotation for named. This situation is not ideal and they are looking for ways to reduce the time taken to reload the named service. Version-Release number of selected component (if applicable): bind-dyndb-ldap-8.0-1.el7.x86_64 on RHEL 7.2. How reproducible: Always Steps to Reproduce: 1. Add a large number of DNS zones and records to IPA 2. systemctl reload named 3. Actual results: the named service takes a large amount of time to become available again. Expected results: named should reload within 1 - 2 seconds. Additional info: Please download sosreports for this issue from: http://jnansi.bne.redhat.com/01705291/
It seems suspicious that the LDAP MOD operations take so long time. The very same scenario on a virtual machine with default IPA 4.4 installation and no LDAP server tunning took 9 seconds. I believe that this can be optimized even more with proper DS tunning. I was testing this with bind-dyndb-ldap-10.0-5.el7.x86_64 389-ds-base-1.3.5.10-11.el7.x86_64 (packages from RHEL 7.3). Having said that, I would start with LDAP server debugging/tunning. An alternative is to use log rotation built into BIND and avoid logrotate and thus avoid periodic reload. Please see: ftp://ftp.isc.org/isc/bind/cur/9.9/doc/arm/Bv9ARM.ch06.html#logging_grammar (channel - file - versions) Optimizing this in bind-dyndb-ldap would be a major change. Also, it is unclear what we should exactly change. "man rndc" defines reload command like this: > reload > Reload configuration file and zones. so some kind of work is inevitable. Let me know if I can help you somehow.
I have recommended the customer to change named over to logging with syslog. This should alleviate the need to reload named when rotating logs. Steps are as follows: ~~~ 1. An example configuration, based on the database-log channel configuration in your named.conf is as follows: channel database-log { syslog local1; severity info; print-category yes; print-severity yes; print-time no; }; You should create similar channels for all categories (including default_debug) that you are already logging. You probably want to log all channels to the same syslog level (local1 in this case) - named will log the category of the message, so you will be able to identify the category of the message. The syslog daemon will timestamp the messages, so you don't need named to timestamp the message. 2. You should add the following configuration line to rsyslog.conf: ~~~ local1.* /var/log/named-syslog.log ~~~ 3. You should add /var/log/named-syslog.log to the list of log files in the logrotate configuration file for syslog. 4. Delete the logrotate configuration file for named, so named is not restarted. ~~~
The customer is happy with the workaround of named logging to syslog and has closed the case. It probably isn't worth wasting any more engineering cycles on this bug, so should just close this bug?
Closing per comment 7 and per triage from Oct 25.