Bug 235964

Summary: poor openldap performance on HP dl320 and dl380 G4 G5 and dl585
Product: [Fedora] Fedora Reporter: Ching-Che Yen <ccyen>
Component: openldapAssignee: Jan Safranek <jsafrane>
Status: CLOSED WONTFIX QA Contact:
Severity: high Docs Contact:
Priority: medium    
Version: 6CC: triage
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard: bzcl34nup
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-05-06 19:29:55 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
ldap bdb config file
none
ldap server config file none

Description Ching-Che Yen 2007-04-11 03:53:41 UTC
Description of problem:
I have posted these bugs:

Bugzilla Bug 234801: openldap got poor performance with RHEL5 on HP DL380 G4 and G5
Bugzilla Bug 235165: Poor performance of openldap with RHEL 3,4,5 on HP DL380 G4
and G5

I found that the the peformance of openldap with Fedora core6 is very bad on HP
 dl380 G4 G5 and HP DL585.The openldap server could just handle about 4~12
authentication queries per second.

But on others platform(PC with P4 2.8Ghz and PC with AMD 1.7Ghz),the openldap
server could handle about 120~160 authentication queries per second.

All the database and configuration(slapd.conf,DB_CONFIG) are the same.

I wonder that is there anyone who had the same problem with HP DL380,DL585 or
other hardware platform?



Version-Release number of selected component (if applicable):

All openldap rpm packages I tested is the newest version that Redhat released.
(openldap-2.3.27-4 on Fedora core 6 )

Comment 1 Jan Safranek 2007-04-30 15:28:11 UTC
How do you measure the authentication count? If you have some script for testing
the speed, please attach it so I can try to reproduce the exact behavior.

And please attach also your configuration files, especially /etc/openldap/* and
your database (if possible).

Comment 2 Ching-Che Yen 2007-05-01 02:33:15 UTC
Created attachment 153850 [details]
ldap bdb config file

Comment 3 Ching-Che Yen 2007-05-01 02:34:08 UTC
Created attachment 153851 [details]
ldap server config file

Comment 4 Ching-Che Yen 2007-05-01 02:47:29 UTC
Dear Sir:

     After these days,I tested more and I got more info.

I found that if I disable the ldap log setting(local4.*   /var/log/ldap.log) in
syslog.conf,and redirect the ldap log with command "slapd -d 512 >>
/var/log/ldap.log",the performance of ldap will be 140~160 queries/second !!!

I think maybe there are some I/O problem in syslog(while handling ldap log) on
these hardwares I tested.
But the performance of mail looks OK,and I have tried to add "-" in frot of
"/var/log/ldap.log" in the syslog.conf,the performance become to 40~50 queries 
per second.

Thank you very much and sorry for my poor english.

Comment 5 Ching-Che Yen 2007-05-01 10:19:13 UTC
Dear Sir:
    Because gotting replying from you,I tested the ldap further today.
I found that the performance of ldap on Fedora Core 6 become to 200 queries per
second !!! 
Because the Fedora Core6 server running yum-update every day.Now I am trying to
find out which updated-rpm to make this.

On the other hand,the ldap performance on RHEL5(still no rpm update now) is
still very bad (4~12 queries per second).

Thank you 

Comment 6 Ching-Che Yen 2007-05-02 10:13:25 UTC
I still could not find out which rpm fixed this problem,I post another Bugzilla
Bug with RHEL5 on

Bugzilla Bug 238686: Strange syslog performance with sldap log (had been fixed
in FC6,but RHEL5 is not)

I hope that this problem on RHEL5 could be fixed asap.

Thank you very much.

Comment 7 Jan Safranek 2007-05-02 11:04:01 UTC
Please do not open new bugs for the same issue - post all your findings here, so
we can find all the relevant informations on one place.

If I underestand it correctly, the slapd is working fast on FC-6 and RHEL5 if
the logging is disabled and if you enable the default logging to syslog, it is
slow on RHEL5 and not-updated FC6, but updated FC6 is working fast? 

I have looked at the updates in #238686 and I have found out, that selinux
policies were changed and syslogd got higher priority. Do you use SELinux? If
so, could you please turn it off (in /etc/selinux/config) on your updated FC-6
machine, reboot, try your tests and tell me results? Is it still fast or is it
slow again?

Comment 8 Jan Safranek 2007-05-02 11:04:51 UTC
*** Bug 238686 has been marked as a duplicate of this bug. ***

Comment 9 Jan Safranek 2007-05-02 11:14:26 UTC
One more idea: turn on SELinux (if you turned it off as suggested in #7), try to
downgrade selinux to selinux-*-2.4.6-49.fc6.noarch.rpm and run your tests. Is it
slow or fast?

Comment 10 Ching-Che Yen 2007-05-03 03:15:27 UTC
Dear Sir:

     I did ever doubted that the selinux system is the reason to cause of this
problem,but on the RHEL5,the status of SELinux is disabled(Using command
"getenforce"),and never been changed.(by checking the modified date of file
"/etc/selinux/config")

On the Fedora Core6 ,the status of SELinux is Permissive now and the file
"/etc/selinux/config" have been changed on 17 April.(from Enforcing to Disabled)

I have tried to replace the selinux rpm on Fedora Core6 with these rpms:
selinux-policy-2.4.6-49.fc6.noarch.rpm
selinux-policy-2.4.6-54.fc6.noarch.rpm 
selinux-policy-2.4.6-57.fc6.noarch.rpm 
selinux-policy-devel-2.4.6-49.fc6.noarch.rpm
selinux-policy-devel-2.4.6-54.fc6.noarch.rpm
selinux-policy-devel-2.4.6-57.fc6.noarch.rpm
selinux-policy-mls-2.4.6-49.fc6.noarch.rpm
selinux-policy-mls-2.4.6-54.fc6.noarch.rpm
selinux-policy-mls-2.4.6-57.fc6.noarch.rpm
selinux-policy-strict-2.4.6-49.fc6.noarch.rpm
selinux-policy-strict-2.4.6-54.fc6.noarch.rpm
selinux-policy-strict-2.4.6-57.fc6.noarch.rpm
selinux-policy-targeted-2.4.6-49.fc6.noarch.rpm 
selinux-policy-targeted-2.4.6-54.fc6.noarch.rpm 
selinux-policy-targeted-2.4.6-57.fc6.noarch.rpm

I test the performance of slapd for 6 times(for each different seliunx version
and different current mode of SELinux),all results are fast now.

I can not reboot this server because there are some other important services on
it,I will test this problem on another server if I got another HP DL380 server
G5 some days later.

Thank you very much. 

Comment 11 Jan Safranek 2007-05-03 08:57:57 UTC
Except selinux I don't see anything suspicious in the list of updated packages.
If you have lot of time you can try one by one, but I am quite pesimistic.

What process does take most of the CPU on your RHEL box? Is it slapd or syslog?
Will it help if you simply increase syslog priority?

Comment 12 Ching-Che Yen 2007-05-04 02:47:54 UTC
Dear Sir:
     Because it's hard to get another hp dl380 server,
I am preparing to update my online RHEL5 server rpm packages one by one,
then maybe I could find out the answer. 

The slapd and syslog take about 1% of the CPU on my RHEL5 server,but I could see
the info of command "top" show that the I/O wait is about 80 to 90 percent on
one core of one CPU.

I used command "renice" to change the priority of syslog from 0 to -20,but the
performance of slapd is still very slow.

Thank you very much.

Comment 13 Bug Zapper 2008-04-04 06:50:05 UTC
Fedora apologizes that these issues have not been resolved yet. We're
sorry it's taken so long for your bug to be properly triaged and acted
on. We appreciate the time you took to report this issue and want to
make sure no important bugs slip through the cracks.

If you're currently running a version of Fedora Core between 1 and 6,
please note that Fedora no longer maintains these releases. We strongly
encourage you to upgrade to a current Fedora release. In order to
refocus our efforts as a project we are flagging all of the open bugs
for releases which are no longer maintained and closing them.
http://fedoraproject.org/wiki/LifeCycle/EOL

If this bug is still open against Fedora Core 1 through 6, thirty days
from now, it will be closed 'WONTFIX'. If you can reporduce this bug in
the latest Fedora version, please change to the respective version. If
you are unable to do this, please add a comment to this bug requesting
the change.

Thanks for your help, and we apologize again that we haven't handled
these issues to this point.

The process we are following is outlined here:
http://fedoraproject.org/wiki/BugZappers/F9CleanUp

We will be following the process here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping to ensure this
doesn't happen again.

And if you'd like to join the bug triage team to help make things
better, check out http://fedoraproject.org/wiki/BugZappers

Comment 14 Bug Zapper 2008-05-06 19:29:54 UTC
This bug is open for a Fedora version that is no longer maintained and
will not be fixed by Fedora. Therefore we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen thus bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.