Bug 147581 - Slow rlogin to RHEL3 when hosts.equiv is too big
Slow rlogin to RHEL3 when hosts.equiv is too big
Status: CLOSED CANTFIX
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: rsh (Show other bugs)
3.0
All Linux
medium Severity medium
: ---
: ---
Assigned To: Karel Zak
Ben Levenson
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2005-02-09 10:37 EST by Sherif Abdelgawad
Modified: 2013-08-05 23:19 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2005-09-08 06:19:07 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sherif Abdelgawad 2005-02-09 10:37:15 EST
Description of problem:
RHEL 3 (stock & updated to U4) server with /etc/hosts.equiv contains 
about 20460 servers, and all of such lines are defined in /etc/hosts 
as well or most of them (exactly in the test was 1220 in /etc/hosts).

rlogin from any platform to that RHEL 3 server takes long time to 
login from 30 sec to min or more.

That does not happen when rlogin to Sun with exact same configuration 
and same size of /etc/hosts.equiv & /etc/hosts.


Version-Release number of selected component (if applicable):
rsh-server-0.17-17, glibc-2.3.2-95.30, kernel-smp-2.4.21-20.EL

How reproducible:
Always

Steps to Reproduce:
1. Configure RHEL to remote login without password
2. Add 20K servers in /etc/hosts.equiv & /etc/hosts
3. rlogin to the server
    

Actual Results:  very slow login process till it lookup the server.

Expected Results:  fast lookup process as in solaris

Additional info:
Comment 1 Karel Zak 2005-02-10 05:17:55 EST
Do you run "nscd" (name service cache daemon) with enabled check-files?

/etc/nscd.conf:
check-files             hosts           yes
Comment 2 Sherif Abdelgawad 2005-02-10 10:11:04 EST
There is no nscd running. Here is the tests done:

- Connecting from Sun and/or RHEL to RHEL using rsh/rlogin
- GLIBC updated to the latest version from RHN

Two tests:
----------

1- Connecting passing through the firewall routers which does not allow 113 
packets
2- Connecting on same subnet allowing 113 packets

Results 
-------

* Case1:  

- 4 times trial of getting ident .. From tcpdump the following line comes 4 
times taking about 20 seconds.

auth: S [tcp sum ok] 737814781:737814781(0) win 5840 <mss 1460,sackOK,timestamp 
961178536 0,nop,wscale 0> (DF) (ttl 64, id 16603, 

- It then time out the auth packets , from strace ( -e trace=open ), following 
shows the files opened:

--- SIGSTOP (Stopped (signal)) @ 0 (0) ---
--- SIGSTOP (Stopped (signal)) @ 0 (0) ---
open("/etc/ld.so.preload", O_RDONLY)    = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libcrypt.so.1", O_RDONLY)    = 3
open("/lib/libpam.so.0", O_RDONLY)      = 3
open("/lib/libdl.so.2", O_RDONLY)       = 3
open("/lib/libpam_misc.so.0", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
open("/etc/resolv.conf", O_RDONLY)      = 3
open("/etc/nsswitch.conf", O_RDONLY)    = 3
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libnss_files.so.2", O_RDONLY) = 3
open("/etc/host.conf", O_RDONLY)        = 3
open("/etc/hosts", O_RDONLY)            = 3
--- SIGHUP (Hangup) @ 0 (0) ---

- That takes few seconds and allows the login session.

* Case 2:

- Tcpdum shows ident goes through fine
- strace showes:

--- SIGSTOP (Stopped (signal)) @ 0 (0) ---
--- SIGSTOP (Stopped (signal)) @ 0 (0) ---
open("/etc/hosts.allow", O_RDONLY)      = 0
open("/etc/ld.so.preload", O_RDONLY)    = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libcrypt.so.1", O_RDONLY)    = 3
open("/lib/libutil.so.1", O_RDONLY)     = 3
open("/lib/libdl.so.2", O_RDONLY)       = 3
open("/lib/libpam.so.0", O_RDONLY)      = 3
open("/lib/libpam_misc.so.0", O_RDONLY) = 3
open("/lib/tls/libc.so.6", O_RDONLY)    = 3
open("/etc/nsswitch.conf", O_RDONLY)    = 3
open("/etc/ld.so.cache", O_RDONLY)      = 3
open("/lib/libnss_files.so.2", O_RDONLY) = 3
open("/etc/resolv.conf", O_RDONLY)      = 3
open("/etc/host.conf", O_RDONLY)        = 3
open("/etc/hosts", O_RDONLY)            = 3
open("/etc/hosts", O_RDONLY)            = 3
open("/etc/protocols", O_RDONLY)        = 3
open("/etc/pam.d/rlogin", O_RDONLY)     = 3
open("/lib/security/pam_nologin.so", O_RDONLY) = 4
open("/lib/security/pam_securetty.so", O_RDONLY) = 4
open("/lib/security/pam_env.so", O_RDONLY) = 4
open("/lib/security/pam_rhosts_auth.so", O_RDONLY) = 4
open("/lib/security/pam_stack.so", O_RDONLY) = 4
open("/etc/pam.d/other", O_RDONLY)      = 3
open("/lib/security/$ISA/pam_deny.so", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/lib/security/../../lib/security/pam_deny.so", O_RDONLY) = 4
open("/etc/nologin", O_RDONLY)          = -1 ENOENT (No such file or directory)
open("/etc/passwd", O_RDONLY)           = 3
open("/etc/hosts", O_RDONLY)            = 3
open("/etc/passwd", O_RDONLY)           = 4
open("/etc/hosts.equiv", O_RDONLY)      = 3
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/ld.so.cache", O_RDONLY)      = 4
open("/lib/libnss_dns.so.2", O_RDONLY)  = 4
open("/lib/libresolv.so.2", O_RDONLY)   = 4
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/hosts", O_RDONLY)            = 4
.
.
.
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/hosts", O_RDONLY)            = 4
open("/etc/localtime", O_RDONLY)        = 3
open("/etc/pam.d/system-auth", O_RDONLY) = 3
open("/lib/security/$ISA/pam_env.so", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/lib/security/../../lib/security/pam_env.so", O_RDONLY) = 4
open("/lib/security/$ISA/pam_warn.so", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/lib/security/../../lib/security/pam_warn.so", O_RDONLY) = 4
open("/lib/security/$ISA/pam_unix.so", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/lib/security/../../lib/security/pam_unix.so", O_RDONLY) = 4
open("/etc/ld.so.cache", O_RDONLY)      = 4
open("/lib/libnsl.so.1", O_RDONLY)      = 4
open("/lib/security/$ISA/pam_deny.so", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/lib/security/$ISA/pam_cracklib.so", O_RDONLY) = -1 ENOENT (No such file 
or directory)
open("/lib/security/../../lib/security/pam_cracklib.so", O_RDONLY) = 4
open("/etc/ld.so.cache", O_RDONLY)      = 4
open("/usr/lib/libcrack.so.2", O_RDONLY) = 4
open("/lib/security/$ISA/pam_limits.so", O_RDONLY) = -1 ENOENT (No such file or 
directory)
open("/lib/security/../../lib/security/pam_limits.so", O_RDONLY) = 4
open("/etc/pam.d/other", O_RDONLY)      = 3
open("/etc/passwd", O_RDONLY)           = 3
open("/etc/shadow", O_RDONLY)           = 3
open("/etc/passwd", O_RDONLY)           = 3
open("/etc/group", O_RDONLY)            = 3
open("/etc/security/pam_env.conf", O_RDONLY) = 3
open("/etc/environment", O_RDONLY)      = -1 ENOENT (No such file or directory)
open("/dev/ptmx", O_RDWR)               = 3
open("/dev/pts/8", O_RDWR|O_NOCTTY)     = 4


It loops through /etc/hosts for hmmm range of 1000s times.  

P.S. the /etc/hosts has about 20460 hosts defined, and /etc/hosts.equiv has 
about 1220 hosts defined.

The question is why is the different in behavior between the two senarios? Is 
there a way to disable the auth packets from taking place? Is this a bug in 
in.rlogind or some configuration need to be altered to match same performance 
of Solaris?

Comment 4 Karel Zak 2005-02-10 13:24:25 EST
Please, please... start nscd at your server. Please! :-)

The in.rlogind doesn't check something -- all is done by pam_rhosts.
Comment 5 Sherif Abdelgawad 2005-02-10 13:35:45 EST
I did :) but the question still valid. There are a delay still exists to allow
the login. NSCD reduced the delay, yet there are two delays now:

1- 113 (auth) packets delay from the server to the client which is being
dropped. It is firewalled by the network, yet it still tries 4 times before
giving up. Is there a way to disable this?

2- 3 to 6 seconds till it login after the auth packets. This with nscd running.
Any explanation ? 

This is not happening on solaris so it is hard to tell why the delay at all! If
the host.equiv has only one line it just work as charm. Any explanation ?:) 

Note You need to log in before you can comment on or make changes to this bug.