Bug 1679115

Summary: sudo 1.8.23 is significantly slower than 1.8.19p2
Product: Red Hat Enterprise Linux 8 Reporter: Robert Scheck <redhat-bugzilla>
Component: sudoAssignee: Radovan Sroka <rsroka>
Status: CLOSED WONTFIX QA Contact: BaseOS QE Security Team <qe-baseos-security>
Severity: low Docs Contact:
Priority: medium    
Version: 8.0CC: dapospis, dkopecek, rmetrich, robert.scheck
Target Milestone: rcKeywords: Performance, Triaged
Target Release: 8.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-03-14 18:30:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Robert Scheck 2019-02-20 11:25:54 UTC 
This bug was initially created as a copy of bug #1673906 because the RHEL 7 one was closed without an answer whether the decreased sudo performance is a bug or not, but due to upcoming Maintenance Support 1 Phase of RHEL 7.


Description of problem:

Comparing the time required to execute sudo between RHEL7.5 (1.8.19p2-14) and RHEL7.6 (1.8.23-3), it appears that sudo and "sudo -i" are significantly slower on RHEL7.6:

RHEL7.5:

$ ./sudoloop 
'sudo pwd' for 1000 iterations:

real	0m8.242s
user	0m3.197s
sys	0m4.651s

'sudo -i pwd' for 1000 iterations:

real	0m22.686s
user	0m11.812s
sys	0m10.437s


RHEL7.6:

$ ./sudoloop 
'sudo pwd' for 1000 iterations:

real	0m15.117s
user	0m5.029s
sys	0m7.871s

'sudo -i pwd' for 1000 iterations:

real	0m27.075s
user	0m13.154s
sys	0m12.031s


Version-Release number of selected component (if applicable):

sudo-1.8.23-3.el7


How reproducible:

Always


Steps to Reproduce:

1. Create a user account

  # useradd -G wheel user
  # echo "%wheel	ALL=(ALL)	NOPASSWD: ALL" > /etc/sudoers.d/wheel

2. As user 'user', execute the reproducer (sudoloop, in attachment)

  $ ./sudoloop

3. Downgrade sudo

  # yum -y downgrade sudo

4. As user 'user', Execute the reproducer again

  $ ./sudoloop


Actual results:

See Description above.


Additional info:

This is problematic for sudo intensive applications, such as the start of 3rd party application 'Zimbra' which spawns sudo > 200 times when executing "/opt/zimbra/libexec/zmconfigd".
Comment 1 Renaud Métrich 2019-02-20 14:44:56 UTC 
sudo-1.8.25p1-4.el8.x86_64 (RHEL 8 beta) is even worse:

$ ./sudoloop 
'sudo pwd' for 1000 iterations:

real	0m17.785s
user	0m7.002s
sys	0m6.678s

'sudo -i pwd' for 1000 iterations:

real	0m38.551s
user	0m18.306s
sys	0m15.340s
Comment 2 Renaud Métrich 2019-02-21 15:23:16 UTC 
Stracing shows that the time is spent mostly in closing invalid descriptors before executing /usr/sbin/unix_chkpwd.
The more the hard limit on number of files is, the slower it will be.

strace excerpt with 4096 hard limit on number of files (70 ms spent):

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
1414  16:16:37.363666 prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=1024, rlim_max=4*1024}) = 0 <0.000004>
1414  16:16:37.363686 close(4095)       = -1 EBADF (Bad file descriptor) <0.000004>
...
1414  16:16:37.433905 close(10)         = -1 EBADF (Bad file descriptor) <0.000003>
1414  16:16:37.433921 close(9<pipe:[33864]>) = 0 <0.000004>
1414  16:16:37.433943 close(8<pipe:[33864]>) = 0 <0.000003>
1414  16:16:37.433964 close(7</var/lib/sss/mc/group>) = 0 <0.000004>
1414  16:16:37.433985 close(6</var/lib/sss/mc/initgroups>) = 0 <0.000003>
1414  16:16:37.434004 close(5</etc/sudoers>) = 0 <0.000003>
1414  16:16:37.434021 close(4<UNIX:[33798->33812]>) = 0 <0.000003>
1414  16:16:37.434039 close(3</var/lib/sss/mc/passwd>) = 0 <0.000003>
1414  16:16:37.434058 geteuid()         = 0 <0.000003>
1414  16:16:37.434073 setuid(0)         = 0 <0.000005>
1414  16:16:37.434088 execve("/usr/sbin/unix_chkpwd", ["/usr/sbin/unix_chkpwd", "root", "chkexpiry"], []) = 0 <0.000339>
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------

With 65536 hard limit on number of files (1.1 second spent):

-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
1457  16:20:52.847951 prlimit64(0, RLIMIT_NOFILE, NULL, {rlim_cur=1024, rlim_max=64*1024}) = 0 <0.000004>
1457  16:20:52.847971 close(65535)      = -1 EBADF (Bad file descriptor) <0.000004>
...
1457  16:20:53.911885 close(10)         = -1 EBADF (Bad file descriptor) <0.000003>
1457  16:20:53.911901 close(9<pipe:[37346]>) = 0 <0.000004>
1457  16:20:53.911923 close(8<pipe:[37346]>) = 0 <0.000003>
1457  16:20:53.911942 close(7</var/lib/sss/mc/group>) = 0 <0.000004>
1457  16:20:53.911963 close(6</var/lib/sss/mc/initgroups>) = 0 <0.000003>
1457  16:20:53.911982 close(5</etc/sudoers>) = 0 <0.000003>
1457  16:20:53.911999 close(4<UNIX:[37282->37296]>) = 0 <0.000003>
1457  16:20:53.912018 close(3</var/lib/sss/mc/passwd>) = 0 <0.000003>
1457  16:20:53.912036 geteuid()         = 0 <0.000003>
1457  16:20:53.912050 setuid(0)         = 0 <0.000005>
1457  16:20:53.912065 execve("/usr/sbin/unix_chkpwd", ["/usr/sbin/unix_chkpwd", "root", "chkexpiry"], []) = 0 <0.000208>
-------- 8< ---------------- 8< ---------------- 8< ---------------- 8< --------
Comment 3 Robert Scheck 2019-02-22 17:38:45 UTC 
Cross-filed case 02323061 at the Red Hat customer portal.