RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1813200 - gssproxy memory leak with rpc.gssd
Summary: gssproxy memory leak with rpc.gssd
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: gssproxy
Version: 8.2
Hardware: Unspecified
OS: Linux
high
high
Target Milestone: rc
: 8.3
Assignee: Robbie Harwood
QA Contact: anuja
URL: https://github.com/gssapi/gssproxy/pu...
Whiteboard:
Depends On:
Blocks: 1894575
TreeView+ depends on / blocked
 
Reported: 2020-03-13 08:43 UTC by Yongcheng Yang
Modified: 2021-05-18 14:42 UTC (History)
9 users (show)

Fixed In Version: gssproxy-0.8.0-19.el8
Doc Type: No Doc Update
Doc Text:
Clone Of: 1809277
Environment:
Last Closed: 2021-05-18 14:42:07 UTC
Type: ---
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
valgrind output with gssproxy-0.8.3-3.fc33 (103.84 KB, text/plain)
2020-08-03 08:18 UTC, Yongcheng Yang 		no flags Details
Entire valgrid log with gssproxy disabled (42.08 KB, text/plain)
2020-08-07 11:01 UTC, Steve Dickson 		no flags Details
Entire valgrid log with gssproxy enabled using an Window AD (87.56 KB, text/plain)
2020-08-15 15:06 UTC, Steve Dickson 		no flags Details
Entire valgrid log with gssproxy enabled with gss_release_oid() added (246.65 KB, text/plain)
2020-08-24 18:47 UTC, Steve Dickson 		no flags Details
Entire valgrid log with gssproxy disabled with gss_release_oid() added (292.72 KB, text/plain)
2020-08-24 18:49 UTC, Steve Dickson 		no flags Details
View All

Comment 23 Robbie Harwood 2020-07-31 20:33:33 UTC 
gssproxy-0.8.3-3.fc33 should contain the fixes.  If you would like a RHEL build to advance test with, I'm happy to provide one.
Comment 24 Yongcheng Yang 2020-08-03 08:18:02 UTC 
Created attachment 1703242 [details]
valgrind output with gssproxy-0.8.3-3.fc33
Comment 25 Yongcheng Yang 2020-08-03 08:26:01 UTC 
I have checked the reproducer on fedora33 (Fedora-Rawhide-20200802.n.0 from beaker).

Please find the above attached output.
Comment 27 Steve Dickson 2020-08-04 21:39:32 UTC 
(In reply to Yongcheng Yang from comment #24)
> Created attachment 1703242 [details]
> valgrind output with gssproxy-0.8.3-3.fc33

What was the valgrind command line did you use?
Comment 28 Yongcheng Yang 2020-08-05 00:55:26 UTC 
(In reply to Steve Dickson from comment #27)
> (In reply to Yongcheng Yang from comment #24)
> > Created attachment 1703242 [details]
> > valgrind output with gssproxy-0.8.3-3.fc33
> 
> What was the valgrind command line did you use?

valgrind --log-file=valgrind.out.xxx --trace-children=yes --leak-check=full --track-origins=yes -v --show-reachable=yes rpc.gssd
Comment 30 Steve Dickson 2020-08-06 11:12:21 UTC 
(In reply to Yongcheng Yang from comment #28)
> (In reply to Steve Dickson from comment #27)
> > (In reply to Yongcheng Yang from comment #24)
> > > Created attachment 1703242 [details]
> > > valgrind output with gssproxy-0.8.3-3.fc33
> > 
> > What was the valgrind command line did you use?
> 
> valgrind --log-file=valgrind.out.xxx --trace-children=yes --leak-check=full
> --track-origins=yes -v --show-reachable=yes rpc.gssd

I ran the above valgrind command with rpc.gssd not using 
gssproxy (/etc/nfs.conf: use-gss-proxy=0) over night and
got the following results:

==115828== LEAK SUMMARY:
==115828==    definitely lost: 0 bytes in 0 blocks
==115828==    indirectly lost: 0 bytes in 0 blocks
==115828==      possibly lost: 0 bytes in 0 blocks
==115828==    still reachable: 12,942 bytes in 44 blocks
==115828==         suppressed: 88 bytes in 1 blocks
==115828== 
==115828== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

So the interaction between rpc.gssd and gssproxy is causing the memory leaks.

Yongcheng, would you mind verifying this case in your testing?
Comment 31 Yongcheng Yang 2020-08-06 13:27:15 UTC 
(In reply to Steve Dickson from comment #30)
...
> Yongcheng, would you mind verifying this case in your testing?

Yes, have just checked that NO memleak found with "use-gss-proxy=0" set under "[gssd]" in /etc/nfs.conf.
I.e. https://beaker.engineering.redhat.com/jobs/4467749

But why the nfs krb5 mounting can success even when rpc.gssd not using gssproxy. (I thought it's mis-configured before..)

Btw, seemingly the usage of "use-gss-proxy" is not mentioned in the man page rpc.gssd(8).
Comment 33 Steve Dickson 2020-08-07 11:01:32 UTC 
Created attachment 1710785 [details]
Entire valgrid log with gssproxy disabled
Comment 35 Steve Dickson 2020-08-07 11:10:49 UTC 
(In reply to Yongcheng Yang from comment #31)
> (In reply to Steve Dickson from comment #30)
> ...
> > Yongcheng, would you mind verifying this case in your testing?
> 
> Yes, have just checked that NO memleak found with "use-gss-proxy=0" set
> under "[gssd]" in /etc/nfs.conf.
> I.e. https://beaker.engineering.redhat.com/jobs/4467749
Thanks!

> 
> But why the nfs krb5 mounting can success even when rpc.gssd not using
> gssproxy. (I thought it's mis-configured before..)
It is only required on the server since it replaced rpc.svcgssd(8)
which was a bit buggy... It is optional on the client side.

> 
> Btw, seemingly the usage of "use-gss-proxy" is not mentioned in the man page
> rpc.gssd(8).
This is probably a doc bug...
Comment 41 Steve Dickson 2020-08-15 15:06:13 UTC 
Created attachment 1711485 [details]
Entire valgrid log with gssproxy enabled using an Window AD

Just for grins... I ran the set up in comment 39 using 
a Window's AD as the KDC... Doing a quick scan it appears
the errors are similar which is not too surprising. 

Just trying to look at this from all angels
Comment 46 Steve Dickson 2020-08-24 18:47:32 UTC 
Created attachment 1712420 [details]
Entire valgrid log with gssproxy enabled with gss_release_oid() added
Comment 47 Steve Dickson 2020-08-24 18:49:00 UTC 
Created attachment 1712422 [details]
Entire valgrid log with gssproxy disabled with gss_release_oid() added
Comment 48 Robbie Harwood 2020-08-26 18:38:59 UTC 
Thanks, that first one is the log I was looking for.  With that information, the correct fix is on the gssproxy side and you don't need to do anything for it: https://github.com/gssapi/gssproxy/pull/10
Comment 69 anuja 2020-12-02 09:09:49 UTC 
Verified with sanity run as per comment #68 and comment #67
Verfied bug using dev compose:
gssproxy-0.8.0-19.el8.x86_64
ipa-server-4.8.7-13.module+el8.3.0+8376+0bba7131.x86_64

Adding report using bash::ipa-client-automount:

2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho] ====================================== Final Report =======================================
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]             Test Date: Wed Dec  2 03:52:40 EST 2020         
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]                Total : [57]       
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]                Passed: [57]        
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]                Failed: [0]        
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]            Unfinished: [0]  
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]                Abort : [0]       
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]                Crash : [0]     
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]  +-----------------------------[RPMs & OS: [RedHat - x86_64]-----------------------------+
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho] |       ipa-client-4.8.7-13.module+el8.3.0+8376+0bba7131.x86_64
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho] |       ipa-client-common-4.8.7-13.module+el8.3.0+8376+0bba7131.noarch
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho] |       sssd-ipa-2.4.0-2.el8.x86_64
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho] ------------------------------------------------------------------------------------------
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]  +-----------------------------------------------------------------------------------------+
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]      Test:[/ipa-server/rhel80/ipa-client-automount/root]: [ Pass(57/57): 100% ] 
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]  +-----------------------------------------------------------------------------------------+
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]  +----------------------------------------------------------------------+
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]                     Fail / unfinished / ABORT [ Fail(0/57): 0% ]
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho]  +----------------------------------------------------------------------+
2020-12-02T08:52:41+0000 [ci-vm-10-0-155-38.ho] =========================== end of report [/tmp/rhts.report.23479.txt]===============================
 
Based on this marking Verified:Tested SanityOnly
Comment 73 anuja 2020-12-17 07:28:43 UTC 
Verified with sanity run as per comment #68 and comment #67

Adding report using bash::ipa-client-automount:

2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho] ====================================== Final Report =======================================
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]             Test Date: Thu Dec 17 07:17:45 UTC 2020         
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]                Total : [57]       
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]                Passed: [57]        
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]                Failed: [0]        
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]            Unfinished: [0]  
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]                Abort : [0]       
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]                Crash : [0]     
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]  +-----------------------------[RPMs & OS: [RedHat - x86_64]-----------------------------+
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho] |       ipa-client-4.9.0-0.5.rc3.module+el8.4.0+9124+ced20601.x86_64
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho] |       ipa-client-common-4.9.0-0.5.rc3.module+el8.4.0+9124+ced20601.noarch
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho] |       sssd-ipa-2.4.0-3.el8.x86_64
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho] ------------------------------------------------------------------------------------------
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]  +-----------------------------------------------------------------------------------------+
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]      Test:[/ipa-server/rhel80/ipa-client-automount/root]: [ Pass(57/57): 100% ] 
2020-12-17T07:17:46+0000 [ci-vm-10-0-107-23.ho]  +-----------------------------------------------------------------------------------------+

Based on this marking verified.
Comment 75 errata-xmlrpc 2021-05-18 14:42:07 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (gssproxy bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1592
Note You need to log in before you can comment on or make changes to this bug.