RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2021088 - Expired/Revoked kernel keyring keys not garbage collected in 8.3, 8-stream kernels
Summary: Expired/Revoked kernel keyring keys not garbage collected in 8.3, 8-stream ke...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: kernel
Version: CentOS Stream
Hardware: x86_64
OS: Unspecified
unspecified
high
Target Milestone: rc
: ---
Assignee: David Howells
QA Contact: Linqing Lu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-11-08 10:37 UTC by Ben Roberts
Modified: 2023-12-07 04:25 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-08 07:28:25 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
CentOS 18343 0 None None None 2021-11-08 10:37:23 UTC
Red Hat Issue Tracker RHELPLAN-101952 0 None None None 2021-11-08 10:38:28 UTC

Description Ben Roberts 2021-11-08 10:37:24 UTC
Description of problem:

On CentOS 8 Stream kernel 4.18.0-348.el8.x86_64 (originally discovered on CentOS 8.3), I'm seeing multiple users exceed their kernel keyring keys quota, and checking the output of `/proc/keys` for the affected users almost all keys are expired/revoked with a refcount of 1 or higher. Being revoked, these cannot be interacted with using keyctl. It appears that either garbage collection is not running, or these expired/revoked keys are not being cleaned up when GC does run.

I have verified that downgrading to an 8.1 kernel (leaving the userspace on 8.3), the GC does run correctly and and expired keys do not hang around indefinitely.

Version-Release number of selected component (if applicable):
CentOS 8.3 userspace with CentOS 8 Stream 4.18.0-348.el8.x86_64 kernel

How reproducible:
Always on affected kernel versions

Kernel versions tested:
- 8.1 4.18.0-147.5.1.el8.x86_64 NOT affected
- 8.3 4.18.0-240.1.1.el8.x86_64 affected
- 8.3 4.18.0-240.22.1.el8.x86_64 affected
- 8-stream 4.18.0-348.el8.x86_64 affected


Steps to Reproduce:
- Boot system with 8 Stream, or CentOS 8.3/8.4 kernel
- SSH in to the host, logout, repeat a few times
- Observe from /proc/key-users that the number of keys in use is increasing
- Observe from /proc/keys that keys relating to the logged out SSH connections are revoked/expired
- Wait > 5 minutes for the GC routine to run
- Observe that expired/revoked keys have not been cleaned up
- On a system with sufficient activity, observe that the per-user quota limit will be reached and subsequently new keys cannot be created.

Actual results:

- /proc/keys fills with expired/revoked keys
- /proc/key-users shows the per-user quota limit is reached
- Attempts to create new keyrings fail with quota error

Expected results:
- Revoked/expired keys to be garbage collected in a timely fashion

Additional info:

Example list of keys when the host is in this situation.
```
$ cat /proc/keys
00757a86 IR-Q--- 2 expd 3f030000 15171 5000 keyring _ses: empty
00aebcec IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
00d874cc IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
00f354b4 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
01317f10 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
01fc9be6 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
0274d916 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
02fa88a1 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
0301f42c IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
0307418a IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
0337fe9a IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
0363e414 I--Q--- 3 perm 1f3f0000 15171 65534 keyring _uid.15171: empty
03a58bbc IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
03ded0ca IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
03e00b58 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
047825a4 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
049eb199 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
04d34978 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
052f1971 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
065b1604 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
065df0d3 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
066f89d9 I--Q--- 1 perm 1f3f0000 15171 65534 keyring _uid_ses.15171: 1
06749a56 IR-Q--- 1 expd 3f030000 15171 5000 keyring _ses: empty
0680f7be IR-Q--- 18 expd 3f030000 15171 5000 keyring _ses: empty
# ...
```

Comment 5 RHEL Program Management 2023-05-08 07:28:25 UTC
After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Comment 6 Red Hat Bugzilla 2023-12-07 04:25:45 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days


Note You need to log in before you can comment on or make changes to this bug.