RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1711520 - IPSet List - Size in Memory Wildly Inconsistent
Summary: IPSet List - Size in Memory Wildly Inconsistent
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: kernel
Version: 7.6
Hardware: x86_64
OS: Linux
low
low
Target Milestone: rc
: 7.8
Assignee: Stefano Brivio
QA Contact: yiche
Marc Muehlfeld
URL:
Whiteboard:
Depends On:
Blocks: 1714111 1723958
TreeView+ depends on / blocked
 
Reported: 2019-05-18 09:17 UTC by NOYB
Modified: 2020-03-31 19:20 UTC (History)
7 users (show)

Fixed In Version: kernel-3.10.0-1093.el7
Doc Type: Bug Fix
Doc Text:
.The `ipset list` command reports consistent memory for `hash` set types When you add entries to a `hash` set type, the `ipset` utility must resize the in-memory representation to for new entries by allocating an additional memory block. Previously, `ipset` set the total per-set allocated size to only the size of the new block instead of adding the value to the current in-memory size. As a consequence, the `ip list` command reported an inconsistent memory size. With this update, `ipset` correctly calculates the in-memory size. As a result, the `ipset list` command now displays the correct in-memory size of the set, and the output matches the actual allocated memory for `hash` set types.
Clone Of:
: 1714111 (view as bug list)
Environment:
Last Closed: 2020-03-31 19:18:03 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Reproducer: create two hash:ip sets, add entries, swap (863 bytes, application/x-shellscript)
2019-05-26 15:41 UTC, Stefano Brivio 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2020:1016 0 None None None 2020-03-31 19:20:33 UTC

Description NOYB 2019-05-18 09:17:42 UTC 
Description of problem:
Size in memory reported by ipset list <list_name> changes wildly.  Sometimes it will be around 10K and sometimes around 90K.

Version-Release number of selected component (if applicable):
ipset v6.38, protocol version: 6

How reproducible:
Don't have a good handle on that.  Maybe 25%.  Don't know how to consistently trigger it.

Steps to Reproduce:
1. Create a nearly identical list but with a few new entries.
2. Swap with existing list.
3. ipset list <list_name>

Actual results:
Sometimes a seemingly wildly incorrect size in memory will be reported for the new list.

Expected results:
Correct size in memory reported.

Additional info:
Sometimes the reported size in memory is way out of line with what is expected.  Other times it seems reasonable.
Like say size in memory of 8184 for an inet list of 4345 entries.
Then other times the size in memory reported will by about 95224 or so for nearly the same list after swap with just a few more entries (4360 entries).

A script is running every 15 minutes that creates a new list with some additional entries (CIDR and single IP address) sorted via sort -V.  The new list is then swapped with the existing list.  Sometimes the size in memory reported is about 10K and other times it is about 90K.  Even though the number of entries has changed very little.
Comment 2 Stefano Brivio 2019-05-26 15:41:58 UTC 
Created attachment 1573620 [details]
Reproducer: create two hash:ip sets, add entries, swap

Hi NOYB,

(In reply to NOYB from comment #0)
> Sometimes the reported size in memory is way out of line with what is
> expected.  Other times it seems reasonable.
> Like say size in memory of 8184 for an inet list of 4345 entries.
> Then other times the size in memory reported will by about 95224 or so for
> nearly the same list after swap with just a few more entries (4360 entries).
> 
> A script is running every 15 minutes that creates a new list with some
> additional entries (CIDR and single IP address) sorted via sort -V.  The new
> list is then swapped with the existing list.  Sometimes the size in memory
> reported is about 10K and other times it is about 90K.  Even though the
> number of entries has changed very little.

Thanks for reporting this. The issue can be reproduced rather easily with two
hash:ip sets and the number of entries you provided, see the attached
reproducer.

I identified an issue in the kernel implementation of ipset, that leads to an
inconsistent in-memory size reporting caused by an unexpected reset of
memory accounting during hash table resize with allocation of new array blocks.

I'm about to send a patch to fix this to the upstream ipset maintainer. Can
you please give me a real name so that I can credit you for reporting this
issue? I can also use "NOYB" and the email address you provided, or omit this
altogether. Please let me know.
Comment 3 NOYB 2019-05-26 19:15:24 UTC 
Thanks for your work on this.  Glad you are able to easily reproduce the issue and provide a fix.
For credit NOYB and email address is fine.
Thanks
Comment 4 yiche 2019-05-27 03:17:26 UTC 
Thank you NOYB for report this bug.
Thank Stefano Brivio to provides such good reproducer.
Set qa_ack +
Comment 5 Stefano Brivio 2019-05-27 14:47:58 UTC 
Patch upstream at:

    http://patchwork.ozlabs.org/patch/1105608/
Comment 7 Jan Stancek 2019-09-17 07:34:54 UTC 
Patch(es) committed on kernel-3.10.0-1093.el7
Comment 12 errata-xmlrpc 2020-03-31 19:18:03 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:1016
Note You need to log in before you can comment on or make changes to this bug.