Hide Forgot
Description of problem: Size in memory reported by ipset list <list_name> changes wildly. Sometimes it will be around 10K and sometimes around 90K. Version-Release number of selected component (if applicable): ipset v6.38, protocol version: 6 How reproducible: Don't have a good handle on that. Maybe 25%. Don't know how to consistently trigger it. Steps to Reproduce: 1. Create a nearly identical list but with a few new entries. 2. Swap with existing list. 3. ipset list <list_name> Actual results: Sometimes a seemingly wildly incorrect size in memory will be reported for the new list. Expected results: Correct size in memory reported. Additional info: Sometimes the reported size in memory is way out of line with what is expected. Other times it seems reasonable. Like say size in memory of 8184 for an inet list of 4345 entries. Then other times the size in memory reported will by about 95224 or so for nearly the same list after swap with just a few more entries (4360 entries). A script is running every 15 minutes that creates a new list with some additional entries (CIDR and single IP address) sorted via sort -V. The new list is then swapped with the existing list. Sometimes the size in memory reported is about 10K and other times it is about 90K. Even though the number of entries has changed very little.
Created attachment 1573620 [details] Reproducer: create two hash:ip sets, add entries, swap Hi NOYB, (In reply to NOYB from comment #0) > Sometimes the reported size in memory is way out of line with what is > expected. Other times it seems reasonable. > Like say size in memory of 8184 for an inet list of 4345 entries. > Then other times the size in memory reported will by about 95224 or so for > nearly the same list after swap with just a few more entries (4360 entries). > > A script is running every 15 minutes that creates a new list with some > additional entries (CIDR and single IP address) sorted via sort -V. The new > list is then swapped with the existing list. Sometimes the size in memory > reported is about 10K and other times it is about 90K. Even though the > number of entries has changed very little. Thanks for reporting this. The issue can be reproduced rather easily with two hash:ip sets and the number of entries you provided, see the attached reproducer. I identified an issue in the kernel implementation of ipset, that leads to an inconsistent in-memory size reporting caused by an unexpected reset of memory accounting during hash table resize with allocation of new array blocks. I'm about to send a patch to fix this to the upstream ipset maintainer. Can you please give me a real name so that I can credit you for reporting this issue? I can also use "NOYB" and the email address you provided, or omit this altogether. Please let me know.
Thanks for your work on this. Glad you are able to easily reproduce the issue and provide a fix. For credit NOYB and email address is fine. Thanks
Thank you NOYB for report this bug. Thank Stefano Brivio to provides such good reproducer. Set qa_ack +
Patch upstream at: http://patchwork.ozlabs.org/patch/1105608/
Patch(es) committed on kernel-3.10.0-1093.el7
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2020:1016