Bug 1468410 – System Python gc does not reclaim when large memory is allocated

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1468410 - System Python gc does not reclaim when large memory is allocated

Summary:	System Python gc does not reclaim when large memory is allocated

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	python (Show other bugs)
Sub Component:
Version:	7.3
Hardware:	x86_64 Linux

Priority	unspecified Severity urgent
Target Milestone:	rc
Target Release:	---
Assigned To:	Python Maintainers
QA Contact:	Mirosław Zalewski
Docs Contact:

URL:
Whiteboard:
Keywords:

Depends On:
Blocks:	1473612
	Show dependency tree / graph

Reported:	2017-07-06 22:23 EDT by ckim0419
Modified:	2018-04-10 11:00 EDT (History)
CC List:	3 users (show)

See Also:
Fixed In Version:	python-2.7.5-63.el7
Doc Type:	If docs needed, set a value
Doc Text:
Story Points:	---
Clone Of:
Environment:
Last Closed:	2018-04-10 11:00:08 EDT
Type:	Bug
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
screenshot of top and python (292.44 KB, image/png) 2017-07-06 22:23 EDT, ckim0419	no flags	Details
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Python	11849	None	None	None	2017-10-12 11:40 EDT
Red Hat Product Errata	RHBA-2018:0833	None	None	None	2018-04-10 11:00 EDT

Groups: None (edit)

Description ckim0419 2017-07-06 22:23:35 EDT

Created attachment 1295144 [details]
screenshot of top and python

Description of problem:
gc of python that comes with RHEL7.3 does not collect all memory it allocates

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux Server release 7.3 (Maipo)
Linux localhost.localdomain 3.10.0-514.26.2.el7.x86_64 #1 SMP Fri Jun 30 05:26:04 UTC 2017 x86_64 x86_
Python 2.7.5 (default, Aug  2 2016, 04:20:16)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2

How reproducible:
d = [ {'sdfdfsdfsfef': 1} for x in xrange(10000000)]
del d
import gc; gc.collect()

Steps to Reproduce:
1. Open python on RHEL7 and enter following python code
2. d = [ {'sdfdfsdfsfef': 1} for x in xrange(10000000)]
3. del d
4. import gc; gc.collect()
5. Run top or ps to get memory allocation of the process

Actual results:
The process still uses 2.4 GB of Memory

Expected results:
Mem usage less than 100KB

Additional info:

Comment 2 Petr Viktorin 2017-10-05 09:40:03 EDT

I plan to look into this in the RHEL 7.5 devel phase.

Comment 3 Petr Viktorin 2017-10-12 11:40:45 EDT

Note that the memory is not leaked; it is only not "returned" to the OS; subsequent allocations will re-use the memory.

That being said, this was fixed upstream (in Python 3.3 and 2.7.7) by using mmap() to allocate arenas and using pre-allocated pools for dicts. From an engineering POV, backporting the patch shouldn't be difficult.

Comment 9 errata-xmlrpc 2018-04-10 11:00:08 EDT

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:0833

Note You need to log in before you can comment on or make changes to this bug.