Bug 961388 - glibc does not unload libpython (circular dependencies?)
glibc does not unload libpython (circular dependencies?)
Product: Fedora
Classification: Fedora
Component: glibc (Show other bugs)
Unspecified Unspecified
unspecified Severity medium
: ---
: ---
Assigned To: Carlos O'Donell
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2013-05-09 10:18 EDT by Jan Safranek
Modified: 2016-11-24 11:10 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2013-05-09 16:21:31 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Reproducer (735 bytes, application/x-compressed-tar)
2013-05-09 10:18 EDT, Jan Safranek
no flags Details
output of LD_DEBUG=all with loaded datetime.so (551.19 KB, text/plain)
2013-05-09 10:20 EDT, Jan Safranek
no flags Details
output of LD_DEBUG=all without datetime.so (just plain "print 'hello world'") (475.85 KB, text/plain)
2013-05-09 10:21 EDT, Jan Safranek
no flags Details

  None (edit)
Description Jan Safranek 2013-05-09 10:18:43 EDT
Created attachment 745676 [details]

Description of problem:
I have a network daemon, which dlopen()s a plugin. This plugin is linked with libpython2.7.so.1, so it gets loaded too. When the daemon dlclose()s the plugin, the plugin is unloaded (as visible in /proc/123/maps), but libpython is *not* unloaded.

I think that's because python loads other modules (e.g. /usr/lib64/python2.7/lib-dynload/datetime.so), which depend back on libpython2.7.1.so (->circular dependencies) and glibc does not detect this.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

See attached reproducer, it contains:
- main.c (="network daemon"), which loads libmylib.so.
- mylib.c (="plugin"), which initializes python and runs a command in it.

1. yum install python-devel
2. make
3. ./main "import datetime; print datetime.datetime.now()"
4. when prompted to press ENTER, run in another terminal:
   grep python /proc/`pidof main`/maps

Actual results:
python is still loaded:
335c56d000-335c5aa000 rw-p 0016d000 fd:00 182911                         /usr/lib64/libpython2.7.so.1.0
7fbf17772000-7fbf17783000 r-xp 00000000 fd:00 200309                     /usr/lib64/python2.7/lib-dynload/datetime.so

Expected results:
python is not loaded

If I run ./main "print 'hello world'", the python does not load any additional .so and it is removed from memory after dlclose() of libmylib.so.

Whole goal of this exercise is to work around a Python bug. Python does not like to be re-initialized, it uses some global variables which are not cleaned correctly on Py_Finalize(). So unloading libpython  from address space of the network daemon would ensure that with next dlopen() and linking libpython, all global variables would be reset to sane values and the daemon could instantiate Python interpreter for second time.
Comment 1 Jan Safranek 2013-05-09 10:20:51 EDT
Created attachment 745677 [details]
output of LD_DEBUG=all with loaded datetime.so
Comment 2 Jan Safranek 2013-05-09 10:21:57 EDT
Created attachment 745678 [details]
output of LD_DEBUG=all without datetime.so (just plain "print 'hello world'")
Comment 3 Jan Safranek 2013-05-09 10:27:47 EDT
Without datetime.so was generated by:

./main "print 'hello world'"

Notice unloading of libmylib:

calling fini: ./libmylib.so [0]
calling fini: /lib64/libpython2.7.so.1.0 [0]
calling fini: /lib64/libutil.so.1 [0]
calling fini: /lib64/libm.so.6 [0]

With datetime.so was generated by:

./main "import datetime; print datetime.datetime.now()"

Notice unloading of libmylib:
calling fini: ./libmylib.so [0]

[and nothing else, libpython stays loaded until ./main exits]
Comment 4 Carlos O'Donell 2013-05-09 16:21:31 EDT
I quickly walked through and reviewed the unload sequence (_dl_close_worker) in your test case, thanks for providing that.

This is a python bug.

Python dlopen's datetime.so, but fails to dlcose datetime.so during Py_Finalize(). It is python's responsibility to dlclose any DSO it manually dlopens, otherwise you are going to see the problems you see in this issue.

When you unload libmylib.so the dynamic loader walks the entire list of dependencies an is fully ready to unload libpython and all the associated DSOs. 
When we reach DSO #10, which is datetime.so, we notice that it was *not* loaded as a dependency (l_direct_opencount > 0), but rather manually loaded by libpython. The dynamic loader can't automatically unload a manually loaded DSO as that would violate the API. Therefore to keep datetime.so you have to keep all the other loaded DSOs, and because datetime.so depends on python, you can't unload python.

The solution is to have Py_Finalize() unload all the DSOs it loaded during the execution of any script.

I'm marking this CLOSED/NOTABUG.
Comment 5 Jan Safranek 2013-05-10 02:59:48 EDT
Thanks a lot for detailed analysis!

Note You need to log in before you can comment on or make changes to this bug.