RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1215452 - Shared object compiled with g++ is not unloaded
Summary: Shared object compiled with g++ is not unloaded
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: glibc
Version: 7.0
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Carlos O'Donell
QA Contact: qe-baseos-tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-26 17:10 UTC by Max Dmitrichenko
Modified: 2015-12-15 20:53 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-12-15 19:58:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Test case (10.00 KB, application/x-tar)
2015-04-26 17:10 UTC, Max Dmitrichenko
no flags Details

Description Max Dmitrichenko 2015-04-26 17:10:46 UTC
Created attachment 1019069 [details]
Test case

Description of problem:
I've faced with strange behavior when I investigated a bug on RHEL7.

My project heavily uses plugins which are dynamically loaded with
dlopen() and unloaded with dlclose() functions. I've got report that
some (not all) plugins are not unloaded: they remain in the process
map and if you put a new version of binary it's not loaded until
program is restarted. This happens only on RHEL 7.x (gcc 4.8), while on RHEL 6.x (gcc 4.4) everything works as expected. Checked only on x86_64 architecture.

So I "bisected" the code to find why it happens. I've found that if
plugin contains a static variable in the inlined function, then
dlclose() leaves the SO in the memory. Also I've found that it happens
only if it is compiled with g++. The same example renamed to *.c and
compiled with gcc works OK.

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux 7,

How reproducible:

Attached is the sample which reproduces this effect. Use make command
to build 'test' executable and 'test.so' shared object, then run:
$ ./test

If it outputs something like:
7fc80cdf2000-7fc80cdf3000 r-xp 00000000 fe:02 5902461
  /home/tests/so/test.so
7fc80cdf3000-7fc80cff2000 ---p 00001000 fe:02 5902461
  /home/tests/so/test.so
7fc80cff2000-7fc80cff3000 rw-p 00000000 fe:02 5902461
  /home/tests/so/test.so

then shared object was not unloaded. Remove "inline" keyword in so.cpp
and everything works fine.

Steps to Reproduce:
1. Download attached file and untar it
2. type make to build the test
3. run the test typing ./test

Actual results:
Test outputs lines of /proc/self/maps file, which are related to tested shared object (test.so)
7fc80cdf2000-7fc80cdf3000 r-xp 00000000 fe:02 5902461
  /home/tests/so/test.so
7fc80cdf3000-7fc80cff2000 ---p 00001000 fe:02 5902461
  /home/tests/so/test.so
7fc80cff2000-7fc80cff3000 rw-p 00000000 fe:02 5902461
  /home/tests/so/test.so

Expected results:
Test output should be empty, as the shared object should be unloaded.

Additional info:

Comment 2 Max Dmitrichenko 2015-04-26 18:43:19 UTC
Added to CC

Comment 4 Jakub Jelinek 2015-12-15 18:51:13 UTC
That is simply how STB_GNU_UNIQUE symbols behave, those symbols are pushed by the dynamic linker into a global hash table, and that makes the shared libraries non-unloadable. You can use -fno-gnu-unique if you are fine with the consequences.

If anything would change here, it would be the dynamic linker, that in theory could support tracking the reference counts of the unique symbols and unload when all of the symbols are no longer referenced.  But I guess the runtime cost could be prohibitive.

Comment 5 Florian Weimer 2015-12-15 18:55:46 UTC
(In reply to Jakub Jelinek from comment #4)
> That is simply how STB_GNU_UNIQUE symbols behave, those symbols are pushed
> by the dynamic linker into a global hash table, and that makes the shared
> libraries non-unloadable. You can use -fno-gnu-unique if you are fine with
> the consequences.
> 
> If anything would change here, it would be the dynamic linker, that in
> theory could support tracking the reference counts of the unique symbols and
> unload when all of the symbols are no longer referenced.  But I guess the
> runtime cost could be prohibitive.

We have existing applications that rely on the fact that callback functions (such as destructors) are not unloaded.  If we start unloading DSOs when we did not do so before, these applications would be broken.  I'm not sure if this is a reasonable trade-off, although affected applications could be considered latently buggy.

Comment 6 Jeff Law 2015-12-15 19:17:41 UTC
So more context.

The use of the static variable in an inlined function in C++ triggers use of STB_GNU_UNIQUE symbols, which currently have the property that when used, they prevent the library from unloading.

As Jakub notes, reference counting the unique symbols so their associated libraries could be unloaded would be prohibitively expensive.  Additionally, as Florian notes, it's possible (likely?) that existing applications are relying on this behaviour and unloading when were not previously unloading could cause those applications to break.

In all, I think things are working as designed.

Comment 7 Max Dmitrichenko 2015-12-15 19:45:36 UTC
First of all, may be the things are working as it was worked around (or dirty hacked) some time ago. But hardly I can call this with such a word as "design" )

At least such behavior isn't documented at all in any document which a simple programmer could read. And not only me, but a lot of programmers were confused with it (according to what I've learnt when I was searching for any info on this). But hardly any of them traced the problem down to a simple example and got an answer in the gcc mailing. The feature that is not document and causes misbehavior is a bug.

The second thing is the runtime cost. But what cost we are talking about? The reference counting here happens only when additional SO is opened or closed by the process. During opening the linker should increment reference count, during close - decrement and unload SO if it not needed anymore. Do applications open and close SO often? I doubt so. Memory? Additional counter for each local static variable in the inlined method?

On the other hand, a lot of software support plugins which needs to be updated from time to time. I faced with it, when I tried to update plugin and it was not updated until program restart. And the problem was not in the plugin itself, but in the fact it was using boost library, which has somewhere a static variable in the inline method which was inlined in the plugin.

Comment 8 Carlos O'Donell 2015-12-15 20:53:38 UTC
I agree with the designation of CLOSED/WONTFIX for RHEL7.

The compiler option is quite clear:

-fno-gnu-unique
On systems with recent GNU assembler and C library, the C++ compiler uses the
"STB_GNU_UNIQUE" binding to make sure that definitions of template static data
members and static local variables in inline functions are unique even in the
presence of "RTLD_LOCAL"; this is necessary to avoid problems with a library
used by two different "RTLD_LOCAL" plugins depending on a definition in one of
them and therefore disagreeing with the other one about the binding of the
symbol.  But this causes "dlclose" to be ignored for affected DSOs; if your
program relies on reinitialization of a DSO via "dlclose" and "dlopen", you can
use -fno-gnu-unique.

It is unknown how much it would cost to reference count everything and consider the DSO for dlclose. That would certainly be an interesting RFC, but it would have to start upstream.


Note You need to log in before you can comment on or make changes to this bug.