It is my understanding that this issue arises from a design issue in the dynamic linker.  The dynamic linker invokes ELF constructors and destructors (essentially callback functions) while internal locks are acquired.  To enable dynamic linker usage from these callback functions without self-deadlocking, those locks are recursive locks.  However, recursive locks do not help if the callback spawns another thread to do the work, or hands off the work to another thread and waits until that thread signals the work is complete.

This code needs to be rewritten upstream, so that no locks are acquired while the dynamic linker calls into user code.  To my knowledge, no such patches exist yet, and this has not worked in any glibc release.

(In reply to Florian Weimer from comment #3)
> It is my understanding that this issue arises from a design issue in the
> dynamic linker.  The dynamic linker invokes ELF constructors and destructors
> (essentially callback functions) while internal locks are acquired.  To
> enable dynamic linker usage from these callback functions without
> self-deadlocking, those locks are recursive locks.  However, recursive locks
> do not help if the callback spawns another thread to do the work, or hands
> off the work to another thread and waits until that thread signals the work
> is complete.

Exactly right.

We have had several instances of this problem in the last year where applications are doing too much in constructors or destructors, and it usually involves starting, controlling, and stopping threads. This is quickly leads to deadlocks.
 
> This code needs to be rewritten upstream, so that no locks are acquired
> while the dynamic linker calls into user code.  To my knowledge, no such
> patches exist yet, and this has not worked in any glibc release.

It is indeed a problem to call foreign functions (constructors and destructors) with locks held, and it is a long-term goal to simplify this in the dynamic loader to attempt to make it possible while still maintaining the consistency of the loaded libraries.

I have also never seen any patches to fix this upstream. The closest was a discussion I had with Mathieu Desnoyers (lttng, liburcu) at LPC 2016 where I discussed the use of liburcu and RCU in general to break the internal dynamic loader load lock.

As Florian points out, this has never worked, and if anything this is a new feature request.

Changing this to an RFE based on comment 4.

Given that this bug requires extensive work upstream to resolve I'm going to mark this as CLOSED/UPSTREAM.

We are going to review and track the bug upstream here:
https://sourceware.org/bugzilla/show_bug.cgi?id=15686

The solution upstream is that we have to be able to release the locks while constructors run, but this is quite a hard thing to implement and keep the expected semantics.