Bug 1295189

Created attachment 1111139 [details]
Program to reproduce the problem

Description of problem:
Yes i mark this thing urgent as it has imo the potential to break any notably loaded web services where apache is forking subprocesses, what i guess will be quite a number.

A multithreaded process is hanging during fork after having done some malloc and free calls. The problem is more likely, the more threads the process has.

Version-Release number of selected component (if applicable): glibc-2.22.6


How reproducible:
Compile and run the attached program. Using preprocessor flags one can specify the number of threads (macro NTHREADS, default=40) and the likelyness a thread does a fork in it's endless loop (default 0.1). Example:
gcc tmf.c -o tmf -lpthread -g -DNTHREADS=32

What the program does is to create a number of threads. Each of the threads is an endless loop doing malloc or free of a list of pointers. With a certain chance (default 10%) it does a fork and executes usleep 1 in a subprocess.

The problem is, that the more threads are started, the more likely all threads of the process hang in a futex system call with a stack like this:
#0  0xb7757bc8 in __kernel_vsyscall ()
#1  0xb764e392 in __lll_lock_wait_private () from /lib/libc.so.6
#2  0xb75bf454 in malloc () from /lib/libc.so.6
#3  0x08048885 in th (nth=0xe) at tmf.c:57
#4  0xb771b452 in start_thread () from /lib/libpthread.so.0
#5  0xb76402fe in clone () from /lib/libc.so.6

On my Pentium single processor machine with 8 threads the process almost never hangs and uses all CPU it can get, as it should. With 40 threads the process always hangs and inbetween the more likely, the more threads it has.

Steps to Reproduce:
1. gcc tmf.c -o tmf -lpthread -DNTHREADS=40
2. ./tmf
3. if you like, attach strace or gdb to the process and see more details

Actual results:
Process hangs

Expected results:
All threads are continuously running, doing malloc and free and fork ...

Additional info:
The phenomenon is never seen with lower glibc versions e.g. on Fedora 21 or 22.
So imo this is clearly a new problem with the potential to break many services.
There is a similar bug open: https://bugzilla.redhat.com/show_bug.cgi?id=906468 . Nonetheless as this one refers to RHEL 7 and dates back to 2013, i open this new one, because my suspicion is, that it is a different issue. Could be, that trying to fix 906468 this problem here has been introduced.