Bug 656756 (CVE-2010-4249)

http://lkml.org/lkml/2010/11/23/395

Reported by Vegard Nossum:
"I found this program lying around on my laptop. It kills my box (2.6.35) instantly by consuming a lot of memory (allocated by the kernel, so the process doesn't get killed by the OOM killer). As far as I can tell, the memory isn't being freed when the program exits either. Maybe it will eventually get cleaned up the UNIX socket garbage collector thing, but in that case it doesn't get called quickly enough to save my machine at least."

Reproducer: http://lkml.org/lkml/2010/11/23/395
Partial fix: http://lkml.org/lkml/2010/11/23/450
Remaining fix: http://marc.info/?l=linux-netdev&m=129059035929046&w=2

From Eric Dumazet:
"we can eat all LOWMEM memory before unix_gc() being called from unix_release_sock(). Moreover, the thread blocked in unix_gc() can consume huge amount of time to perform cleanup because of huge working set.

One way to handle this is to have a sensible limit on unix_tot_inflight, tested from wait_for_unix_gc() and to force a call to unix_gc() if this limit is hit.

This solves the OOM and also reduce overall latencies, and should not slowdown normal workloads."

Acknowledgements:

Red Hat would like to thank Vegard Nossum for reporting this issue.