The problem ----------- In case of an application contains a lot* of uninitialized data (mapping to the BSS), kernel fails to load it into the memory. It looks like just out of memory, but the kernel does not control result of bss allocation. So, BSS'a pages become overlapped with the executable pages. Finally, kernel tries to run the application. It causes segmentation fault. --- * "a lot" - actuall size of BSS may vary. Usually it is about 1G. Version-Release number ---------------------- 2.4.2-2smp (Red Hat 7.1) 2.4.18-0.1 (Red Hat 7.3b2) How reproducible: Always Steps to Reproduce ------------------ 1. Take attached test-case 2. Build it using any C compiler. For instance: gcc main.c 3. Execute ./a.out Actual Results -------------- Given application failed to run: $ ./a.out Segmentation fault (core dumped) The basic GDB's information is: $ gdb a.out core ... Program terminated with signal 11, Segmentation fault. #0 0x40001dc0 in var () (gdb) bt #0 0x40001dc0 in var () (gdb) We see that the kernel passed control to the variable (var) defined onthe BSS. It's definitely wrong! Expected Results ---------------- Expected result is just an unambiguous behaviour of kernel: - run executable in case memory allocated successfully (then you will see "Passed" message from the given test-case); - kernel's error message showing that the application could not be loaded into the memory.
Created attachment 51604 [details] The failing test-case
looks like ld.so overlaps libs with the bss...
Yes, this is an expected result. Don't do that, use malloc or mmap instead... The kernel is loading the dynamic linker first and the default mmap address is 0x40000000. Red Hat Linux Advanced Server 2.1 allows to tweak that address at runtime for a process and all its future children, or you can prelink(8) the dynamic linker to some other address, so that kernel doesn't mmap it at address 0 (which means choose any), but at some specific address.