Description of problem: opendir(3) return incorrect errno value if memory allocation failed. Version-Release number of selected component (if applicable): 2.3.2-101.4 (also all prior ones that were included into FC1) How reproducible: make a __wrap_malloc function that returns zero and call opendir(), it will return NULL, errno will contain something strange (in my case EINTR that was there just prior to the call to opendir). errno should contain ENOMEM I believe. We came across this problem while using UserModeLinux, the wrapper malloc used tried to allocate memory with kernel's kmalloc implementation that is limited to 128k, by default and opendir tried to allocate a slightly bigger amount, but errno diverted our attention and we looked elsewhere for the problem until actually I singlestepped through opendir and found the real reason.
Returns zero and doesn't set errno to ENOMEM? glibc of course relies on malloc/calloc/realloc setting errno to ENOMEM if it fails.
Hm. Our malloc implementation does not set errno to ENOMEM and according to malloc man page in FC1: For calloc() and malloc(), the value returned is a pointer to the allo- cated memory, which is suitably aligned for any kind of variable, or NULL if the request fails. and not a word about setting errno. Or is this sort of internal undocumented feature glibc is relying on and should we actually go ahead and set the errno in our malloc wrapper?
It is mandated by the standards. See e.g. http://www.opengroup.org/onlinepubs/007904975/functions/malloc.html Otherwise, it shall return a null pointer and set errno to indicate the error. The malloc() function shall fail if: [ENOMEM] Insufficient storage space is available.