Bug 120982

Summary: opendir returns wrong errno when memory allocation fails.
Product: [Fedora] Fedora Reporter: Oleg Drokin <green>
Component: glibcAssignee: Jakub Jelinek <jakub>
Status: CLOSED NOTABUG QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 1   
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2004-04-15 21:34:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Oleg Drokin 2004-04-15 20:44:17 UTC 
Description of problem:

opendir(3) return incorrect errno value if memory allocation failed.

Version-Release number of selected component (if applicable):
2.3.2-101.4 (also all prior ones that were included into FC1)

How reproducible:

make a __wrap_malloc function that returns zero and call opendir(),
it will return NULL, errno will contain something strange (in my case
EINTR that was there just prior to the call to opendir).
errno should contain ENOMEM I believe.
We came across this problem while using UserModeLinux,
the wrapper malloc used tried to allocate memory with kernel's kmalloc
implementation that is limited to 128k, by default and opendir
tried to allocate a slightly bigger amount, but errno diverted our
attention and we looked elsewhere for the problem until actually
I singlestepped through opendir and found the real reason.
Comment 1 Jakub Jelinek 2004-04-15 20:56:53 UTC 
Returns zero and doesn't set errno to ENOMEM?
glibc of course relies on malloc/calloc/realloc setting errno to ENOMEM
if it fails.
Comment 2 Oleg Drokin 2004-04-15 21:24:06 UTC 
Hm. Our malloc implementation does not set errno to ENOMEM and
according to malloc man page in FC1:
       For calloc() and malloc(), the value returned is a pointer to
the allo-
       cated  memory,  which  is suitably aligned for any kind of
variable, or
       NULL if the request fails.

and not a word about setting errno. Or is this sort of internal
undocumented feature glibc is relying on and should we actually go
ahead and set the errno in our malloc wrapper?
Comment 3 Jakub Jelinek 2004-04-15 21:34:08 UTC 
It is mandated by the standards.
See e.g.
http://www.opengroup.org/onlinepubs/007904975/functions/malloc.html

Otherwise, it shall return a null pointer and set errno to indicate the error.
The malloc() function shall fail if: [ENOMEM] Insufficient storage space is available.