Description of problem: memchr() with long length returns NULL although searched character exists in the searched string. Problem does not exist in glibc 2.9 or older. Problem was found because VMware customers are reporting ASSERTs being hit in VMware's code due to unexpected memchr() return value. Version-Release number of selected component (if applicable): 2.10.1 according to the http://communities.vmware.com/thread/214671. How reproducible: Problem is 100% reproducible using code below. Steps to Reproduce: /* make sure to build this as 64bit binary. Code should report 0x2525b80 0x2525bc9. On current Fedora (with 2.10.1 glibc) it returns 0x2525b80 (nil). */ #include <sys/mman.h> #include <stdio.h> #include <string.h> void main(void) { char* abc; abc = mmap(0x00002525000, 16384, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); abc += 0xB80; strcpy(abc, "/mnt/data/dan/VMware/Windows XP Professional/Windows XP Professional.vmdk"); printf("%p %p\n", abc, memchr(abc, 0, -(unsigned long)abc)); } Actual results: 0x2525b80 (nil) Expected results: 0x2525b80 0x2525bc9 Additional info: memchr() behavior above causes vmware to crash, as it expects that memchr(string, 0, -(unsigned long)string) on NUL terminated string returns NULL only if string is not terminated at all.
Introduced in 2.10.1 by http://sourceware.org/cgi-bin/cvsweb.cgi/libc/sysdeps/x86_64/memchr.S.diff?cvsroot=glibc&r1=NONE&r2=1.1 It uses signed comparsion between number of characters processed (%rsi) and requested for processing (%rdx): cmp %rsi,%rdx jle xxx It means that memchr() with length argument over 2^63 will immediately fail these tests, returning NULL. Switching comparsions to unsigned (and depending on the fact that before %rsi can reach 2^63 process will be killed by #GP or #PF so there is no possibility of looping forever for length=-1..-15) seems to do the trick.
Fixed in git so far.
This is fixed in rawhide for quite sime time as well.