From Bugzilla Helper: User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.5) Gecko/20031007 Description of problem: I'm trying to use Sybase ASE 11 on Enterprise Linux Release 3. It works fine on {E,W,A}S Release 2.1 and Redhat < 9, but not on the newer versions. The problem appears to occur when attaching a shared memory segment. An strace from the older versions contains lines like this: shmget(1493352882, 260300800, IPC_CREAT|IPC_EXCL|0600) = 32768 shmat(32768, 0x41000000, 0) = 0x41000000 On WS 3 I see lines like this: shmget(16957874, 259203072, IPC_CREAT|IPC_EXCL|0600) = 491520 shmat(491520, 0xb8000000, 0) = -1 EINVAL (Invalid argument) The address is actually valid (it's computed by ASE from the data in /proc/sys/maps, and I've verified that a block of the size requested fits in that location.) According to shmop(2), when shmat fails with EINVAL it could be: Invalid shmid value, unaligned (i.e., not page-aligned and SHM_RND was not specified) or invalid shmaddr value, or failing attach at brk. The shmid is clearly valid having been returned by shmget (and can be seen using ipcs -a). The address is page-aligned and valid. I don't know what could cause brk to fail other than lack of memory, and as already stated, that doesn't appear to be the case. Here's the contents of /proc/self/maps, also gleaned from the strace output: 08048000-0837b000 r-xp 00000000 03:02 1311284 /opt/sybase/bin/dataserver 0837b000-083f4000 rw-p 00332000 03:02 1311284 /opt/sybase/bin/dataserver 083f4000-0840a000 rw-p 00000000 00:00 0 b7485000-b7486000 rw-p 00000000 00:00 0 b7486000-b75b9000 r-xp 00000000 03:02 49578 /lib/i686/libc-2.2.4.so b75b9000-b75be000 rw-p 00132000 03:02 49578 /lib/i686/libc-2.2.4.so b75be000-b75c2000 rw-p 00000000 00:00 0 b75c2000-b75e4000 r-xp 00000000 03:02 49580 /lib/i686/libm-2.2.4.so b75e4000-b75e5000 rw-p 00021000 03:02 49580 /lib/i686/libm-2.2.4.so b75e5000-b75e8000 r-xp 00000000 03:02 2818441 /lib/libdl-2.2.4.so b75e8000-b75e9000 rw-p 00002000 03:02 2818441 /lib/libdl-2.2.4.so b75e9000-b75ea000 rw-p 00000000 00:00 0 b75ea000-b7600000 r-xp 00000000 03:02 2818428 /lib/ld-2.2.4.so b7600000-b7601000 rw-p 00015000 03:02 2818428 /lib/ld-2.2.4.so bfff4000-c0000000 rwxp ffff7000 00:00 0 I tried booting a WS 2.1 box with the 2.4.21-9.EL kernel and the problem then started occuring there. I've also tried building a non-Redhat 2.4.21 kernel (i.e. one built from the upstream sources without any RH patches) and the problem doesn't occur with it. In summary, the problem occurs with 2.4.21-4.EL and 2.4.21-9.EL, but not with 2.4.9-e.24 or 2.4.21 [unpatched, non-Redhat]. Version-Release number of selected component (if applicable): kernel-2.4.21-9.EL How reproducible: Always Steps to Reproduce: I don't have access to the application's source code and I'm not a skilled enough C programmer to try to recreate the problem on my own. Additional info:
shmget(16957874, 259203072, IPC_CREAT|IPC_EXCL|0600) = 491520 shmat(491520, 0xb8000000, 0) = -1 EINVAL (Invalid argument) the application asks for 259203072 bytes at address 0xb8000000.. with the process address space ending at 0xc0000000 (and the stack being there) that doesn't look like it will it..... sounds like a bug in the program to me; it asks for something invalid and the kernel returns -EINVAL
Huh! I'll be... that makes perfect sense. If I'd researched the format of /proc/$$/maps I might've worked that out for myself. Thanks very much for the incredibly quick reply.
Hmmm... Not really sure if this should be classified as "NOTABUG" because sybase works properly under other non-RHEL3 versions, but OK. The buzz in the sybase community is that older versions of Sybase do not work, which will make folks avoid RHEL3 for sybase applications. Personally, I think the folks at RedHat tweeked the output of /proc/self/maps to the point that it breaks some applications. It would be nice if RedHat took this seriously to figure out why this is happening, or provide a workaround. When sybase boots, it will only allow a maximum of 128MB of shared memory. My guess is that whatever line it is reading in /proc/self/maps incorrectly reports the amount of free shared memory that is available. For the benefit of others, here is a workaround that makes older versions of sybase 11.x, work properly under RHEL3. To fix sybase 11.0.3.3, it is necessary to edit the sybase "dataserver" binary to fool sybase into thinking that "/proc/self/maps" is invalid. Older versions of Sybase consulted this file during initialization to figure out where to map the shared memory. If sybase doesn't find this file, it consults the kernel which aparently works. Do the following: vi -b dataserver search for the **FIRST** occurrence "/proc/self/maps": /proc/self/maps^@NULL^@key: 0x%lx id: %d file: ^@...^@^@0x%lx,0x%lx,0x%lx,0x%x^@os_create_keyfile Change "/proc" to something different... I used "/Proc". Change ONLY the above entry. Save the binary and sybase will now properly assign the shared memory, as defined in /proc/sys/kernel/shmmax. Mike