Bug 128735 - 32bit Java application doesn't work
32bit Java application doesn't work
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: kernel (Show other bugs)
x86_64 Linux
medium Severity medium
: ---
: ---
Assigned To: Peter Martuccelli
Brian Brock
Depends On:
Blocks: 132991
  Show dependency treegraph
Reported: 2004-07-28 16:06 EDT by H.J. Lu
Modified: 2007-11-30 17:07 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2007-07-12 13:33:59 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
A patch (1.04 KB, patch)
2004-07-28 16:07 EDT, H.J. Lu
no flags Details | Diff

  None (edit)
Comment 1 H.J. Lu 2004-07-28 16:07:52 EDT
Created attachment 102266 [details]
 A patch
Comment 2 Jim Paradis 2004-08-21 02:17:19 EDT
The proposed fix serves only to disable exec-shield data-space
protection for 32-bit binaries.  It does not solve the real problem.

A little digging reveals that the faulting PC is in the heap.  My
guess is that the JVM is generating code in the heap then jumping to
it.  With exec-shield in place, by default code cannot be executed out
of data regions (e.g. heap, stack).  In order to do so, one must use
mprotect() to enable PROT_EXEC on the affected pages.  This is really
a problem in the JVM.

As a workaround, on x86_64 one can allow code to execute in data
segments for 32-bit programs only via:

     "echo 1 > /proc/sys/kernel/exec-shield32"

Comment 3 H.J. Lu 2004-08-21 09:45:54 EDT
It sounds odd. 64bit trampoline is also on the heap and kernel allows

        ((current->thread.flags & THREAD_IA32) ? \
         (vm_data_default_flags32 & ~((current->flags & 

Any particular reason for kernel to allow execution on heap for 64bit
and disallow for 32bit?
Comment 4 Jim Paradis 2004-08-30 23:48:02 EDT
Actually, we *should* be enforcing the heap-no-exec on the 64bit side
as well; that you can do it is a bug :(
Comment 5 H.J. Lu 2004-08-31 14:55:45 EDT
There are 2 problems here:

1. The same 32bit application works on ia32, but fails on EM64T.
2. The same application works in 64bit, but fails in 32bit.
Comment 6 H.J. Lu 2004-11-22 15:04:18 EST
FYI, 2.6.9-1.730_EL kernel works fine. Can it be backported to 2.4
Comment 7 Ernie Petrides 2004-12-15 16:49:09 EST
H.J., could you please retest this with the latest U4 beta kernel?


Please let us know whether this bug report is fully resolved in U4 (which
is scheduled for release on Monday, 12/20).

If not, please tell us exactly which kernel RPM was installed on the EM64T
machine when you generated a failure.  And also, please indicate whether the
command "objdump -x <executable_image_name> | grep STACK" provides any output.

Thanks in advance.  -ernie
Comment 8 H.J. Lu 2004-12-15 17:25:36 EST
I tried 2.4.21-27.EL. The small testcase didn't work.

[hjl@gnu-64 java-1]$ LD_LIBRARY_PATH=/usr/gcc-3.4/lib ./foo
[hjl@gnu-64 java-1]$ LD_LIBRARY_PATH=/usr/gcc-3.4/lib ldd ./foo
        libgcc_s.so.1 => /usr/gcc-3.4/lib/libgcc_s.so.1 (0x40017000)
        libgcj.so.5 => /usr/gcc-3.4/lib/libgcj.so.5 (0x40020000)
        libm.so.6 => /lib/tls/libm.so.6 (0x40953000)
        libpthread.so.0 => /lib/tls/libpthread.so.0 (0x40975000)
        libz.so.1 => /usr/lib/libz.so.1 (0x40985000)
        libdl.so.2 => /lib/libdl.so.2 (0x40993000)
        libc.so.6 => /lib/tls/libc.so.6 (0x40996000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
[hjl@gnu-64 java-1]$ objdump  -x /usr/gcc-3.4/lib/libgcj.so.5 | grep STACK
   STACK off    0x00000000 vaddr 0x00000000 paddr 0x00000000 align 2**2
[hjl@gnu-64 java-1]$ readelf -l /usr/gcc-3.4/lib/libgcj.so.5

Elf file type is DYN (Shared object file)
Entry point 0x39e780
There are 5 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000000 0x00000000 0x00000000 0x6ea034 0x6ea034 R E 0x1000
  LOAD           0x6ea040 0x006eb040 0x006eb040 0x218d3c 0x2377c0 RW  0x1000
  DYNAMIC        0x8f7180 0x008f8180 0x008f8180 0x000e8 0x000e8 RW  0x4
  GNU_EH_FRAME   0x6c6ed0 0x006c6ed0 0x006c6ed0 0x23164 0x23164 R   0x4
  GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RWE 0x4

 Section to Segment mapping:
  Segment Sections...
00     .hash .dynsym .dynstr .gnu.version .gnu.version_r .rel.dyn .rel.plt .init
 .plt .text .fini .rodata .eh_frame_hdr
01     .data .eh_frame .gcc_except_table .dynamic .ctors .dtors .jcr .got .bss
   02     .dynamic
   03     .eh_frame_hdr
[hjl@gnu-64 java-1]$ uname -a
Linux gnu-64.sc.intel.com 2.4.21-27.EL #1 Wed Dec 1 22:07:17 EST 2004 x86_64 
x86_64 x86_64 GNU/Linux
Comment 9 Ernie Petrides 2004-12-15 19:05:04 EST
Thanks, H.J.  I'm inferring from the "uname -a" output that you installed
kernel-2.4.21-27.EL.ia32e.rpm (if this isn't true, please correct me).

So, I think this is a case where there's a bug in the JVM (in that it does
not specify PROT_EXEC for a region it needs to execute code from), and the
binary does not qualify as a "legacy" app (and thus PF_RELOCEXEC is set due
to the existence of a PT_GNU_STACK elf header), and so the no-exec feature
of the cpu correctly prevents execution.

One possible work-around is to boot the kernel with "noexec32=off".  But
I think the bottom line on this is that the kernel is working correctly.

Jim, if you agree, please close this as NOTABUG.
Comment 10 H.J. Lu 2004-12-15 20:55:08 EST
However, the same 32bit binary works fine under RHEL 4 RC and the 64bit binary
compiled from the same source works fine under RHEL 3 U4. The binary itself
doesn't have the `E' bit in PT_GNU_STACK. But the shared library which depends
on has. It has to be a bug somewhere.
Comment 11 Jim Paradis 2004-12-17 11:17:27 EST
I seem to remember a similar issue from sometime back regarding inheritance and
overriding of execute permissions between executable and shared library...
Comment 12 jordan hargrave 2004-12-17 13:56:11 EST
Sun has released a new version of the JDK .. 1.5.0\

We saw an issue with 1.4.x on RHEL3, U3 (but with x86 mode) that was fixed with 
the new JVM.
Comment 14 Red Hat Bugzilla 2007-03-18 18:22:53 EDT
User jparadis@redhat.com's account has been closed
Comment 15 John Feeney 2007-07-12 13:33:59 EDT
With RHEL3 now in maintenance mode, where only critical customer issues
can be fixed, this bugzilla has been closed as wont fix due to its severity
level and a lack of recent activity. 

Note You need to log in before you can comment on or make changes to this bug.