Bug 481682

Summary: linux-2.6-misc-utrace-update.patch contains incorrect optimization
Product: Red Hat Enterprise Linux 5 Reporter: Dmitry V. Levin <ldv>
Component: kernelAssignee: Vitaly Mayatskikh <vmayatsk>
Status: CLOSED ERRATA QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: medium Docs Contact:
Priority: low    
Version: 5.5CC: anton, jmarchan, kolyshkin, roland, vmayatsk
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
URL: http://bugzilla.openvz.org/show_bug.cgi?id=1150
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 484871 (view as bug list) Environment:
Last Closed: 2009-09-02 09:02:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 484871    
Attachments:
Description Flags
proposed fix
none
test case source
none
test case source
none
fix posted upstream none

Description Dmitry V. Levin 2009-01-27 00:58:37 UTC
Created attachment 330051 [details]
proposed fix

in kernel-2.6.18-128.el5 and prior versions, arch/x86_64/ia32/ia32entry.S hunk of the linux-2.6-misc-utrace-update.patch contains incorrect optimization.
As result, out-of-table 32-bit syscalls on the x86_64 kernel do not return ENOSYS (unless the caller is being ptraced).

For example, glibc-2.9+ popen() goes mad when pipe2 syscall returns its number 331 instead of fail with ENOSYS.
As result, FC10+ 32-bit processes on RHEL5 x86_64 kernel break once popen(3) is called.

Comment 1 Kirill Kolyshkin 2009-01-27 09:29:59 UTC
The issue was found while running Fedora 10 containers on an RHEL5+OpenVZ kernel.

Relative OpenVZ bug: http://bugzilla.openvz.org/show_bug.cgi?id=1150

Comment 2 Roland McGrath 2009-02-07 01:59:52 UTC
Created attachment 331182 [details]
test case source

Simple test case, compile with -m32 and run on x86-64 kernel.

The RHEL5 code is the same (broken) as upstream.  I'll fix it upstream and then RHEL5 should backport the change so it continues to match upstream.

Comment 3 Roland McGrath 2009-02-07 02:02:48 UTC
Created attachment 331183 [details]
test case source

Test case fixed to exit 0 for correct and nonzero for bug.

Also, I forgot to note that to reproduce you have to have auditd disabled:
/sbin/chkconfig auditd off; reboot

Comment 4 Roland McGrath 2009-02-07 02:34:13 UTC
Created attachment 331187 [details]
fix posted upstream

Wait to see if upstream takes this as is, but almost certain they will.
With s,/x86/,/x86_64/,g this same patch applies to RHEL5 fine.

Comment 5 Vitaly Mayatskikh 2009-02-07 09:57:47 UTC
> The RHEL5 code is the same (broken) as upstream.

RHEL5 is not broken, I wasn't able to reproduce it. As I understood,  this is due to this check:

        testl $(_TIF_SYSCALL_TRACE|_TIF_SYSCALL_AUDIT|_TIF_SECCOMP),threadinfo_flags(%r10)
        jnz ia32_tracesys

And we have TIF_SYSCALL_AUDIT set.

Also, it's possible to make the patch shorter:

--- a/arch/x86/ia32/ia32entry.S
+++ b/arch/x86/ia32/ia32entry.S
@@ -417,6 +417,7 @@ ENTRY(ia32_syscall)
        GET_THREAD_INFO(%r10)
        orl   $TS_COMPAT,TI_status(%r10)
        testl $_TIF_WORK_SYSCALL_ENTRY,TI_flags(%r10)
+       movq $-ENOSYS,RAX-ARGOFFSET(%rsp)
        jnz ia32_tracesys
 ia32_do_syscall:
        cmpl $(IA32_NR_syscalls-1),%eax

I've tested it yesterday, but didn't post upstream yet.

Comment 6 Roland McGrath 2009-02-07 22:21:41 UTC
It is broken.  See comment#3 on how to reproduce it.
Shorter patch does not mean shorter code path, which is what matters.
Anyway, I've already posted upstream.

Comment 8 Vitaly Mayatskikh 2009-02-07 22:30:45 UTC
(In reply to comment #6)
> It is broken.  See comment#3 on how to reproduce it.

Yeah, I forgot to reboot machine after disabling audit.

Comment 9 RHEL Program Management 2009-02-11 10:09:48 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 10 RHEL Program Management 2009-02-16 15:44:53 UTC
Updating PM score.

Comment 11 Don Zickus 2009-03-04 20:01:22 UTC
in kernel-2.6.18-133.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Please do NOT transition this bugzilla state to VERIFIED until our QE team
has sent specific instructions indicating when to do so.  However feel free
to provide a comment indicating that this fix has been verified.

Comment 14 errata-xmlrpc 2009-09-02 09:02:56 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-1243.html