Bug 720176

Summary: Segmentation fault in dynamic loader on AVX enabled CPU
Product: [Fedora] Fedora Reporter: MK <mk>
Component: glibcAssignee: Andreas Schwab <schwab>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 14CC: fweimer, jakub, schwab
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: glibc-2.14.90-8 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 790399 (view as bug list) Environment:
Last Closed: 2011-09-13 06:12:04 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
test case for SIGILL none

Description MK 2011-07-10 16:01:38 UTC
Description of problem:

A problem reported WRT glibc 2.12 Fedora 13 on AVX enabled hardware:

http://sourceware.org/bugzilla/show_bug.cgi?format=multiple&id=12113

Seems to also affect F13's glibc 2.13.  I have attached the "small reproducer" which was attached to the other bug report, as it does reproduce the issue.

I ran across this after my openVZ VPS was migrated to Xeon "Sandy Bridge" hardware and apache mod_perl applications crashed.  Here are the first five contexts from running apache in gdb after triggering the problem.

Program received signal SIGILL, Illegal instruction.
_dl_x86_64_save_sse () at ../sysdeps/x86_64/dl-trampoline.S:189
189		vmovdqa %ymm0, %fs:RTLD_SAVESPACE_SSE+0*YMM_SIZE
(gdb) bt
#0  _dl_x86_64_save_sse () at ../sysdeps/x86_64/dl-trampoline.S:189
#1  0x00002aaaaad0f3e0 in add_dependency (undef_name=0x2aaabc349503 "Perl_Istack_sp_ptr", undef_map=0x2aaab612d230, 
    ref=0x7fffffffd4d0, symbol_scope=0x2aaab612d588, version=0x0, type_class=1, flags=1, skip_map=0x0)
    at dl-lookup.c:628
#2  _dl_lookup_symbol_x (undef_name=0x2aaabc349503 "Perl_Istack_sp_ptr", undef_map=0x2aaab612d230, ref=0x7fffffffd4d0, 
    symbol_scope=0x2aaab612d588, version=0x0, type_class=1, flags=1, skip_map=0x0) at dl-lookup.c:831
#3  0x00002aaaaad12220 in _dl_fixup (l=<value optimized out>, reloc_arg=<value optimized out>)
    at ../elf/dl-runtime.c:118
#4  0x00002aaaaad18db5 in _dl_runtime_resolve () at ../sysdeps/x86_64/dl-trampoline.S:41
#5  0x00002aaabc349b16 in boot_Apache2__Const (my_perl=0x2aaab60425f0, cv=<value optimized out>) at Const.c:89

Which is identical to the initial backtrace posted by Vitaly Slobodskoy on the sourceware.org report.  According to wikipedia, these new "Sandy Bridge" Xeons use the AVX extensions.


Version-Release number of selected component (if applicable):

glibc 2.13


How reproducible:

1) Execute simple mod_perl module via apache using Apache2::Const.
OR
2) Compile and run attachment glibc_avx.zip.

Actual results from #2:

Hello, World!
Illegal instruction


Expected results:
Hello, World!
Hello, World!

Comment 1 MK 2011-07-10 16:03:42 UTC
(In reply to comment #0)
> Seems to also affect F13's glibc 2.13.  I have attached the "small reproducer"
> which was attached to the other bug report, as it does reproduce the issue.

Sorry, that should read "F14", not "F13".

Comment 2 MK 2011-07-10 16:06:54 UTC
Created attachment 512095 [details]
test case for SIGILL

Comment 3 MK 2011-07-10 16:08:42 UTC
(In reply to comment #0)
> 2) Compile and run attachment glibc_avx.zip.

Sorry, the .zip referred to did not attach with the first message.  The attachment labelled "test case for SIGILL" is a tar.gz containing the same code.

Comment 4 Andreas Schwab 2011-07-11 08:51:25 UTC
Please provide /proc/cpuinfo.

Comment 5 MK 2011-07-11 13:10:46 UTC
(In reply to comment #4)
> Please provide /proc/cpuinfo.

This is the first of four identical cores:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      :           Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
stepping        : 7
cpu MHz         : 3192.925
cache size      : 8192 KB
physical id     : 0
siblings        : 8
core id         : 0
cpu cores       : 4
apicid          : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse s
se2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc arat pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr ss
e4_1 sse4_2 popcnt lahf_lm
bogomips        : 6385.85
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management: [8]

Comment 6 Andreas Schwab 2011-07-11 13:26:54 UTC
It doesn't look like the cpu supports AVX.

Comment 7 MK 2011-07-11 15:51:14 UTC
Okay, although the E31230 is listed here:

http://en.wikipedia.org/wiki/Sandy_Bridge#Server_processors
http://en.wikipedia.org/wiki/Advanced_Vector_Extensions#CPUs_with_AVX

As supporting AVX.  But I have no further confirmation of that.  The provider definitely seems to believe the new hardware uses "Sandy Bridge", etc.

Anyway, presuming that it actually does *not* support AVX, or that the kernel does not consider it so, what could cause this problem?  This is on an openVZ slice, so I did not/cannot compile the kernel (2.6.32 built with gcc 4.1.2).  

Your patience is appreciated, and please excuse my ignorance here.  This does not look like my mistake. Is there something else that could produce the identical backtrace and cause that "Hello World" test to receive a SIGILL?

Comment 8 Andreas Schwab 2011-07-12 06:26:01 UTC
What is "openVZ slice"?

Comment 9 MK 2011-07-12 12:26:17 UTC
(In reply to comment #8)
> What is "openVZ slice"?

OpenVZ is akin to Xen; it's a method for "container-based virtualization".  Ie, the system in question is a VPS (virtual private server) "slice".  I have never used the openVZ software myself, but I do work on small remote servers that depend on it or Xen.

http://wiki.openvz.org/Main_Page

Since the kernel used may be a factor, I'll get on the openVZ mail list and see if anyone has any thoughts about possible causes, and perhaps the kernel devel list too.

Comment 10 Fedora Update System 2011-08-15 10:58:13 UTC
glibc-2.14.90-5 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/glibc-2.14.90-5

Comment 11 Fedora Update System 2011-08-15 20:25:42 UTC
Package glibc-2.14.90-5:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing glibc-2.14.90-5'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/glibc-2.14.90-5
then log in and leave karma (feedback).

Comment 12 Fedora Update System 2011-08-24 15:27:33 UTC
glibc-2.14.90-6 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/glibc-2.14.90-6

Comment 13 MK 2011-08-27 21:25:06 UTC
I upgraded to this glibc-2.14.90.5 on the avx hardware under openVZ, fedora 14, and the test case succeeded.  After recompiling apache + perl + mod_perl the original issue was also resolved.

Thanks much.

Comment 14 Fedora Update System 2011-09-02 07:19:16 UTC
glibc-2.14.90-7 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/glibc-2.14.90-7

Comment 15 Fedora Update System 2011-09-13 06:11:25 UTC
glibc-2.14.90-8 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.