Bug 1274707 - tar when used with -z segfaults under high CPU load
tar when used with -z segfaults under high CPU load
Status: CLOSED INSUFFICIENT_DATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: tar (Show other bugs)
6.6
x86_64 Linux
unspecified Severity low
: rc
: ---
Assigned To: Pavel Raiskup
qe-baseos-daemons
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-23 08:11 EDT by Freddy Wissing
Modified: 2015-12-09 10:48 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-12-09 10:48:39 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
abrt core of tar (13.21 MB, application/x-gzip)
2015-10-23 09:47 EDT, Freddy Wissing
no flags Details
core dump of tar without LD_LIBRARY_PATH env (7.56 MB, application/x-gzip)
2015-11-02 09:45 EST, Freddy Wissing
no flags Details

  None (edit)
Description Freddy Wissing 2015-10-23 08:11:37 EDT
Description of problem:

The tar command, versions  2:1.23-11.el6 and 2:1.23-13.el6 fail with SEGFAULT when run under high CPU load with the -z (gzip) flag.


Version-Release number of selected component (if applicable):

2:1.23-11.el6 and 2:1.23-13.el6

How reproducible:

The customer can reproduce this reliably.


Steps to Reproduce:

1) Create a simple tar.gz file: 
echo "THIS IS A TEST" > test.txt; tar -cvzf test.txt.tar.gz test.txt

2) Create high CPU scenario: (Session-1)
fulload() { dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null & }; fulload; read; killall dd

3a) Run tar with the un-gunzip option 10 times and there will be sporadic errors: (Session-2)
cp test.txt.tar.gz  copy.test.txt.tar.gz ; tar vxzf copy.test.txt.tar.gz 
  
Segmentation fault (core dumped)

3b) Running gunzip then tar does not generate errors:  
cp test.txt.tar.gz copy.test.txt.tar.gz; gunzip -df copy.test.txt.tar.gz; tar vxf copy.test.txt.tar


Actual results:
Segmentation fault (core dumped)


Expected results:

List of extracted files 

Additional info:

I have not been able to reproduce this myself, but the customer reports being able to do so with both tar versions.
Comment 2 Pavel Raiskup 2015-10-23 09:07:15 EDT
(In reply to Freddy Wissing from comment #0)
> The customer can reproduce this reliably.
...
> I have not been able to reproduce this myself, but the customer reports
> being able to do so with both tar versions.

How can I help here then?  Its hard to guess -- at least in libvirt
single/multi-core environment I'm not able to reproduce either.

Why do you think tar command segfaults and not something else?

Btw., tar executes 'gzip -d' which is not completely the same as 'gunzip'.
Why do not we have coredump attached?
Comment 3 Freddy Wissing 2015-10-23 09:47 EDT
Created attachment 1085850 [details]
abrt core of tar

I have attached a core file.  

# cat reason 
Process /bin/tar was killed by signal 11 (SIGSEGV)
Comment 4 Pavel Raiskup 2015-10-23 10:28:59 EDT
I can't get relevant backtrace from the coredump.  Are you able to communicate
with customer to get that?

Btw., customer is not using up to date kernel and maybe other packages:

$ cat dso_list 
/opt/quest/lib64/libvtsmartcache.so.1.0.0 vasclnt-4.0.3-222.x86_64 (Quest Software, Inc.) 1428157387
/lib64/ld-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/lib64/libacl.so.1.1.0 libacl-2.2.49-6.el6.x86_64 (Red Hat, Inc.) 1412507476
/usr/lib/locale/locale-archive glibc-common-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154833
/lib64/libattr.so.1.1.0 libattr-2.4.44-7.el6.x86_64 (Red Hat, Inc.) 1412507474
/lib64/libfreebl3.so nss-softokn-freebl-3.14.3-18.el6_6.x86_64 (Red Hat, Inc.) 1428154828
/lib64/libresolv-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/opt/quest/lib64/libvtutil.so.1.0.0 vasclnt-4.0.3-222.x86_64 (Quest Software, Inc.) 1428157387
/lib64/librt-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/lib64/libnss_files-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/lib64/libc-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/lib64/libcrypt-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/lib64/libselinux.so.1 libselinux-2.0.94-5.8.el6.x86_64 (Red Hat, Inc.) 1428154842
/lib64/libdl-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
/opt/quest/lib64/libvtcacheipc.so.1.0.0 vasclnt-4.0.3-222.x86_64 (Quest Software, Inc.) 1428157387
/opt/quest/lib64/nss/libnss_vas4.so.2 vasclnt-4.0.3-222.x86_64 (Quest Software, Inc.) 1428157387
/bin/tar tar-2:1.23-13.el6.x86_64 (Red Hat, Inc.) 1445343026
/lib64/libpthread-2.12.so glibc-2.12-1.149.el6_6.5.x86_64 (Red Hat, Inc.) 1428154836
$ cat kernel 
2.6.32-504.3.3.el6.x86_64

Those DSOs from '/opt' look suspicious.

$ cat environ | grep PATH
LD_LIBRARY_PATH=/sybase/15.7/DataAccess64/ODBC/lib:/sybase/15.7/DataAccess/ODBC/lib:/sybase/15.7/OCS-15_0/lib:/sybase/15.7/OCS-15_0/lib3p64:/sybase/15.7/OCS-15_0/lib3p:/:/opt/cyberfusion/libs:/opt/CA/SharedComponents/lib:/opt/CA/CAlib:/usr/lib:/opt/CA/SharedComponents/Csam/SockAdapter/lib
FIRMBASE_LOGPATH=/firmqa/firmchg/logs/current
PATH=/sybase/15.7/OCS-15_0/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/CyberFusion:/opt/cyberfusion:/etc:/usr/bin/X11:/usr/local/share/curl:/firmqa/firm/scripts:/firmqa/qrm/bin:/firmqa/qrm/etc:.:/usr/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/cyberfusion:/cyberfusion:/opt/CA/SharedComponents/bin
CLASSPATH=:/firmqa/firm/java/classes:/firmqa/firm/java/lib/sqljdbc4.jar:/firmqa/firm/java/lib/commons-io-2.1.jar:/firmqa/firm/java/lib/commons-net-3.1.jar:/firmqa/firm/java/lib/mail.jar:/sybase/15.7/jConnect-7_0/classes/jconn4.jar
FIRMBASE_INIPATH=/firmqa/firmchg/ini

We need a bit more cleaned environment probably.  Freddy, is this notabug?
Comment 8 Freddy Wissing 2015-10-26 13:40:53 EDT
They advise they were able to reproduce without an LD_LIBRARY_PATH.  

I have requested they upload the core to the case.  I will attach it here when it is received.
Comment 10 Freddy Wissing 2015-11-02 09:45 EST
Created attachment 1088604 [details]
core dump of tar without LD_LIBRARY_PATH env

Note You need to log in before you can comment on or make changes to this bug.