Bug 847539

Summary: Repeatable core dump with --block-size of 2G or greater
Product: [Fedora] Fedora Reporter: Allan Engelhardt <allane>
Component: xzAssignee: Jindrich Novy <jnovy>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 17CC: a.badger, jnovy, pknirsch
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-09-21 23:56:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Allan Engelhardt 2012-08-12 18:57:10 UTC
Description of problem:
xz dumps core if using --block-size=2G or greater


Version-Release number of selected component (if applicable):
xz-5.1.1-2alpha.fc17.x86_64

How reproducible:
100%

Steps to Reproduce:
1. xz --compress --keep -9e --block-size=2G --threads=4 --verbose allane-bin.tar

  
Actual results:
*** glibc detected *** xz: double free or corruption (out): 0x00000000027bc9f0 ***
======= Backtrace: =========
/lib64/libc.so.6[0x34b767c7ce]
/lib64/liblzma.so.5[0x34c8207aac]
/lib64/liblzma.so.5[0x34c8208232]
/lib64/liblzma.so.5[0x34c8203359]
/lib64/liblzma.so.5(lzma_end+0x1b)[0x34c82036eb]
/lib64/liblzma.so.5(lzma_stream_encoder_mt+0x74)[0x34c82090f4]
xz[0x4044a5]
xz[0x40292f]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x34b7621735]
xz[0x402c15]
======= Memory map: ========
00400000-00410000 r-xp 00000000 fd:01 6333711                            /usr/bin/xz
0060f000-00610000 r--p 0000f000 fd:01 6333711                            /usr/bin/xz
00610000-00611000 rw-p 00010000 fd:01 6333711                            /usr/bin/xz
00611000-00615000 rw-p 00000000 00:00 0 
00810000-00811000 rw-p 00010000 fd:01 6333711                            /usr/bin/xz
027ba000-027db000 rw-p 00000000 00:00 0                                  [heap]
34b7200000-34b7220000 r-xp 00000000 fd:01 6296130                        /usr/lib64/ld-2.15.so
34b741f000-34b7420000 r--p 0001f000 fd:01 6296130                        /usr/lib64/ld-2.15.so
34b7420000-34b7421000 rw-p 00020000 fd:01 6296130                        /usr/lib64/ld-2.15.so
34b7421000-34b7422000 rw-p 00000000 00:00 0 
34b7600000-34b77ac000 r-xp 00000000 fd:01 6297262                        /usr/lib64/libc-2.15.so
34b77ac000-34b79ac000 ---p 001ac000 fd:01 6297262                        /usr/lib64/libc-2.15.so
34b79ac000-34b79b0000 r--p 001ac000 fd:01 6297262                        /usr/lib64/libc-2.15.so
34b79b0000-34b79b2000 rw-p 001b0000 fd:01 6297262                        /usr/lib64/libc-2.15.so
34b79b2000-34b79b7000 rw-p 00000000 00:00 0 
34b7e00000-34b7e16000 r-xp 00000000 fd:01 6293169                        /usr/lib64/libpthread-2.15.so
34b7e16000-34b8016000 ---p 00016000 fd:01 6293169                        /usr/lib64/libpthread-2.15.so
34b8016000-34b8017000 r--p 00016000 fd:01 6293169                        /usr/lib64/libpthread-2.15.so
34b8017000-34b8018000 rw-p 00017000 fd:01 6293169                        /usr/lib64/libpthread-2.15.so
34b8018000-34b801c000 rw-p 00000000 00:00 0 
34b8a00000-34b8a07000 r-xp 00000000 fd:01 6333663                        /usr/lib64/librt-2.15.so
34b8a07000-34b8c06000 ---p 00007000 fd:01 6333663                        /usr/lib64/librt-2.15.so
34b8c06000-34b8c07000 r--p 00006000 fd:01 6333663                        /usr/lib64/librt-2.15.so
34b8c07000-34b8c08000 rw-p 00007000 fd:01 6333663                        /usr/lib64/librt-2.15.so
34b8e00000-34b8e15000 r-xp 00000000 fd:01 6311444                        /usr/lib64/libgcc_s-4.7.0-20120507.so.1
34b8e15000-34b9014000 ---p 00015000 fd:01 6311444                        /usr/lib64/libgcc_s-4.7.0-20120507.so.1
34b9014000-34b9015000 rw-p 00014000 fd:01 6311444                        /usr/lib64/libgcc_s-4.7.0-20120507.so.1
34c8200000-34c8223000 r-xp 00000000 fd:01 6334944                        /usr/lib64/liblzma.so.5.0.99
34c8223000-34c8423000 ---p 00023000 fd:01 6334944                        /usr/lib64/liblzma.so.5.0.99
34c8423000-34c8424000 r--p 00023000 fd:01 6334944                        /usr/lib64/liblzma.so.5.0.99
34c8424000-34c8425000 rw-p 00024000 fd:01 6334944                        /usr/lib64/liblzma.so.5.0.99
7fed94000000-7fed94021000 rw-p 00000000 00:00 0 
7fed94021000-7fed98000000 ---p 00000000 00:00 0 
7fed99bad000-7fed9ffda000 r--p 00000000 fd:01 6323297                    /usr/lib/locale/locale-archive
7fed9ffda000-7fed9ffde000 rw-p 00000000 00:00 0 
7feda0003000-7feda0005000 rw-p 00000000 00:00 0 
7fff9711f000-7fff97141000 rw-p 00000000 00:00 0                          [stack]
7fff971ff000-7fff97200000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)



Expected results:
Error message if this large a block size is not supported

Additional info:
ABRT complains as usual that it is an 'unusable backtrace'. Don't think I ever do not get that one. But since this one is easy to reproduce I thought I'd include it manually.

Comment 1 Jindrich Novy 2012-08-13 06:20:41 UTC
Thanks for reporting. May I ask what is the size of the file you are trying to compress? I'm getting this:

$ xz --compress --keep -9e --block-size=2G --threads=4 --verbose <file>
xz: bsd: Cannot allocate memory

with almost any file I try to compress in this way (having 8G of RAM).

Comment 2 Allan Engelhardt 2012-08-13 06:44:05 UTC
$ stat /var/backup/allane/allane-bin.tar
  File: `/var/backup/allane/allane-bin.tar'
  Size: 395512023040	Blocks: 772484504  IO Block: 4096   regular file
Device: fd08h/64776d	Inode: 12          Links: 1
Access: (0640/-rw-r-----)  Uid: ( 1000/  allane)   Gid: ( 1000/  allane)
Context: unconfined_u:object_r:file_t:s0
Access: 2012-08-12 19:07:34.802218071 +0100
Modify: 2012-08-12 19:07:21.251065908 +0100
Change: 2012-08-12 19:07:21.251065908 +0100
 Birth: -

$ head -n 2 /proc/meminfo 
MemTotal:       16435520 kB
MemFree:         4478664 kB

$ egrep -A 4 '^processor\s+:' /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
--
processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
--
processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
--
processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 23
model name	: Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz

Comment 3 Jindrich Novy 2012-08-13 18:21:08 UTC
Discussed this bug with upstream (Lasse Collin) and the problem (double-free) happens when the setting requires too much memory (28G in your case).

The following commit fixes it:
http://git.tukaani.org/?p=xz.git;a=commitdiff;h=ab50ae3ef40c81e5bf613905ca3fd636548b75e7

and the fix for it is a part of xz-5.1.2alpha which is already in f18 and newer. There shouldn't be any major breakage while updating to 5.1.2alpha so I will schedule an update.

Comment 4 Fedora Update System 2012-08-14 05:01:25 UTC
xz-5.1.2-1alpha.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/xz-5.1.2-1alpha.fc17

Comment 5 Fedora Update System 2012-08-14 09:21:46 UTC
Package xz-5.1.2-1alpha.fc17:
* should fix your issue,
* was pushed to the Fedora 17 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing xz-5.1.2-1alpha.fc17'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2012-11856/xz-5.1.2-1alpha.fc17
then log in and leave karma (feedback).

Comment 6 Allan Engelhardt 2012-08-16 07:49:03 UTC
All working now, thanks, and I added the karma to the update.

Comment 7 Fedora Update System 2012-09-03 07:17:13 UTC
perl-Compress-Raw-Lzma-2.052-2.fc17,xz-5.1.2-1alpha.fc17 has been submitted as an update for Fedora 17.
https://admin.fedoraproject.org/updates/perl-Compress-Raw-Lzma-2.052-2.fc17,xz-5.1.2-1alpha.fc17

Comment 8 Fedora Update System 2012-09-21 23:56:58 UTC
perl-Compress-Raw-Lzma-2.052-2.fc17, xz-5.1.2-1alpha.fc17 has been pushed to the Fedora 17 stable repository.  If problems still persist, please make note of it in this bug report.