Bug 760657

Summary: uncompress fails to uncompress a file it compressed
Product: Red Hat Enterprise Linux 6 Reporter: daryl herzmann <akrherz>
Component: ncompressAssignee: Pavel Raiskup <praiskup>
Status: CLOSED CURRENTRELEASE QA Contact: qe-baseos-daemons
Severity: high Docs Contact:
Priority: urgent    
Version: 6.2CC: azelinka, kdudka, masahiko.ando.yv, mdarade, mfuruta, msvoboda, psklenar, rdassen, yoshihide.sonoda.ua
Target Milestone: rcKeywords: Regression, ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
The ncompress utility previously relied on the glibc implementation of the memcpy() function. A recent glibc update optimized memcpy(), which resulted in data corruption in ncompress file compression and decompression. This update replaces memcpy() with the memmove() function and ncompress now works as expected.
Story Points: ---
Clone Of:
: 821708 (view as bug list) Environment:
Last Closed: 2012-10-09 12:41:20 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 781973, 821708    

Description daryl herzmann 2011-12-06 17:51:38 UTC
Updated to RHEL6.2 and my uncompress command started to fail to uncompress a file it compressed in the first place.  Here is my reproducer:


$ wget http://mesonet.agron.iastate.edu/data/gis/images/900913/USCOMP/n0q_0.tif
$ compress n0q_0.tif 
$ uncompress n0q_0.tif 
insize:1036 posbits:169 inbuf:E4 20 B0 0E 3C (1)
uncompress: corrupt input
$ rpm -q ncompress
ncompress-4.2.4-54.el6.x86_64
$ ls -l n0q_0.tif.Z
-rw-rw-r--. 1 akrherz akrherz 2303871 Dec  6 11:43 n0q_0.tif.Z
$ file n0q_0.tif.Z
n0q_0.tif.Z: compress'd data 16 bits

64bit here, didn't try on 32bit.  Thanks

Comment 1 daryl herzmann 2011-12-06 17:52:41 UTC
sorry, my example dropped the Z

$ uncompress n0q_0.tif.Z 
insize:1036 posbits:169 inbuf:E4 20 B0 0E 3C (1)
uncompress: corrupt input

Comment 3 Ondrej Vasik 2011-12-07 07:56:52 UTC
Thanks for report, there was no update of ncompress package in RHEL-6.2, therefore probably related to something else. You mentioned that ncompress started to fail after 6.2 update - you updated from 6.1, right?

It works for me on 32-bit system (ok, not really 100% 6.2, but very close to it)... is it always reproducible for you (and/or is it reproducible with other files) ?

Comment 4 daryl herzmann 2011-12-07 13:48:20 UTC
I am reproducing it 100% on 64bit RHEL6.2, I don't have 32bit around to try it. Are you able to try it on 64bit?  Thanks for the help.

Comment 5 Ondrej Vasik 2011-12-07 13:54:52 UTC
Yes, no problem, I'll try it on 64bit, but it still will be useful to get answer for the questions from comment #3 prior that.

Comment 6 daryl herzmann 2011-12-07 14:02:52 UTC
Sorry, yes, each day I cron "yum -y update", so it was an updated 6.1 prior to the 6.2 rollout yesterday, when things broke.  It is not reproducible for all file types though.

Comment 8 daryl herzmann 2012-01-03 13:26:48 UTC
Why is this now a private bug?  please revert.

Comment 9 Kamil Dudka 2012-01-03 13:48:07 UTC
(In reply to comment #7)
> I see memory overlap during compress but when I do uncompress it is working
> fine.

Thanks for debugging it.  This is a clear bug in ncompress.  We have to use memmove() instead of memcpy() when shifting within outbuf.  I believe this will fix it:

--- a/compress42.c
+++ b/compress42.c
@@ -1441,7 +1441,7 @@ compress(fdin, fdout)
                    boff = -(((OBUFSIZ<<3)-boff)%(n_bits<<3));
                    bytes_out += OBUFSIZ;

-                   memcpy(outbuf, outbuf+OBUFSIZ, (outbits>>3)+1);
+                   memmove(outbuf, outbuf+OBUFSIZ, (outbits>>3)+1);
                    memset(outbuf+(outbits>>3)+1, '\0', OBUFSIZ);
                }

Comment 11 daryl herzmann 2012-01-04 13:16:04 UTC
So the bug is private again, why?

Comment 22 Miroslav Svoboda 2012-01-20 13:22:03 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The ncompress utility previously relied on the glibc implementation of the memcpy() function. A recent glibc update optimized memcpy(), which resulted in data corruption in ncompress file compression and decompression. This update replaces memcpy() with the memmove() function and ncompress now works as expected.

Comment 23 Jiri Pallich 2012-10-09 12:41:20 UTC
Since this is a parent bug of an issue that has already been released via Z-Stream (e.g. rhel-6.3.z), this bug is going to be CLOSED as CURRENTRELEASE.