Bug 114610 - gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: gcc (Show other bugs)
3.0
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Jakub Jelinek
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2004-01-29 22:50 EST by Richard Chan
Modified: 2007-11-30 17:07 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2004-05-11 23:40:21 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN (11.04 KB, text/plain)
2004-01-29 22:53 EST, Richard Chan
no flags Details
Test vectors header file (53.71 KB, text/plain)
2004-01-29 22:53 EST, Richard Chan
no flags Details
typedef header file (53 bytes, text/plain)
2004-01-29 22:54 EST, Richard Chan
no flags Details

  None (edit)
Description Richard Chan 2004-01-29 22:50:11 EST
Description of problem:

gcc miscompiles sha256.c  with -O2 and -march=pentium3
-march=pentium4. All test vectors fail.



Version-Release number of selected component (if applicable):
3.2.3-24



How reproducible:


Steps to Reproduce:
1. Compile a linux 2.6 kernel with sha256 as module.
insmod tcrypt. Kernel is configured to use -O2 and
-march=pentium3 or -march=pentium4
2. All sha256 test vectors fail
3.  OR  just take the kernel file and compile
it in user space (delete all references to kernel headers).
Same thing - all sha256 test vectors fail.

Note: ok if using gcc3.3 or gcc3.2 from Fedora Core 1.
ok if using march=i686 or -O

  
Actual results:
gcc -O2 -march=pentium4 sha256.c -DMAIN -otsha256.bad

4BDC38C481DC38711BB54CE4F858F5C0B68E896D853598358E438A9C5BAC2E97
BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
33F031667B47FB810A22CFA42B5C23712E0C2B75BA7E13E28C635FDA3C8C3F7E
248D6A61D20638B8E5C026930C3E6039A33CE45964FF2167F6ECEDD419DB06C1

(odd lines are output, even lines are testvectors; there
are two test vectors here the first is the sha256 hash
or "abc", the second is the sha256 hash of
"abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq")

Expected results:
gcc -march=pentium4 sha256.c -DMAIN -otsha256.good

BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
248D6A61D20638B8E5C026930C3E6039A33CE45964FF2167F6ECEDD419DB06C1
248D6A61D20638B8E5C026930C3E6039A33CE45964FF2167F6ECEDD419DB06C1


Additional info:
Comment 1 Richard Chan 2004-01-29 22:53:00 EST
Created attachment 97356 [details]
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN

This file is taken from linux 2.6.2-rc2/crypto/sha256.c

I have edited it to be compilable in user space and included the test vectors
from tcrypt.h
Comment 2 Richard Chan 2004-01-29 22:53:34 EST
Created attachment 97357 [details]
Test vectors header file
Comment 3 Richard Chan 2004-01-29 22:54:02 EST
Created attachment 97358 [details]
typedef header file
Comment 4 Jakub Jelinek 2004-01-30 05:19:20 EST
I think http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03233.html
with http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03272.html
backported to gcc-3_2-rhl8-branch (and gcc-3_3-rhl-branch and gcc-3_4-branch)
should fix it, but I'd like to give that patch a week or two testing
on the mainline first.  Likely http://gcc.gnu.org/ml/gcc-patches/2003-04/msg01602.html
needs to be backported as well.

On the other side, sha256.c in the kernel should be fixed because it
is unnecessary inefficient.  Replacing const u8 padding[64] = { 0x80, };
with static const u8 padding[64] = { 0x80, }; should do the job.
Without that, padding needs to be memset'ed every time the routine is
run, not to mention that kernel should not waste precious kernel stack
space.
Comment 5 James Morris 2004-01-30 10:26:17 EST
Thanks for the info on the padding, I'll submit that upstream and also
look for any similar code.
Comment 6 Richard Chan 2004-02-04 04:37:18 EST
Source code patch (linux-kernel list) by jmorris does indeed workaround
the problem for this file. But I guess the real fix will have to wait
for the next 3ES update to the gcc package.
Comment 7 Jakub Jelinek 2004-02-16 16:42:02 EST
Should be fixed in gcc-3.2.3-28.
Comment 8 John Flanagan 2004-05-11 23:40:21 EDT
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2004-142.html

Note You need to log in before you can comment on or make changes to this bug.