Bug 114610

Summary: gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
Product: Red Hat Enterprise Linux 3 Reporter: Richard Chan <rc556677>
Component: gccAssignee: Jakub Jelinek <jakub>
Status: CLOSED ERRATA QA Contact:
Severity: medium Docs Contact:
Priority: medium    
Version: 3.0CC: jmorris
Target Milestone: ---   
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2004-05-12 03:40:21 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN
none
Test vectors header file
none
typedef header file none

Description Richard Chan 2004-01-30 03:50:11 UTC 
Description of problem:

gcc miscompiles sha256.c  with -O2 and -march=pentium3
-march=pentium4. All test vectors fail.



Version-Release number of selected component (if applicable):
3.2.3-24



How reproducible:


Steps to Reproduce:
1. Compile a linux 2.6 kernel with sha256 as module.
insmod tcrypt. Kernel is configured to use -O2 and
-march=pentium3 or -march=pentium4
2. All sha256 test vectors fail
3.  OR  just take the kernel file and compile
it in user space (delete all references to kernel headers).
Same thing - all sha256 test vectors fail.

Note: ok if using gcc3.3 or gcc3.2 from Fedora Core 1.
ok if using march=i686 or -O

  
Actual results:
gcc -O2 -march=pentium4 sha256.c -DMAIN -otsha256.bad

4BDC38C481DC38711BB54CE4F858F5C0B68E896D853598358E438A9C5BAC2E97
BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
33F031667B47FB810A22CFA42B5C23712E0C2B75BA7E13E28C635FDA3C8C3F7E
248D6A61D20638B8E5C026930C3E6039A33CE45964FF2167F6ECEDD419DB06C1

(odd lines are output, even lines are testvectors; there
are two test vectors here the first is the sha256 hash
or "abc", the second is the sha256 hash of
"abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq")

Expected results:
gcc -march=pentium4 sha256.c -DMAIN -otsha256.good

BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
248D6A61D20638B8E5C026930C3E6039A33CE45964FF2167F6ECEDD419DB06C1
248D6A61D20638B8E5C026930C3E6039A33CE45964FF2167F6ECEDD419DB06C1


Additional info:
Comment 1 Richard Chan 2004-01-30 03:53:00 UTC 
Created attachment 97356 [details]
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN

This file is taken from linux 2.6.2-rc2/crypto/sha256.c

I have edited it to be compilable in user space and included the test vectors
from tcrypt.h
Comment 2 Richard Chan 2004-01-30 03:53:34 UTC 
Created attachment 97357 [details]
Test vectors header file
Comment 3 Richard Chan 2004-01-30 03:54:02 UTC 
Created attachment 97358 [details]
typedef header file
Comment 4 Jakub Jelinek 2004-01-30 10:19:20 UTC 
I think http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03233.html
with http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03272.html
backported to gcc-3_2-rhl8-branch (and gcc-3_3-rhl-branch and gcc-3_4-branch)
should fix it, but I'd like to give that patch a week or two testing
on the mainline first.  Likely http://gcc.gnu.org/ml/gcc-patches/2003-04/msg01602.html
needs to be backported as well.

On the other side, sha256.c in the kernel should be fixed because it
is unnecessary inefficient.  Replacing const u8 padding[64] = { 0x80, };
with static const u8 padding[64] = { 0x80, }; should do the job.
Without that, padding needs to be memset'ed every time the routine is
run, not to mention that kernel should not waste precious kernel stack
space.
Comment 5 James Morris 2004-01-30 15:26:17 UTC 
Thanks for the info on the padding, I'll submit that upstream and also
look for any similar code.
Comment 6 Richard Chan 2004-02-04 09:37:18 UTC 
Source code patch (linux-kernel list) by jmorris does indeed workaround
the problem for this file. But I guess the real fix will have to wait
for the next 3ES update to the gcc package.
Comment 7 Jakub Jelinek 2004-02-16 21:42:02 UTC 
Should be fixed in gcc-3.2.3-28.
Comment 8 John Flanagan 2004-05-12 03:40:21 UTC 
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2004-142.html