Bug 114610 - gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: gcc (Show other bugs)
i686 Linux
medium Severity medium
: ---
: ---
Assigned To: Jakub Jelinek
Depends On:
  Show dependency treegraph
Reported: 2004-01-29 22:50 EST by Richard Chan
Modified: 2007-11-30 17:07 EST (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2004-05-11 23:40:21 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN (11.04 KB, text/plain)
2004-01-29 22:53 EST, Richard Chan
no flags Details
Test vectors header file (53.71 KB, text/plain)
2004-01-29 22:53 EST, Richard Chan
no flags Details
typedef header file (53 bytes, text/plain)
2004-01-29 22:54 EST, Richard Chan
no flags Details

  None (edit)
Description Richard Chan 2004-01-29 22:50:11 EST
Description of problem:

gcc miscompiles sha256.c  with -O2 and -march=pentium3
-march=pentium4. All test vectors fail.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Compile a linux 2.6 kernel with sha256 as module.
insmod tcrypt. Kernel is configured to use -O2 and
-march=pentium3 or -march=pentium4
2. All sha256 test vectors fail
3.  OR  just take the kernel file and compile
it in user space (delete all references to kernel headers).
Same thing - all sha256 test vectors fail.

Note: ok if using gcc3.3 or gcc3.2 from Fedora Core 1.
ok if using march=i686 or -O

Actual results:
gcc -O2 -march=pentium4 sha256.c -DMAIN -otsha256.bad


(odd lines are output, even lines are testvectors; there
are two test vectors here the first is the sha256 hash
or "abc", the second is the sha256 hash of

Expected results:
gcc -march=pentium4 sha256.c -DMAIN -otsha256.good


Additional info:
Comment 1 Richard Chan 2004-01-29 22:53:00 EST
Created attachment 97356 [details]
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN

This file is taken from linux 2.6.2-rc2/crypto/sha256.c

I have edited it to be compilable in user space and included the test vectors
from tcrypt.h
Comment 2 Richard Chan 2004-01-29 22:53:34 EST
Created attachment 97357 [details]
Test vectors header file
Comment 3 Richard Chan 2004-01-29 22:54:02 EST
Created attachment 97358 [details]
typedef header file
Comment 4 Jakub Jelinek 2004-01-30 05:19:20 EST
I think http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03233.html
with http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03272.html
backported to gcc-3_2-rhl8-branch (and gcc-3_3-rhl-branch and gcc-3_4-branch)
should fix it, but I'd like to give that patch a week or two testing
on the mainline first.  Likely http://gcc.gnu.org/ml/gcc-patches/2003-04/msg01602.html
needs to be backported as well.

On the other side, sha256.c in the kernel should be fixed because it
is unnecessary inefficient.  Replacing const u8 padding[64] = { 0x80, };
with static const u8 padding[64] = { 0x80, }; should do the job.
Without that, padding needs to be memset'ed every time the routine is
run, not to mention that kernel should not waste precious kernel stack
Comment 5 James Morris 2004-01-30 10:26:17 EST
Thanks for the info on the padding, I'll submit that upstream and also
look for any similar code.
Comment 6 Richard Chan 2004-02-04 04:37:18 EST
Source code patch (linux-kernel list) by jmorris does indeed workaround
the problem for this file. But I guess the real fix will have to wait
for the next 3ES update to the gcc package.
Comment 7 Jakub Jelinek 2004-02-16 16:42:02 EST
Should be fixed in gcc-3.2.3-28.
Comment 8 John Flanagan 2004-05-11 23:40:21 EDT
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.