Bug 114610 - gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
Summary: gcc 3.2.3-24 miscompiles sha256.c from linux 2.6 kernel
Alias: None
Product: Red Hat Enterprise Linux 3
Classification: Red Hat
Component: gcc   
(Show other bugs)
Version: 3.0
Hardware: i686 Linux
Target Milestone: ---
Assignee: Jakub Jelinek
QA Contact:
Depends On:
TreeView+ depends on / blocked
Reported: 2004-01-30 03:50 UTC by Richard Chan
Modified: 2007-11-30 22:07 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2004-05-12 03:40:21 UTC
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN (11.04 KB, text/plain)
2004-01-30 03:53 UTC, Richard Chan
no flags Details
Test vectors header file (53.71 KB, text/plain)
2004-01-30 03:53 UTC, Richard Chan
no flags Details
typedef header file (53 bytes, text/plain)
2004-01-30 03:54 UTC, Richard Chan
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2004:142 normal SHIPPED_LIVE GCC bugfix update 2004-05-11 04:00:00 UTC

Description Richard Chan 2004-01-30 03:50:11 UTC
Description of problem:

gcc miscompiles sha256.c  with -O2 and -march=pentium3
-march=pentium4. All test vectors fail.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. Compile a linux 2.6 kernel with sha256 as module.
insmod tcrypt. Kernel is configured to use -O2 and
-march=pentium3 or -march=pentium4
2. All sha256 test vectors fail
3.  OR  just take the kernel file and compile
it in user space (delete all references to kernel headers).
Same thing - all sha256 test vectors fail.

Note: ok if using gcc3.3 or gcc3.2 from Fedora Core 1.
ok if using march=i686 or -O

Actual results:
gcc -O2 -march=pentium4 sha256.c -DMAIN -otsha256.bad


(odd lines are output, even lines are testvectors; there
are two test vectors here the first is the sha256 hash
or "abc", the second is the sha256 hash of

Expected results:
gcc -march=pentium4 sha256.c -DMAIN -otsha256.good


Additional info:

Comment 1 Richard Chan 2004-01-30 03:53:00 UTC
Created attachment 97356 [details]
sha256.c from kernel 2.6.2-rc2 compilable in userspace with -DMAIN

This file is taken from linux 2.6.2-rc2/crypto/sha256.c

I have edited it to be compilable in user space and included the test vectors
from tcrypt.h

Comment 2 Richard Chan 2004-01-30 03:53:34 UTC
Created attachment 97357 [details]
Test vectors header file

Comment 3 Richard Chan 2004-01-30 03:54:02 UTC
Created attachment 97358 [details]
typedef header file

Comment 4 Jakub Jelinek 2004-01-30 10:19:20 UTC
I think http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03233.html
with http://gcc.gnu.org/ml/gcc-patches/2004-01/msg03272.html
backported to gcc-3_2-rhl8-branch (and gcc-3_3-rhl-branch and gcc-3_4-branch)
should fix it, but I'd like to give that patch a week or two testing
on the mainline first.  Likely http://gcc.gnu.org/ml/gcc-patches/2003-04/msg01602.html
needs to be backported as well.

On the other side, sha256.c in the kernel should be fixed because it
is unnecessary inefficient.  Replacing const u8 padding[64] = { 0x80, };
with static const u8 padding[64] = { 0x80, }; should do the job.
Without that, padding needs to be memset'ed every time the routine is
run, not to mention that kernel should not waste precious kernel stack

Comment 5 James Morris 2004-01-30 15:26:17 UTC
Thanks for the info on the padding, I'll submit that upstream and also
look for any similar code.

Comment 6 Richard Chan 2004-02-04 09:37:18 UTC
Source code patch (linux-kernel list) by jmorris does indeed workaround
the problem for this file. But I guess the real fix will have to wait
for the next 3ES update to the gcc package.

Comment 7 Jakub Jelinek 2004-02-16 21:42:02 UTC
Should be fixed in gcc-3.2.3-28.

Comment 8 John Flanagan 2004-05-12 03:40:21 UTC
An errata has been issued which should help the problem described in this bug report. 
This report is therefore being closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files, please follow the link below. You may reopen 
this bug report if the solution does not work for you.


Note You need to log in before you can comment on or make changes to this bug.