+++ This bug was initially created as a clone of Bug #1199041 +++

The test-suite in libcrypt fails when the library is built with -O2 on big endian arches (ppc64, s390x), but passes on eg. ppc64le. And because they pass when -O1 is used, I suspect gcc produces incorrect machine code.

Version-Release number of selected component (if applicable):
gcc-5.0.0-0.18.fc22.ppc64
gcc-5.0.0-0.18.fc22.s390x

libgcrypt-1.6.2-4.fc23


see eg http://s390.koji.fedoraproject.org/koji/taskinfo?taskID=1749932 for full build.log, but it was retested locally on F-22 s390x and ppc64 with the latest gcc.

--- Additional comment from Dan Horák on 2015-03-05 06:42:37 EST ---

from build.log

...
make  check-TESTS
make[2]: Entering directory '/builddir/build/BUILD/libgcrypt-1.6.2/tests'
version:1.6.2:
ciphers:arcfour:blowfish:cast5:des:aes:twofish:serpent:rfc2268:seed:camellia:idea:salsa20:gost28147:
pubkeys:dsa:elgamal:rsa:ecc:
digests:crc:gostr3411-94:md4:md5:rmd160:sha1:sha256:sha512:tiger:whirlpool:stribog:
rnd-mod:linux:
cpu-arch::
mpi-asm:generic/mpih-add1.c:generic/mpih-sub1.c:generic/mpih-mul1.c:generic/mpih-mul2.c:generic/mpih-mul3.c:generic/mpih-lshift.c:generic/mpih-rshift.c:
threads:none:
hwflist:
fips-mode:n:n:
rng-type:standard:1:
PASS: version
PASS: mpitests
PASS: tsexp
PASS: t-convert
PASS: t-mpi-bit
PASS: t-mpi-point
PASS: curves
PASS: t-lock
PASS: prime
RFC2268 selftest failed (RFC2268 encryption test 1 failed.).
pass 0, algo 307, mode 1, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 307, mode 2, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 307, mode 5, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 307, mode 3, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 307, mode 3, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 307, mode 6, gcry_cipher_setkey failed: Selftest failed
FAIL: basic
PASS: keygen
...

--- Additional comment from Dan Horák on 2015-03-05 18:18:39 EST ---

when working with upstream (http://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgcrypt.git;a=summary)

[sharkcz@tyan-openpower-01 libgcrypt]$ ./tests/basic 
selftest for CFB failed - see syslog for details
pass 0, algo 4, mode 1, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 4, mode 2, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 4, mode 5, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 4, mode 3, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 4, mode 3, gcry_cipher_setkey failed: Selftest failed
pass 0, algo 4, mode 6, gcry_cipher_setkey failed: Selftest failed

syslog/journal contains
Libgcrypt warning: BLOWFISH-CFB-64 test failed (plaintext mismatch)

it seems when functions do_encrypt() and do_encrypt_block() (http://git.gnupg.org/cgi-bin/gitweb.cgi?p=libgcrypt.git;a=blob_plain;f=cipher/blowfish.c;hb=HEAD) both have __attribute__((optimize("O0"))) then the test passes

More info tomorrow.

--- Additional comment from Dan Horák on 2015-03-06 03:39:09 EST ---



--- Additional comment from Marek Polacek on 2015-03-06 18:41:00 EST ---

For the ppc64 issue (comment 2) it appears the code is violating C aliasing rules.  With -fno-strict-aliasing the test passes; adding __attribute__ ((optimize("no-strict-aliasing"))) to buf_xor_n_copy helps as well.  I think buf_xor_n_copy_2 accesses "unsigned char *" as a "uintptr_t *".

--- Additional comment from Jakub Jelinek on 2015-03-12 05:56:27 EDT ---

So, I had a quick look at the s390x tests/basic issue, and bisected that to
http://gcc.gnu.org/r220249 - miscompiled? source is cipher/rfc2268.c in that case.

--- Additional comment from Dan Horák on 2015-03-12 06:55:40 EDT ---

(In reply to Marek Polacek from comment #4)
> For the ppc64 issue (comment 2) it appears the code is violating C aliasing
> rules.  With -fno-strict-aliasing the test passes; adding __attribute__
> ((optimize("no-strict-aliasing"))) to buf_xor_n_copy helps as well.  I think
> buf_xor_n_copy_2 accesses "unsigned char *" as a "uintptr_t *".

thanks for looking, Marek, I will clone this bug for libgcrypt itself