Bug 1326024

Summary: ECC key generation under valgrind reports: impossible has happened
Product: [Fedora] Fedora Reporter: Nikos Mavrogiannopoulos <nmavrogi>
Component: opensslAssignee: Tomas Mraz <tmraz>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-05-02 07:58:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Nikos Mavrogiannopoulos 2016-04-11 15:33:50 UTC 
A part of the gnutls test suite runs under valgrind, and in fedora 23 it occasionally fails, something that I have traced it to softhsm library and particular hardware.

The failures are due to the softhsm library which uses openssl underneath and in particular EC key generation. This can be reproduced using only the openssl tools:

$ valgrind openssl genpkey -algorithm ec  -pkeyopt ec_paramgen_curve:P-256 

vex: the `impossible' happened:
   isZeroU
vex storage: T total 270175784 bytes allocated
vex storage: P total 640 bytes allocated

valgrind: the 'impossible' happened:
   LibVEX called failure_exit().

host stacktrace:
==28037==    at 0x38083FA8: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x380840C4: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x38084301: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x3808432A: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x3809F842: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x38148388: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x381554ED: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x38159612: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x3815A246: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x3815C158: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x3815D1A6: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x3814618C: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x380A1DCB: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x380D2C6B: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x380D48CF: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)
==28037==    by 0x380E3C46: ??? (in /usr/lib64/valgrind/memcheck-amd64-linux)

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 28037)
==28037==    at 0x5B14C80: ecp_nistz256_avx2_select_w7 (in /usr/lib64/libcrypto.so.1.0.2g)
==28037==    by 0x5AFA92F: EC_POINT_mul (in /usr/lib64/libcrypto.so.1.0.2g)
==28037==    by 0x5AF9876: EC_POINT_new (in /usr/lib64/libcrypto.so.1.0.2g)
==28037==    by 0x5B02835: EC_KEY_generate_key (in /usr/lib64/libcrypto.so.1.0.2g)
==28037==    by 0x5B51B82: EVP_PKEY_keygen (in /usr/lib64/libcrypto.so.1.0.2g)
==28037==    by 0x436A29: ??? (in /usr/bin/openssl)
==28037==    by 0x41A457: ??? (in /usr/bin/openssl)
==28037==    by 0x41A0D6: ??? (in /usr/bin/openssl)
==28037==    by 0x62B357F: (below main) (libc-start.c:289)



The failed system is:
model name	: Intel(R) Core(TM) i7-5600U CPU @ 2.60GHz
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch ida arat epb pln pts dtherm intel_pt tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt
Comment 1 Tomas Mraz 2016-04-11 15:44:11 UTC 
If impossible has happened I would start with asking valgrind maintainer how it is possible that impossible happens.

Or perhaps report the bug into upstream OpenSSL RT which I can do but I am really unable to debug the issue much further.
Comment 2 Nikos Mavrogiannopoulos 2016-04-12 07:10:06 UTC 
No idea either, in a post in the valgrind ML it was suggested that this error is due to an invalid instruction. Most likely this has to go upstream.
Comment 3 Nikos Mavrogiannopoulos 2016-05-02 07:58:46 UTC 
According to upstream this is most likely an issue at the valgrind instruction parsing. I've addressed the issue by disabling openssl CPU-specific optimizations in the test suite so this bug is no longer of concern.
Comment 4 Nikos Mavrogiannopoulos 2016-06-16 14:03:00 UTC 
For the record the issue was worked around by exporting OPENSSL_ia32cap=0x00000000