The openssl-1.0.2k as of RHEL-7.4 contains sophisticated highly optimized code for ECDSA computations with NIST P256 curve. However when this code is run inside virtual machine created with qemu-system-x86_64 from current EPEL 7 on new AMD CPU, it produces erroneous results causing failure.

The failure can be observed with command such as:

openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:prime256v1
Error generating key
140093439174560:error:2D06D075:FIPS routines:fips_pkey_signature_test:test failure:fips_post.c:168:
140093439174560:error:2D08E06B:FIPS routines:FIPS_CHECK_EC:pairwise test failed:ec_key.c:249:

Note that the FIPS mentioned in the error message is misleading, the machine does not have to run in the FIPS mode.

Also cat /proc/cpuinfo shows some weird combination of CPU flags which is also suspicious.

processor	: 1
vendor_id	: AuthenticAMD
cpu family	: 6
model		: 6
model name	: QEMU Virtual CPU version 2.0.0
stepping	: 3
cpu MHz		: 2599.975
cache size	: 512 KB
physical id	: 1
siblings	: 1
core id		: 0
cpu cores	: 1
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 4
wp		: yes
flags		: fpu pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm art nopl pni cx16 popcnt hypervisor lahf_lm svm abm sse4a
bogomips	: 71786.41
TLB size	: 1024 4K pages
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


This issue cannot be reproduced on newer qemu-system such as the one from Fedora 26.