Bug 1509169 (CVE-2017-3736)

There is a carry propagating bug in the x86_64 Montgomery squaring procedure. No
EC algorithms are affected. Analysis suggests that attacks against RSA and DSA
as a result of this defect would be very difficult to perform and are not
believed likely. Attacks against DH are considered just feasible (although very
difficult) because most of the work necessary to deduce information
about a private key may be performed offline. The amount of resources
required for such an attack would be very significant and likely only
accessible to a limited number of attackers. An attacker would
additionally need online access to an unpatched system using the target
private key in a scenario with persistent DH parameters and a private
key that is shared between multiple clients.

This only affects processors that support the BMI1, BMI2 and ADX extensions like
Intel Broadwell (5th generation) and later or AMD Ryzen.

Note: This issue is very similar to CVE-2017-3732 and CVE-2015-3193 but must be
treated as a separate problem.

External References:

https://www.openssl.org/news/secadv/20171102.txt