Bug 1889414 - java-1.8.0-openjdk / rhel-8 / FIPS: TLS connections killed by exception in P11AEADCipher class
Summary: java-1.8.0-openjdk / rhel-8 / FIPS: TLS connections killed by exception in P1...
Keywords:
Status: ON_QA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: java-1.8.0-openjdk
Version: 8.3
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: 8.0
Assignee: Andrew John Hughes
QA Contact: OpenJDK QA
URL:
Whiteboard:
Depends On:
Blocks: 1889497 1889543
TreeView+ depends on / blocked
 
Reported: 2020-10-19 15:47 UTC by zzambers
Modified: 2020-11-25 07:31 UTC (History)
1 user (show)

Fixed In Version: java-1.8.0-openjdk-1.8.0.272.b10-4.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 1889497 1889543 (view as bug list)
Environment:
Last Closed:
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
openjdk bug system JDK-8236512 None None None 2020-10-21 01:14:03 UTC

Description zzambers 2020-10-19 15:47:40 UTC
TLS connection randomly fails by exception thrown by P11AEADCipher class (in pkcs11/FIPS mode). It was discovered by ssl-tests [1].

Affected versions:
java-1.8.0-openjdk-devel-1.8.0.272.b07-0.1.ea.el8_3 and higher

Steps to reproduce:
export JAVA_HOME=...
make ssl-tests TEST_PKCS11_FIPS=1 SSLTESTS_SSL_CONFIG_FILTER="SunJSSE,Default,TLSv1.2,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256" SSLTESTS_CUSTOM_JAVA_PARAMS="-Djdk.tls.ephemeralDHKeySize=2048"
...
java.lang.RuntimeException: wrong refCount value: -1
    at sun.security.pkcs11.NativeKeyHolder.releaseKeyID(P11Key.java:1310)
    at sun.security.pkcs11.P11Key.releaseKeyID(P11Key.java:152)
    at sun.security.pkcs11.P11AEADCipher.reset(P11AEADCipher.java:443)
    at sun.security.pkcs11.P11AEADCipher.implDoFinal(P11AEADCipher.java:708)
    at sun.security.pkcs11.P11AEADCipher.engineDoFinal(P11AEADCipher.java:538)
...
FAILED: SunJSSE/Default: TLSv1.2 + TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256


Where exception in finally block [2] (higher) actually hides PKCS11Exception thrown at:
  [1] sun.security.pkcs11.P11AEADCipher.cancelOperation (P11AEADCipher.java:344)
  [2] sun.security.pkcs11.P11AEADCipher.reset (P11AEADCipher.java:440)
  [3] sun.security.pkcs11.P11AEADCipher.ensureInitialized (P11AEADCipher.java:359)
  [4] sun.security.pkcs11.P11AEADCipher.implDoFinal (P11AEADCipher.java:639)
  [5] sun.security.pkcs11.P11AEADCipher.engineDoFinal (P11AEADCipher.java:538) hidde:
  [1] sun.security.pkcs11.P11AEADCipher.cancelOperation (P11AEADCipher.java:344)
  [2] sun.security.pkcs11.P11AEADCipher.reset (P11AEADCipher.java:440)
  [3] sun.security.pkcs11.P11AEADCipher.ensureInitialized (P11AEADCipher.java:359)
  [4] sun.security.pkcs11.P11AEADCipher.implDoFinal (P11AEADCipher.java:639)
  [5] sun.security.pkcs11.P11AEADCipher.engineDoFinal (P11AEADCipher.java:538)
  ... 


[1] https://github.com/zzambers/ssl-tests
[2] https://hg.openjdk.java.net/jdk8u/jdk8u/jdk/file/ebfae7ddcfc1/src/share/classes/sun/security/pkcs11/P11AEADCipher.java#l708


Note You need to log in before you can comment on or make changes to this bug.