Bug 710298 - intel_aes_decrypt_cbc_256 doesn't work correctly when input and output buffers are the same
Summary: intel_aes_decrypt_cbc_256 doesn't work correctly when input and output buffer...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: nss
Version: 6.2
Hardware: Unspecified
OS: Unspecified
urgent
medium
Target Milestone: rc
: ---
Assignee: Elio Maldonado Batiz
QA Contact: Aleš Mareček
URL:
Whiteboard:
Depends On:
Blocks: FIPS-140-Tracker-6.2 668887 716532
TreeView+ depends on / blocked
 
Reported: 2011-06-02 22:53 UTC by Elio Maldonado Batiz
Modified: 2011-12-06 12:10 UTC (History)
8 users (show)

Fixed In Version: nss-softokn-3.12.9-5.el6
Doc Type: Bug Fix
Doc Text:
Clone Of: 709517
Environment:
Last Closed: 2011-12-06 12:10:42 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Mozilla Foundation 661061 0 None None None Never
Red Hat Product Errata RHBA-2011:1584 0 normal SHIPPED_LIVE nspr, nss, nss-softokn and nss-util bug fix and enhancement update 2011-12-06 00:38:51 UTC

Description Elio Maldonado Batiz 2011-06-02 22:53:18 UTC
+++ This bug was initially created as a clone of Bug #709517 +++

Description of problem:
It looks as though intel_aes_decrypt_cbc_256() doesn't work right when the input and output buffers are the same, though this appears to be something that PK11_CipherOp (which eventually calls this function, on processors that support it) is expected to be able to handle.

Version-Release number of selected component (if applicable):
nss-softokn-freebl-3.12.10-1.fc15.x86_64

How reproducible:
Always, provided you're on a 64-bit processor with native AES instruction support (the "flags:" line in /proc/cpuinfo will include "aes").  I've tested this on an i5 processor and also had it checked on an i7.

Steps to Reproduce:
1. Run reproducer with the "-i" flag, which will cause it to use the same buffer for input and output and default to a 256-bit key.
  
Actual results:
"data mismatch"

Expected results:
"Encrypted/recovered 32 bytes."

Additional info:
While NSS itself doesn't seem to trigger this behavior, the nascent support for using NSS as the backend for krb5's libk5crypto does.

--- Additional comment from nalin on 2011-05-31 17:19:59 EDT ---

Created attachment 502120 [details]
test

Compile:
  gcc -o aest aest.c `pkg-config --cflags --libs nss`
Run:
  ./aest -i                          (uses in-place encryption, errors out)
  env NSS_DISABLE_HW_AES=1 ./aest -i (uses unoptimized implementation, no error)
  ./aest                             (doesn't do encryption in place, no error)

Comment 1 Elio Maldonado Batiz 2011-06-02 22:55:32 UTC
A patch for this bug has been approved upstream.

Comment 9 errata-xmlrpc 2011-12-06 12:10:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2011-1584.html


Note You need to log in before you can comment on or make changes to this bug.