Bug 709517 - intel_aes_decrypt_cbc_256 doesn't work correctly when input and output buffers are the same
Summary: intel_aes_decrypt_cbc_256 doesn't work correctly when input and output buffer...
Alias: None
Product: Fedora
Classification: Fedora
Component: nss-softokn
Version: 15
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
Assignee: Elio Maldonado Batiz
QA Contact: Fedora Extras Quality Assurance
Depends On:
TreeView+ depends on / blocked
Reported: 2011-05-31 21:15 UTC by Nalin Dahyabhai
Modified: 2011-06-26 18:48 UTC (History)
3 users (show)

Fixed In Version: nss-softokn-3.12.10-2.fc15
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 710298 (view as bug list)
Last Closed: 2011-06-26 18:48:14 UTC

Attachments (Terms of Use)
test (4.06 KB, text/plain)
2011-05-31 21:19 UTC, Nalin Dahyabhai
no flags Details

System ID Priority Status Summary Last Updated
Mozilla Foundation 661061 None None None Never

Description Nalin Dahyabhai 2011-05-31 21:15:54 UTC
Description of problem:
It looks as though intel_aes_decrypt_cbc_256() doesn't work right when the input and output buffers are the same, though this appears to be something that PK11_CipherOp (which eventually calls this function, on processors that support it) is expected to be able to handle.

Version-Release number of selected component (if applicable):

How reproducible:
Always, provided you're on a 64-bit processor with native AES instruction support (the "flags:" line in /proc/cpuinfo will include "aes").  I've tested this on an i5 processor and also had it checked on an i7.

Steps to Reproduce:
1. Run reproducer with the "-i" flag, which will cause it to use the same buffer for input and output and default to a 256-bit key.
Actual results:
"data mismatch"

Expected results:
"Encrypted/recovered 32 bytes."

Additional info:
While NSS itself doesn't seem to trigger this behavior, the nascent support for using NSS as the backend for krb5's libk5crypto does.

Comment 1 Nalin Dahyabhai 2011-05-31 21:19:59 UTC
Created attachment 502120 [details]

  gcc -o aest aest.c `pkg-config --cflags --libs nss`
  ./aest -i                          (uses in-place encryption, errors out)
  env NSS_DISABLE_HW_AES=1 ./aest -i (uses unoptimized implementation, no error)
  ./aest                             (doesn't do encryption in place, no error)

Comment 2 Fedora Update System 2011-06-22 23:20:57 UTC
nss-softokn-3.12.10-2.fc15 has been submitted as an update for Fedora 15.

Comment 3 Fedora Update System 2011-06-24 03:29:41 UTC
Package nss-softokn-3.12.10-2.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing nss-softokn-3.12.10-2.fc15'
as soon as you are able to.
Please go to the following url:
then log in and leave karma (feedback).

Comment 4 Fedora Update System 2011-06-26 18:48:09 UTC
nss-softokn-3.12.10-2.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.