Bug 709517

Summary: intel_aes_decrypt_cbc_256 doesn't work correctly when input and output buffers are the same
Product: [Fedora] Fedora Reporter: Nalin Dahyabhai <nalin>
Component: nss-softoknAssignee: Elio Maldonado Batiz <emaldona>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 15CC: emaldona, kengert, rrelyea
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: nss-softokn-3.12.10-2.fc15 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 710298 (view as bug list) Environment:
Last Closed: 2011-06-26 18:48:14 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
test none

Description Nalin Dahyabhai 2011-05-31 21:15:54 UTC 
Description of problem:
It looks as though intel_aes_decrypt_cbc_256() doesn't work right when the input and output buffers are the same, though this appears to be something that PK11_CipherOp (which eventually calls this function, on processors that support it) is expected to be able to handle.

Version-Release number of selected component (if applicable):
nss-softokn-freebl-3.12.10-1.fc15.x86_64

How reproducible:
Always, provided you're on a 64-bit processor with native AES instruction support (the "flags:" line in /proc/cpuinfo will include "aes").  I've tested this on an i5 processor and also had it checked on an i7.

Steps to Reproduce:
1. Run reproducer with the "-i" flag, which will cause it to use the same buffer for input and output and default to a 256-bit key.
  
Actual results:
"data mismatch"

Expected results:
"Encrypted/recovered 32 bytes."

Additional info:
While NSS itself doesn't seem to trigger this behavior, the nascent support for using NSS as the backend for krb5's libk5crypto does.
Comment 1 Nalin Dahyabhai 2011-05-31 21:19:59 UTC 
Created attachment 502120 [details]
test

Compile:
  gcc -o aest aest.c `pkg-config --cflags --libs nss`
Run:
  ./aest -i                          (uses in-place encryption, errors out)
  env NSS_DISABLE_HW_AES=1 ./aest -i (uses unoptimized implementation, no error)
  ./aest                             (doesn't do encryption in place, no error)
Comment 2 Fedora Update System 2011-06-22 23:20:57 UTC 
nss-softokn-3.12.10-2.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/nss-softokn-3.12.10-2.fc15
Comment 3 Fedora Update System 2011-06-24 03:29:41 UTC 
Package nss-softokn-3.12.10-2.fc15:
* should fix your issue,
* was pushed to the Fedora 15 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing nss-softokn-3.12.10-2.fc15'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/nss-softokn-3.12.10-2.fc15
then log in and leave karma (feedback).
Comment 4 Fedora Update System 2011-06-26 18:48:09 UTC 
nss-softokn-3.12.10-2.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.