Created attachment 572312 [details]
crash backtrace got from serial console

Description of problem:

When AES (128 bit) is used for phase 2 in IPSec configuration, kernel crashes as soon as a first encrypted packed arrives from remote end. Prior to this, all SAs are succesfully established.
When 3des encryption is used, everything works fine.

Version-Release number of selected component (if applicable):

Several last 3.2.x kernel versions from official Fedora repositories were tested with the same result. The report is for latest available version, 3.3.0-4.fc16.i686.PAE

How reproducible:

I am able to reproduce it every time I try to set up IPSec tunnel according to configuration included in attachments.

Steps to Reproduce:
1. Install ipsec-tools on two systems (r1, r2)
2. configure and bring up network
3. Put configuration files in /etc/racoon (test configuration attached below)
3. setkey -f /etc/racoon/setkey.conf (on both machines)
4. service racoon start (on both machines)
5. ping LAN side of r2 from r1
6. r2 crashes
  
Actual results:

Kernel panic.

Expected results:

Working IPSec tunnel with AES encryption.

Additional info:

Hardware used: HP DL180 G5
Boot messages, backtrace, some system information and test configuration included as attachments.

The only similar report I was able to find in internet is here: http://www.mail-archive.com/linux-crypto@vger.kernel.org/msg06449.html

The obvious difference between aes and other encryption algorithms is, that aes is compiled as a part of main kernel image, other encryption modules are compiled as ... modules.

I tried this also using Openswan's userspace tools with the same result.

Probably I won't be able to test this anymore on affected systems, they will be put in production environment in 2 days. I don't know yet, whether it happens also on different hardware.