Created attachment 814958 [details]
Output of 'ipsec barf'

Description of problem:

When 384 or 512 bit SHA2 IKE encryption/authentication algorithm is used, pluto keeps aborting on responder machine. With 256 bit SHA2 it works fine.

Version-Release number of selected component (if applicable):

openswan-2.6.32-27.el6

How reproducible:

100%

Steps to Reproduce:

1. Setup initiator and responder as follows (ipsec.conf):

   version 2.0

   config setup
     plutodebug="lifecycle"
     protostack=netkey
     nat_traversal=yes
     virtual_private=
     oe=off

   conn test
     authby=secret
     type=transport
     left=<initiator>
      right=<responder
      ike=aes128-sha2_512 # (or sha2_384)
      esp=3des-sha1
      ikev2=insist 
      auto=add

2. Start ipsec on both initiator and responder (service ipsec start)

3. Initiate connection from initiator (ipsec auto --up test)

4. See /var/log/messages on responder.

Actual results:

Connection does not work.

Responder (/var/log/messages):

002 added connection description "test"
ipsec__plutorun: /usr/libexec/ipsec/_plutorun: line 250: 19521 Aborted                 (core dumped) /usr/libexec/ipsec/pluto --nofork --secretsfile /etc/ipsec.secrets --ipsecdir /etc/ipsec.d --debug-lifecycle --use-netkey --uniqueids --nat_traversal --virtual_private oe=off
ipsec__plutorun: !pluto failure!:  exited with error status 134 (signal 6)
ipsec__plutorun: restarting IPsec after pause...
...
(and repeating)

Expected results:

Connection successful. 

Additional info:

This is *not* a regression in 6.5 from 6.4, in happens in openswan-2.6.32-21.el6_4 as well. In FIPS mode it does not work as well. Output of 'ipsec barf' from responder is attached.