Bug 1229766

Summary: Pluto crashes after stop when I use floating ip address
Product: Red Hat Enterprise Linux 7 Reporter: Jaroslav Aster <jaster>
Component: libreswanAssignee: Paul Wouters <pwouters>
Status: CLOSED ERRATA QA Contact: Jaroslav Aster <jaster>
Severity: medium Docs Contact:
Priority: high    
Version: 7.2CC: jaster, jkurik, jreznik, lmiksik, omoris, tmraz
Target Milestone: rcKeywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
Cause: An IP address moves from behind the remote endpoint to be configured locally. Consequence: pluto IKE daemon would get confused and crash and restart Fix: re-orient the connection properly Result: no more crash.
Story Points: ---
Clone Of:
: 1230141 1268776 (view as bug list) Environment:
Last Closed: 2016-11-03 21:21:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1268776, 1661286    
Attachments:
Description Flags
initiator pluto.log
none
responder pluto.log none

Description Jaroslav Aster 2015-06-09 15:14:16 UTC
Description of problem:

I have test for old bug 609343, this test was manual and few month a go it was created as auto test for beaker. Now I noticed that this test failed. I see segfault in /var/log/messages after ipsec is stopped. It happens on both site, initiator and responder. It is not regression (in libreswan). I see this problem in older versions of libreswan too.

There is this messages in 3.8-5.

...
"test2": ASSERTION FAILED at /builddir/build/BUILD/libreswan-3.8/programs/pluto/connections.c:2371: oriented(*c)
...
"test2": Shunt list:
"test2":
"test2": ABORT at /builddir/build/BUILD/libreswan-3.8/programs/pluto/connections.c:2371
"test2": ABORT at /builddir/build/BUILD/libreswan-3.8/programs/pluto/connections.c:2371

Version-Release number of selected component (if applicable):

libreswan-3.12-10.1.ael7b_1
libreswan-3.12-10.1.el7_1


How reproducible:

100%

Steps to Reproduce:

I,R:

# cat /etc/ipsec.conf 
version 2.0

config setup
    protostack=netkey
    plutodebug=all
    plutostderrlog=/tmp/pluto.log     
    plutorestartoncrash=false
    dumpdir=/tmp

conn test1
    left=172.29.29.1
    right=172.29.29.2
    authby=secret
    auto=add

conn test2
    left=172.29.29.1
    right=172.29.29.3
    authby=secret
    auto=add

conn test3
    left=172.29.29.3
    right=172.29.29.2
    authby=secret
    auto=add

# cat /etc/ipsec.secrets 
: PSK "redhat"


Tunnel is created because of test, I need to have the machines in one network. It is not necessary if the machines are in one network.

I:

# ip tunnel add test mode gre local I_IP remote R_IP
# ip a add 172.29.29.1/24 dev test
# ip l set dev test up

R:

# ip tunnel add test mode gre remote I_IP local R_IP
# ip a add 172.29.29.2/24 dev test
# ip l set dev test up
# ping -c 1 172.29.29.1
PING 172.29.29.1 (172.29.29.1) 56(84) bytes of data.
64 bytes from 172.29.29.1: icmp_seq=1 ttl=64 time=172 ms

--- 172.29.29.1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 172.548/172.548/172.548/0.000 ms

I:

# service ipsec start

R:

# service ipsec start

I:

# ipsec auto --ready
002 listening for IKE messages
002 forgetting secrets
002 loading secrets from "/etc/ipsec.secrets"

R:

# ipsec auto --up test1
002 "test1" #1: initiating Main Mode
104 "test1" #1: STATE_MAIN_I1: initiate
003 "test1" #1: received Vendor ID payload [Dead Peer Detection]
003 "test1" #1: received Vendor ID payload [FRAGMENTATION]
003 "test1" #1: received Vendor ID payload [RFC 3947]
002 "test1" #1: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
002 "test1" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
106 "test1" #1: STATE_MAIN_I2: sent MI2, expecting MR2
003 "test1" #1: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
002 "test1" #1: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
108 "test1" #1: STATE_MAIN_I3: sent MI3, expecting MR3
003 "test1" #1: received Vendor ID payload [CAN-IKEv2]
002 "test1" #1: Main mode peer ID is ID_IPV4_ADDR: '172.29.1.1'
002 "test1" #1: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
004 "test1" #1: STATE_MAIN_I4: ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
002 "test1" #2: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEV2_ALLOW+SAREF_TRACK+IKE_FRAG_ALLOW {using isakmp#1 msgid:3a8e12ba proposal=defaults pfsgroup=OAKLEY_GROUP_MODP2048}
117 "test1" #2: STATE_QUICK_I1: initiate
002 "test1" #2: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
004 "test1" #2: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP=>0x847710df <0xf9e8bed1 xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}

# ip a add 172.29.1.3/24 dev test

# ipsec auto --ready
002 listening for IKE messages
002 adding interface test/test 172.29.1.3:500
002 adding interface test/test 172.29.1.3:4500
003 two interfaces match "test3" (test, test)
002 forgetting secrets
002 loading secrets from "/etc/ipsec.secrets"

# ipsec auto --up test2
002 "test2" #3: initiating Main Mode
104 "test2" #3: STATE_MAIN_I1: initiate
003 "test2" #3: received Vendor ID payload [Dead Peer Detection]
003 "test2" #3: received Vendor ID payload [FRAGMENTATION]
003 "test2" #3: received Vendor ID payload [RFC 3947]
002 "test2" #3: enabling possible NAT-traversal with method RFC 3947 (NAT-Traversal)
002 "test2" #3: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
106 "test2" #3: STATE_MAIN_I2: sent MI2, expecting MR2
003 "test2" #3: NAT-Traversal: Result using RFC 3947 (NAT-Traversal) sender port 500: no NAT detected
002 "test2" #3: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
108 "test2" #3: STATE_MAIN_I3: sent MI3, expecting MR3
003 "test2" #3: received Vendor ID payload [CAN-IKEv2]
002 "test2" #3: Main mode peer ID is ID_IPV4_ADDR: '172.29.1.1'
002 "test2" #3: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
004 "test2" #3: STATE_MAIN_I4: ISAKMP SA established {auth=PRESHARED_KEY cipher=aes_256 integ=sha group=MODP2048}
002 "test2" #4: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEV2_ALLOW+SAREF_TRACK+IKE_FRAG_ALLOW {using isakmp#3 msgid:d3432b40 proposal=defaults pfsgroup=OAKLEY_GROUP_MODP2048}
117 "test2" #4: STATE_QUICK_I1: initiate
002 "test2" #4: transition from state STATE_QUICK_I1 to state STATE_QUICK_I2
004 "test2" #4: STATE_QUICK_I2: sent QI2, IPsec SA established tunnel mode {ESP=>0x8d0a7538 <0xa4f6fda1 xfrm=AES_128-HMAC_SHA1 NATOA=none NATD=none DPD=passive}

# ip a del 172.29.1.3/24 dev test

I:

# ip a add 172.29.1.3/24 dev test

R:

# ipsec auto --ready
002 listening for IKE messages
002 shutting down interface test/test 172.29.1.3:4500
002 shutting down interface test/test 172.29.1.3:500
002 "test2" #4: deleting state (STATE_QUICK_I2)
005 "test2" #4: ESP traffic information: in=0B out=0B
002 "test2" #3: deleting state (STATE_MAIN_I4)
002 forgetting secrets
002 loading secrets from "/etc/ipsec.secrets"

I:

# ipsec auto --ready
002 listening for IKE messages
002 adding interface test/test 172.29.1.3:500
002 adding interface test/test 172.29.1.3:4500
003 two interfaces match "test2" (test, test)
002 forgetting secrets
002 loading secrets from "/etc/ipsec.secrets"

# sleep 30s

R:         

# sleep 30s

I:

# service ipsec stop

# grep -e 'segfault' -e 'unhandled signal' -e 'User process fault' /var/log/messages
Jun  9 16:44:57 sheep-46 kernel: pluto[5973]: segfault at 0 ip 00007f0896acd0a6 sp 00007fff5c7dbe30 error 4 in pluto[7f0896a6c000+ff000]


R:

# service ipsec stop

# grep -e 'segfault' -e 'unhandled signal' -e 'User process fault' /var/log/messages
Jun  9 10:44:04 ibm-p8-03-lp4 kernel: pluto[11251]: unhandled signal 11 at 0000000000000000 nip 000000005488adc0 lr 000000005488adbc code 30001


Actual results:

Segfault in /var/log/messages.

Expected results:

No segfault.

Additional info:

Logs are attached.

Comment 1 Jaroslav Aster 2015-06-09 15:15:15 UTC
Created attachment 1036876 [details]
initiator pluto.log

Comment 2 Jaroslav Aster 2015-06-09 15:15:43 UTC
Created attachment 1036877 [details]
responder pluto.log

Comment 7 Jaroslav Aster 2015-10-22 16:02:39 UTC
Hi Paul,

last build 3.15-5.el7_1 fails. There is segfault message in journal and /var/log/messages after ipsec is stopped.

Oct 22 15:23:14 HOSTNAME kernel: pluto[13053]: segfault at 0 ip 00007f10c0b98ef9 sp 00007ffed2ea2e20 error 4 in pluto[7f10c0b33000+1]

Comment 8 Paul Wouters 2015-10-23 11:05:20 UTC
can you get me a stack trace? We never saw this in our tests

Comment 9 Jaroslav Aster 2015-10-23 13:27:17 UTC
Hi Paul,

I'm afraid not. No coredump was created and I have not been able to persuade the system to create one. I have only this log message (and reproducer).

Comment 13 errata-xmlrpc 2016-11-03 21:21:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2603.html