Bug 1039347

Summary:	VPNaaS' vpn service is DOWN because ipsec fails to run
Product:	Red Hat OpenStack	Reporter:	Rami Vaknin <rvaknin>
Component:	openstack-neutron	Assignee:	Terry Wilson <twilson>
Status:	CLOSED DUPLICATE	QA Contact:	Rami Vaknin <rvaknin>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	4.0	CC:	breeler, chrisw, hateya, lpeer, oblaut, twilson, yeylon
Target Milestone:	rc
Target Release:	4.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-neutron-2013.2-14.el6ost	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-12-13 20:22:44 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Rami Vaknin 2013-12-08 14:03:05 UTC

Version
=======
rhos 4.0 running on rhel6.5 with 2013-12-06.3 puddle, openstack-neutron-2013.2-13.el6ost.


Description
===========
It seems like openswan with nss support requires fips mode enabled.


From the vpn log file
=====================
2013-12-08 15:49:49.874 27108 INFO neutron.openstack.common.rpc.impl_qpid [-] Connected to AMQP server on 10.35.160.29:5672
2013-12-08 15:49:49.893 27108 INFO neutron.agent.l3_agent [-] L3 agent started
2013-12-08 15:50:10.944 27108 ERROR neutron.services.vpn.device_drivers.ipsec [-] Failed to enable vpn process on router b83b5373-6ba8-45ba-9d4d-8233c20a8a72
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec Traceback (most recent call last):
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.6/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 241, in enable
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec     self.start()
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.6/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 392, in start
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec     ipsec_site_conn['id']
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.6/site-packages/neutron/services/vpn/device_drivers/ipsec.py", line 311, in _execute
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec     check_exit_code=check_exit_code)
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/ip_lib.py", line 458, in execute
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec     check_exit_code=check_exit_code)
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec   File "/usr/lib/python2.6/site-packages/neutron/agent/linux/utils.py", line 62, in execute
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec     raise RuntimeError(m)
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec RuntimeError: 
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-b83b5373-6ba8-45ba-9d4d-8233c20a8a72', 'ipsec', 'addconn', '--ctlbase', '/var/lib/neutron/ipsec/b83b5373-6ba8-45ba-9d4d-8233c20a8a72/var/run/pluto.ctl', '--defaultroutenexthop', '10.35.170.20', '--config', '/var/lib/neutron/ipsec/b83b5373-6ba8-45ba-9d4d-8233c20a8a72/etc/ipsec.conf', '89d9bcc9-2357-4bc6-b015-1f216f454096']
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec Exit code: 255
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec Stdout: ''
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec Stderr: '/usr/libexec/ipsec/addconn Non-fips mode set in /proc/sys/crypto/fips_enabled\nconnect(pluto_ctl) failed: No such file or directory\n'
2013-12-08 15:50:10.944 27108 TRACE neutron.services.vpn.device_drivers.ipsec

Comment 2 Terry Wilson 2013-12-09 19:06:38 UTC

Rami, in the future could you post the steps you take to actually produce the error? It's sometimes hard to deduce just from the log file the steps needed to reproduce. I still haven't hit this one yet. Thanks!

Comment 3 Terry Wilson 2013-12-09 19:38:52 UTC

I haven't been able to reproduce this, but there is a decent chance that it was fixed by the combined selinux/packaging fixes. Rami, can you test with the latest poodle + openstack-neutron-2013.2-14.el6ost and if it fails, include the steps to reproduce? Thanks.

Comment 7 Terry Wilson 2013-12-13 20:13:11 UTC

According to mailing list posts (like https://lists.openswan.org/pipermail/users/2012-March/021470.html), the fips stuff is just a warning that gets printed and isn't indicative of the reason for the error.

Comment 8 Terry Wilson 2013-12-13 20:22:44 UTC

I believe this bug is essentially a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1039346. When I run the modified version of openswan with CAP_DAC_OVERRIDE re-enabled (and pluto is successfully running after creating a connection), I can manually run the failing command and it succeeds (and I never see it fail in the logs). Running manually w/o a fixed openswan results in the above failure--which essentially means that pluto isn't running. Closing as duplicate. If you can reproduce after fixing the above issue, feel free to reopen.

*** This bug has been marked as a duplicate of bug 1039346 ***