Bug 1329905 - High CPU load VPNaaS and libreswan (certutil)
Summary: High CPU load VPNaaS and libreswan (certutil)
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-neutron-vpnaas
Version: 8.0 (Liberty)
Hardware: x86_64
OS: Linux
low
high
Target Milestone: ---
: 10.0 (Newton)
Assignee: Assaf Muller
QA Contact: Toni Freger
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-24 16:52 UTC by kevin.olbrich
Modified: 2016-09-14 17:43 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-09-14 17:42:40 UTC
Target Upstream Version:


Attachments (Terms of Use)
vpn-agent log (23.00 KB, text/plain)
2016-04-24 17:00 UTC, kevin.olbrich
no flags Details
ps -ax (36.83 KB, text/plain)
2016-04-24 17:03 UTC, kevin.olbrich
no flags Details

Description kevin.olbrich 2016-04-24 16:52:06 UTC
Description of problem:
When installing libreswan and neutron-vpnaas-agent, I get 100% cpu load on all cores. Problem exists in Liberty and Mitaka deployments via Packstack.

The process consuming the cpu-cycles is "certutil":

certutil -N -d sql:/etc/ipsec.d --empty-password
It spawns serveral times, sometimes the process dies when swap runs full. Running this command on root shell works flawlessly but the problem returns.


Version-Release number of selected component (if applicable):
[root@testnode1 ~]# rpm -qa | grep openstack
openstack-nova-cert-13.0.0-1.el7.noarch
openstack-neutron-lbaas-8.0.0-1.el7.noarch
openstack-swift-plugin-swift3-1.10-1.el7.noarch
openstack-gnocchi-metricd-2.0.2-1.el7.noarch
openstack-ceilometer-collector-6.0.0-2.el7.noarch
openstack-aodh-listener-2.0.0-1.el7.noarch
centos-release-openstack-mitaka-1-2.el7.centos.noarch
openstack-packstack-puppet-8.0.0-0.7.0rc2.el7.noarch
openstack-keystone-9.0.0-1.el7.noarch
python-django-openstack-auth-2.2.0-1.el7.noarch
openstack-neutron-vpnaas-8.0.0-1.el7.noarch
openstack-swift-account-2.6.0-1.el7.noarch
openstack-cinder-8.0.0-1.el7.noarch
openstack-gnocchi-api-2.0.2-1.el7.noarch
openstack-ceilometer-notification-6.0.0-2.el7.noarch
openstack-aodh-common-2.0.0-1.el7.noarch
openstack-nova-compute-13.0.0-1.el7.noarch
openstack-neutron-metering-agent-8.0.0-1.el7.noarch
openstack-glance-12.0.0-1.el7.noarch
openstack-ceilometer-common-6.0.0-2.el7.noarch
openstack-gnocchi-carbonara-2.0.2-1.el7.noarch
openstack-ceilometer-compute-6.0.0-2.el7.noarch
python2-openstacksdk-0.8.3-1.el7.noarch
openstack-selinux-0.6.58-1.el7.noarch
openstack-utils-2015.2-1.el7.noarch
openstack-nova-common-13.0.0-1.el7.noarch
openstack-ceilometer-polling-6.0.0-2.el7.noarch
openstack-nova-conductor-13.0.0-1.el7.noarch
openstack-nova-scheduler-13.0.0-1.el7.noarch
openstack-neutron-common-8.0.0-1.el7.noarch
openstack-neutron-ml2-8.0.0-1.el7.noarch
openstack-neutron-openvswitch-8.0.0-1.el7.noarch
openstack-swift-object-2.6.0-1.el7.noarch
openstack-gnocchi-common-2.0.2-1.el7.noarch
openstack-gnocchi-statsd-2.0.2-1.el7.noarch
openstack-ceilometer-api-6.0.0-2.el7.noarch
openstack-aodh-notifier-2.0.0-1.el7.noarch
openstack-neutron-fwaas-8.0.0-3.el7.noarch
openstack-swift-2.6.0-1.el7.noarch
openstack-gnocchi-indexer-sqlalchemy-2.0.2-1.el7.noarch
openstack-neutron-8.0.0-1.el7.noarch
openstack-swift-proxy-2.6.0-1.el7.noarch
openstack-ceilometer-central-6.0.0-2.el7.noarch
openstack-aodh-api-2.0.0-1.el7.noarch
openstack-nova-console-13.0.0-1.el7.noarch
python-openstackclient-2.2.0-1.el7.noarch
openstack-puppet-modules-8.0.0-1.el7.noarch
openstack-packstack-8.0.0-0.7.0rc2.el7.noarch
openstack-nova-novncproxy-13.0.0-1.el7.noarch
openstack-dashboard-9.0.0-1.el7.noarch
openstack-swift-container-2.6.0-1.el7.noarch
openstack-nova-api-13.0.0-1.el7.noarch
openstack-aodh-evaluator-2.0.0-1.el7.noarch

[root@testnode1 ~]# rpm -qa | grep libresw
libreswan-3.15-5.el7_1.x86_64

How reproducible:
100% of deployments

Steps to Reproduce:
1. Run Packstack All-In-One on CentOS with VPNaaS enabled
2. Create VPN-SiteToSite-VPN
3. Restart neutron-vpn-agent

Actual results:
100% load, no VPN

Expected results:
normal load, working VPN

Additional info:

Comment 2 kevin.olbrich 2016-04-24 17:00:17 UTC
Created attachment 1150200 [details]
vpn-agent log

Comment 3 kevin.olbrich 2016-04-24 17:03:38 UTC
Created attachment 1150201 [details]
ps -ax

Comment 4 kevin.olbrich 2016-04-24 18:05:49 UTC
[root@testnode1 ~]# ps -ax | grep certutil
 6042 ?        R     14:09 certutil -N -d sql:/etc/ipsec.d --empty-password
 8457 ?        R    156:34 certutil -N -d sql:/etc/ipsec.d --empty-password
12071 ?        R    140:10 certutil -N -d sql:/etc/ipsec.d --empty-password
15747 ?        R    122:42 certutil -N -d sql:/etc/ipsec.d --empty-password
20058 ?        R    100:45 certutil -N -d sql:/etc/ipsec.d --empty-password
26105 ?        R     73:14 certutil -N -d sql:/etc/ipsec.d --empty-password
32342 ?        R     44:09 certutil -N -d sql:/etc/ipsec.d --empty-password

Comment 5 kevin.olbrich 2016-04-24 20:07:08 UTC
Just tested OpenSwan from EPEL and it seems to work.

vpn_agent.ini:
vpn_device_driver=neutron_vpnaas.services.vpn.device_drivers.fedora_strongswan_ipsec.FedoraStrongSwanDriver

Comment 6 kevin.olbrich 2016-04-24 20:33:10 UTC
(In reply to kevin.olbrich from comment #5)
> Just tested OpenSwan from EPEL and it seems to work.
> 
> vpn_agent.ini:
> vpn_device_driver=neutron_vpnaas.services.vpn.device_drivers.
> fedora_strongswan_ipsec.FedoraStrongSwanDriver

Sorry, I meant StrongSwan.

Comment 7 Assaf Muller 2016-09-14 17:43:33 UTC
VPNaaS does not align with our team capacity and prioritization. I'd rather mark this as won't fixed and set expectations rather than let the bug rot open for years.


Note You need to log in before you can comment on or make changes to this bug.