Bug 1203904 - NetworkManager-openvpn uses 100% CPU
Summary: NetworkManager-openvpn uses 100% CPU
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager-openvpn
Version: 21
Hardware: x86_64
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dan Williams
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-03-19 22:37 UTC by Patrick MacArthur
Modified: 2015-08-25 13:44 UTC (History)
8 users (show)

Fixed In Version: NetworkManager-1.0.2-1.fc22
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-05-11 19:04:23 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
[PATCH] fix loop in NM on 0.0.0.0/1 route (1.33 KB, patch)
2015-03-20 14:01 UTC, Jirka Klimes
no flags Details | Diff
[PATCH] fix loop in NM on 0.0.0.0/1 route (1.65 KB, patch)
2015-03-23 08:26 UTC, Jirka Klimes
no flags Details | Diff
[PATCH] fix loop in NM on 0.0.0.0/1 route (3.92 KB, patch)
2015-03-23 08:55 UTC, Jirka Klimes
no flags Details | Diff

Description Patrick MacArthur 2015-03-19 22:37:29 UTC
Description of problem:

When connecting to a VPN that pushes a moderately complex set of routes to the client, the connection succeeds but uses 100% CPU and spams the logs with the same message over and over again.  This issue does not occur when connecting to a VPN which pushes only a single default route.

Version-Release number of selected component (if applicable):

NetworkManager-0.9.10.1-3.20150219git.fc21.x86_64
NetworkManager-openvpn-1.0.0-1.fc21.1.x86_64

How reproducible:

Connect to an openvpn server which pushes something like the following two routes:
0.0.0.0/1 -> gateway
128.0.0.0/1 -> gateway

Steps to Reproduce:
1. Set up VPN in NetworkManager client configuration.
2. Connect to VPN.
3. Examine CPU usage with something like top.

Actual results:

NetworkManager indicates that the VPN is up, but does not print out all expected log messages to the systemd journal, and spams the journal with the following message over and over again:

Mar 19 14:57:18 lancelot NetworkManager[9062]: <error> [1426802238.475451] [devices/nm-device.c:1750] nm_device_generate_connection(): (tap0): Generated connection does not verify: ipv4.routes: 1. route is invalid

Expected results:
NetworkManager indicates that the VPN is up and prints messages into the systemd journal concluding with "Activation (tap0) successful, device activated."


Additional info:

Comment 1 Jirka Klimes 2015-03-20 14:00:13 UTC
I reproduced that by setting up testing OpenVPN server using sample files from 
/usr/share/doc/openvpn/sample/...
and adding
push "route remote_host 255.255.255.255 net_gateway"
push "route 0.0.0.0 128.0.0.0 10.8.0.5"
push "route 128.0.0.0 128.0.0.0 10.8.0.5"
to server.conf

The problem is that 0.0.0.0/1 is denied in libnm-util verify() as invalid.

The NM 1.0 and master seems fine. It uses libnm instead and there might be other fixes as well.

Test scratch build with a fix:
http://koji.fedoraproject.org/koji/taskinfo?taskID=9279924

Comment 2 Jirka Klimes 2015-03-20 14:01:58 UTC
Created attachment 1004532 [details]
[PATCH] fix loop in NM on 0.0.0.0/1 route

Allow 0.0.0.0/1 route in libnm-util:verify()

Comment 3 Thomas Haller 2015-03-20 14:28:00 UTC
(In reply to Jirka Klimes from comment #2)
> Created attachment 1004532 [details]
> [PATCH] fix loop in NM on 0.0.0.0/1 route
> 
> Allow 0.0.0.0/1 route in libnm-util:verify()

this looks right.


The loop of regenerating and failing a connection would also need fixing, right?

Comment 4 Thomas Haller 2015-03-20 14:32:14 UTC
Shouldn't we also verify the routes in libnm-util/nm-setting-ip6-config.c?


and actually, it should be

-         if (!nm_ip4_route_get_dest (route)) {
+         if (prefix == 0) {


we want to reject here default routes -- which is a route with prefix==0.

Comment 5 Patrick MacArthur 2015-03-20 16:34:40 UTC
(In reply to Jirka Klimes from comment #1)
> I reproduced that by setting up testing OpenVPN server using sample files
> from 
> /usr/share/doc/openvpn/sample/...
> and adding
> push "route remote_host 255.255.255.255 net_gateway"
> push "route 0.0.0.0 128.0.0.0 10.8.0.5"
> push "route 128.0.0.0 128.0.0.0 10.8.0.5"
> to server.conf
> 
> The problem is that 0.0.0.0/1 is denied in libnm-util verify() as invalid.
> 
> The NM 1.0 and master seems fine. It uses libnm instead and there might be
> other fixes as well.
> 
> Test scratch build with a fix:
> http://koji.fedoraproject.org/koji/taskinfo?taskID=9279924

The scratch build fixed the issue for me.

Comment 6 Jirka Klimes 2015-03-23 08:26:09 UTC
Created attachment 1005246 [details]
[PATCH] fix loop in NM on 0.0.0.0/1 route

Actually, we can just remove the 'if (!nm_ip4_route_get_dest (route)' check.
There's no such check for IPv6 routes.

Comment 7 Thomas Haller 2015-03-23 08:33:01 UTC
(In reply to Jirka Klimes from comment #6)
> Created attachment 1005246 [details]
> [PATCH] fix loop in NM on 0.0.0.0/1 route
> 
> Actually, we can just remove the 'if (!nm_ip4_route_get_dest (route)' check.
> There's no such check for IPv6 routes.

yes, the !prefix check below is enough.

I think ipv6 should validate the routes too, as it does for ipv4.



it still needs fixing of the loop when the generated connection does not verify.

Comment 8 Jirka Klimes 2015-03-23 08:55:19 UTC
(In reply to Thomas Haller from comment #7)
> (In reply to Jirka Klimes from comment #6)
> > Created attachment 1005246 [details]
> > [PATCH] fix loop in NM on 0.0.0.0/1 route
> > 
> > Actually, we can just remove the 'if (!nm_ip4_route_get_dest (route)' check.
> > There's no such check for IPv6 routes.
> 
> yes, the !prefix check below is enough.
> 
> I think ipv6 should validate the routes too, as it does for ipv4.
>
In the next patch iteraction.

> 
> it still needs fixing of the loop when the generated connection does not
> verify.
Yes, we should look into it. But actually it is independent of the routes verification and can be done separately later.

Comment 9 Jirka Klimes 2015-03-23 08:55:46 UTC
Created attachment 1005251 [details]
[PATCH] fix loop in NM on 0.0.0.0/1 route

Comment 10 Dan Williams 2015-03-23 17:00:35 UTC
Looks fine to me.  On 1.0+ Thomas did some work to allow 0.0.0.0 routes in the settings specifically, which is why they work OK.  0.9.10 doesn't have those enhancements so the best we can do is work around it for situations like this.

Comment 11 Jirka Klimes 2015-03-24 11:36:19 UTC
Committed upstream:
ba35c63 libnm-util: allow 0.0.0.0/1 route in verify() (rh #1203904) - master
cd5c904 libnm-util: allow 0.0.0.0/1 route in verify() (rh #1203904) - nm-1-0
5a09a12 libnm-util: allow 0.0.0.0/1 route in verify() (rh #1203904) - nm-0-9-10

Comment 12 Thomas Haller 2015-03-24 11:56:33 UTC
(In reply to Jirka Klimes from comment #8)
> (In reply to Thomas Haller from comment #7)
> > it still needs fixing of the loop when the generated connection does not
> > verify.
> Yes, we should look into it. But actually it is independent of the routes
> verification and can be done separately later.

leaving bug in NEW to fix ^^^ :)

The larger bug is IMO that hell breaks loose if we generate an invalid connection.

Comment 13 Fedora Update System 2015-05-04 10:10:53 UTC
NetworkManager-0.9.10.2-3.fc21 has been submitted as an update for Fedora 21.
https://admin.fedoraproject.org/updates/NetworkManager-0.9.10.2-3.fc21

Comment 14 Fedora Update System 2015-05-05 13:13:37 UTC
NetworkManager-1.0.2-1.fc22,network-manager-applet-1.0.2-1.fc22,NetworkManager-openconnect-1.0.2-1.fc22,NetworkManager-openvpn-1.0.2-1.fc22,NetworkManager-vpnc-1.0.2-1.fc22,NetworkManager-openswan-1.0.2-1.fc22 has been submitted as an update for Fedora 22.
https://admin.fedoraproject.org/updates/NetworkManager-1.0.2-1.fc22,network-manager-applet-1.0.2-1.fc22,NetworkManager-openconnect-1.0.2-1.fc22,NetworkManager-openvpn-1.0.2-1.fc22,NetworkManager-vpnc-1.0.2-1.fc22,NetworkManager-openswan-1.0.2-1.fc22

Comment 15 Patrick MacArthur 2015-05-09 21:36:50 UTC
I can confirm that the issue is no longer present with NetworkManager-0.9.10.2-3.fc21.

Comment 16 Fedora Update System 2015-05-10 23:50:33 UTC
Package NetworkManager-1.0.2-1.fc22, NetworkManager-openconnect-1.0.2-1.fc22, NetworkManager-vpnc-1.0.2-1.fc22, network-manager-applet-1.0.2-1.fc22, NetworkManager-openvpn-1.0.2-1.fc22, NetworkManager-openswan-1.0.2-1.fc22:
* should fix your issue,
* was pushed to the Fedora 22 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing NetworkManager-1.0.2-1.fc22 NetworkManager-openconnect-1.0.2-1.fc22 NetworkManager-vpnc-1.0.2-1.fc22 network-manager-applet-1.0.2-1.fc22 NetworkManager-openvpn-1.0.2-1.fc22 NetworkManager-openswan-1.0.2-1.fc22'
as soon as you are able to, then reboot.
Please go to the following url:
https://admin.fedoraproject.org/updates/FEDORA-2015-7767/NetworkManager-1.0.2-1.fc22,network-manager-applet-1.0.2-1.fc22,NetworkManager-openconnect-1.0.2-1.fc22,NetworkManager-openvpn-1.0.2-1.fc22,NetworkManager-vpnc-1.0.2-1.fc22,NetworkManager-openswan-1.0.2-1.fc22
then log in and leave karma (feedback).

Comment 17 Fedora Update System 2015-05-11 19:04:23 UTC
NetworkManager-1.0.2-1.fc22, NetworkManager-openconnect-1.0.2-1.fc22, NetworkManager-vpnc-1.0.2-1.fc22, network-manager-applet-1.0.2-1.fc22, NetworkManager-openvpn-1.0.2-1.fc22, NetworkManager-openswan-1.0.2-1.fc22 has been pushed to the Fedora 22 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 18 Fedora Update System 2015-05-17 06:42:14 UTC
NetworkManager-0.9.10.2-5.fc21 has been pushed to the Fedora 21 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 19 sukhamay 2015-08-25 13:44:01 UTC
fedora not take update


Note You need to log in before you can comment on or make changes to this bug.