+++ This bug was initially created as a clone of Bug #1211287 +++ Description of problem: After making certain (maybe any) changes to a connection, including activation and deactivation, all network traffic to the machine (such as ping) is blocked for a certain time, even if the machine still has other working connections. [ This could be caused by anything, including the virtual network I use with my virtual machines, of course. So this more a call for debugging help than a bug report. ] Version-Release number of selected component (if applicable): NetworkManager-1.0.0-14.git20150121.b4ea599c.el7.x86_64 How reproducible: Always Steps to Reproduce: 1. Install rhel-server-7.1-x86_64-dvd.iso into a new virtual machine by clicking Next in the virtual-manager UI until done. 2. Add a second network interface to it. (I used default settings.) 3. Make sure that the primary network interface (probably eth0) is connected. 4. Make sure that the second interface (ens9 in my case) is disconnected. 5. Start a ping to the machine from the outside to the IP address of the primary interface. 6. Connect the second interface. Actual results: Ping stops for about 30 seconds. All other connections, such as ssh, also freeze for the same time. Expected results: Machine stays connected normally. Additional info: A screenshot of the VM console is attached, and this is the output of ping. Note the gap from 8 to 46. $ ping 192.168.100.242 PING 192.168.100.242 (192.168.100.242) 56(84) bytes of data. 64 bytes from 192.168.100.242: icmp_seq=1 ttl=64 time=0.169 ms 64 bytes from 192.168.100.242: icmp_seq=2 ttl=64 time=0.216 ms 64 bytes from 192.168.100.242: icmp_seq=3 ttl=64 time=0.218 ms 64 bytes from 192.168.100.242: icmp_seq=4 ttl=64 time=0.197 ms 64 bytes from 192.168.100.242: icmp_seq=5 ttl=64 time=0.241 ms 64 bytes from 192.168.100.242: icmp_seq=6 ttl=64 time=0.231 ms 64 bytes from 192.168.100.242: icmp_seq=7 ttl=64 time=0.222 ms 64 bytes from 192.168.100.242: icmp_seq=8 ttl=64 time=0.159 ms 64 bytes from 192.168.100.242: icmp_seq=46 ttl=64 time=0.854 ms 64 bytes from 192.168.100.242: icmp_seq=47 ttl=64 time=0.311 ms 64 bytes from 192.168.100.242: icmp_seq=48 ttl=64 time=0.412 ms 64 bytes from 192.168.100.242: icmp_seq=49 ttl=64 time=0.544 ms 64 bytes from 192.168.100.242: icmp_seq=50 ttl=64 time=0.554 ms 64 bytes from 192.168.100.242: icmp_seq=51 ttl=64 time=0.281 ms 64 bytes from 192.168.100.242: icmp_seq=52 ttl=64 time=0.489 ms 64 bytes from 192.168.100.242: icmp_seq=53 ttl=64 time=0.724 ms 64 bytes from 192.168.100.242: icmp_seq=54 ttl=64 time=0.426 ms 64 bytes from 192.168.100.242: icmp_seq=55 ttl=64 time=0.248 ms 64 bytes from 192.168.100.242: icmp_seq=56 ttl=64 time=0.636 ms 64 bytes from 192.168.100.242: icmp_seq=57 ttl=64 time=0.261 ms 64 bytes from 192.168.100.242: icmp_seq=58 ttl=64 time=0.655 ms 64 bytes from 192.168.100.242: icmp_seq=59 ttl=64 time=0.327 ms 64 bytes from 192.168.100.242: icmp_seq=60 ttl=64 time=0.401 ms 64 bytes from 192.168.100.242: icmp_seq=61 ttl=64 time=0.279 ms 64 bytes from 192.168.100.242: icmp_seq=62 ttl=64 time=0.486 ms ^C --- 192.168.100.242 ping statistics --- 62 packets transmitted, 25 received, 59% packet loss, time 61000ms rtt min/avg/max/mdev = 0.159/0.381/0.854/0.188 ms --- Additional comment from Marius Vollmer on 2015-04-13 10:01:28 EDT --- NM state for eth0 and ens9 # nmcli c NAME UUID TYPE DEVICE ens9 04afb9b7-3ff7-4843-ae2d-448f3a77723b 802-3-ethernet ens9 eth0 aea527e8-5880-4790-8da5-f15d5f7c5dee 802-3-ethernet eth0 [root@localhost ~]# nmcli c show eth0 connection.id: eth0 connection.uuid: aea527e8-5880-4790-8da5-f15d5f7c5dee connection.interface-name: eth0 connection.type: 802-3-ethernet connection.autoconnect: yes connection.autoconnect-priority: 0 connection.timestamp: 1428933572 connection.read-only: no connection.permissions: connection.zone: -- connection.master: -- connection.slave-type: -- connection.secondaries: connection.gateway-ping-timeout: 0 802-3-ethernet.port: -- 802-3-ethernet.speed: 0 802-3-ethernet.duplex: -- 802-3-ethernet.auto-negotiate: yes 802-3-ethernet.mac-address: -- 802-3-ethernet.cloned-mac-address: -- 802-3-ethernet.mac-address-blacklist: 802-3-ethernet.mtu: auto 802-3-ethernet.s390-subchannels: 802-3-ethernet.s390-nettype: -- 802-3-ethernet.s390-options: ipv4.method: auto ipv4.dns: ipv4.dns-search: ipv4.addresses: ipv4.gateway: -- ipv4.routes: ipv4.route-metric: -1 ipv4.ignore-auto-routes: no ipv4.ignore-auto-dns: no ipv4.dhcp-client-id: -- ipv4.dhcp-send-hostname: yes ipv4.dhcp-hostname: -- ipv4.never-default: no ipv4.may-fail: yes ipv6.method: auto ipv6.dns: ipv6.dns-search: ipv6.addresses: ipv6.gateway: -- ipv6.routes: ipv6.route-metric: -1 ipv6.ignore-auto-routes: no ipv6.ignore-auto-dns: no ipv6.never-default: no ipv6.may-fail: yes ipv6.ip6-privacy: -1 (unknown) ipv6.dhcp-send-hostname: yes ipv6.dhcp-hostname: -- GENERAL.NAME: eth0 GENERAL.UUID: aea527e8-5880-4790-8da5-f15d5f7c5dee GENERAL.DEVICES: eth0 GENERAL.STATE: activated GENERAL.DEFAULT: yes GENERAL.DEFAULT6: no GENERAL.VPN: no GENERAL.ZONE: -- GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/ActiveConnection/0 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Settings/0 GENERAL.SPEC-OBJECT: / GENERAL.MASTER-PATH: -- IP4.ADDRESS[1]: 192.168.100.242/24 IP4.GATEWAY: 192.168.100.1 IP4.DNS[1]: 192.168.100.1 IP4.DOMAIN[1]: mvo.lan DHCP4.OPTION[1]: requested_classless_static_routes = 1 DHCP4.OPTION[2]: requested_rfc3442_classless_static_routes = 1 DHCP4.OPTION[3]: subnet_mask = 255.255.255.0 DHCP4.OPTION[4]: requested_subnet_mask = 1 DHCP4.OPTION[5]: domain_name_servers = 192.168.100.1 DHCP4.OPTION[6]: ip_address = 192.168.100.242 DHCP4.OPTION[7]: requested_static_routes = 1 DHCP4.OPTION[8]: dhcp_server_identifier = 192.168.100.1 DHCP4.OPTION[9]: requested_nis_servers = 1 DHCP4.OPTION[10]: requested_time_offset = 1 DHCP4.OPTION[11]: broadcast_address = 192.168.100.255 DHCP4.OPTION[12]: requested_interface_mtu = 1 DHCP4.OPTION[13]: dhcp_rebinding_time = 3150 DHCP4.OPTION[14]: requested_domain_name_servers = 1 DHCP4.OPTION[15]: dhcp_message_type = 5 DHCP4.OPTION[16]: requested_broadcast_address = 1 DHCP4.OPTION[17]: routers = 192.168.100.1 DHCP4.OPTION[18]: dhcp_renewal_time = 1800 DHCP4.OPTION[19]: requested_domain_name = 1 DHCP4.OPTION[20]: domain_name = mvo.lan DHCP4.OPTION[21]: requested_routers = 1 DHCP4.OPTION[22]: expiry = 1428935072 DHCP4.OPTION[23]: requested_wpad = 1 DHCP4.OPTION[24]: requested_nis_domain = 1 DHCP4.OPTION[25]: requested_ms_classless_static_routes = 1 DHCP4.OPTION[26]: network_number = 192.168.100.0 DHCP4.OPTION[27]: requested_domain_search = 1 DHCP4.OPTION[28]: next_server = 192.168.100.1 DHCP4.OPTION[29]: requested_ntp_servers = 1 DHCP4.OPTION[30]: requested_host_name = 1 DHCP4.OPTION[31]: dhcp_lease_time = 3600 IP6.ADDRESS[1]: fe80::5054:ff:fed5:5bc7/64 IP6.GATEWAY: [root@localhost ~]# nmcli c show ens9 connection.id: ens9 connection.uuid: 04afb9b7-3ff7-4843-ae2d-448f3a77723b connection.interface-name: ens9 connection.type: 802-3-ethernet connection.autoconnect: no connection.autoconnect-priority: 0 connection.timestamp: 1428933572 connection.read-only: no connection.permissions: connection.zone: -- connection.master: -- connection.slave-type: -- connection.secondaries: connection.gateway-ping-timeout: 0 802-3-ethernet.port: -- 802-3-ethernet.speed: 0 802-3-ethernet.duplex: -- 802-3-ethernet.auto-negotiate: yes 802-3-ethernet.mac-address: 52:54:00:BC:91:FE 802-3-ethernet.cloned-mac-address: -- 802-3-ethernet.mac-address-blacklist: 802-3-ethernet.mtu: auto 802-3-ethernet.s390-subchannels: 802-3-ethernet.s390-nettype: -- 802-3-ethernet.s390-options: ipv4.method: auto ipv4.dns: ipv4.dns-search: ipv4.addresses: ipv4.gateway: -- ipv4.routes: ipv4.route-metric: -1 ipv4.ignore-auto-routes: no ipv4.ignore-auto-dns: no ipv4.dhcp-client-id: -- ipv4.dhcp-send-hostname: yes ipv4.dhcp-hostname: -- ipv4.never-default: no ipv4.may-fail: yes ipv6.method: auto ipv6.dns: ipv6.dns-search: ipv6.addresses: ipv6.gateway: -- ipv6.routes: ipv6.route-metric: -1 ipv6.ignore-auto-routes: no ipv6.ignore-auto-dns: no ipv6.never-default: no ipv6.may-fail: yes ipv6.ip6-privacy: -1 (unknown) ipv6.dhcp-send-hostname: yes ipv6.dhcp-hostname: -- GENERAL.NAME: ens9 GENERAL.UUID: 04afb9b7-3ff7-4843-ae2d-448f3a77723b GENERAL.DEVICES: ens9 GENERAL.STATE: activated GENERAL.DEFAULT: no GENERAL.DEFAULT6: no GENERAL.VPN: no GENERAL.ZONE: -- GENERAL.DBUS-PATH: /org/freedesktop/NetworkManager/ActiveConnection/1 GENERAL.CON-PATH: /org/freedesktop/NetworkManager/Settings/1 GENERAL.SPEC-OBJECT: / GENERAL.MASTER-PATH: -- IP4.ADDRESS[1]: 192.168.100.148/24 IP4.GATEWAY: 192.168.100.1 IP4.DNS[1]: 192.168.100.1 IP4.DOMAIN[1]: mvo.lan DHCP4.OPTION[1]: requested_classless_static_routes = 1 DHCP4.OPTION[2]: requested_rfc3442_classless_static_routes = 1 DHCP4.OPTION[3]: subnet_mask = 255.255.255.0 DHCP4.OPTION[4]: requested_subnet_mask = 1 DHCP4.OPTION[5]: domain_name_servers = 192.168.100.1 DHCP4.OPTION[6]: ip_address = 192.168.100.148 DHCP4.OPTION[7]: requested_static_routes = 1 DHCP4.OPTION[8]: dhcp_server_identifier = 192.168.100.1 DHCP4.OPTION[9]: requested_nis_servers = 1 DHCP4.OPTION[10]: requested_time_offset = 1 DHCP4.OPTION[11]: broadcast_address = 192.168.100.255 DHCP4.OPTION[12]: requested_interface_mtu = 1 DHCP4.OPTION[13]: dhcp_rebinding_time = 3023 DHCP4.OPTION[14]: requested_domain_name_servers = 1 DHCP4.OPTION[15]: dhcp_message_type = 5 DHCP4.OPTION[16]: requested_broadcast_address = 1 DHCP4.OPTION[17]: routers = 192.168.100.1 DHCP4.OPTION[18]: dhcp_renewal_time = 1673 DHCP4.OPTION[19]: requested_domain_name = 1 DHCP4.OPTION[20]: domain_name = mvo.lan DHCP4.OPTION[21]: requested_routers = 1 DHCP4.OPTION[22]: expiry = 1428936657 DHCP4.OPTION[23]: requested_wpad = 1 DHCP4.OPTION[24]: requested_nis_domain = 1 DHCP4.OPTION[25]: requested_ms_classless_static_routes = 1 DHCP4.OPTION[26]: network_number = 192.168.100.0 DHCP4.OPTION[27]: requested_domain_search = 1 DHCP4.OPTION[28]: next_server = 192.168.100.1 DHCP4.OPTION[29]: requested_ntp_servers = 1 DHCP4.OPTION[30]: requested_host_name = 1 DHCP4.OPTION[31]: dhcp_lease_time = 3600 IP6.ADDRESS[1]: fe80::5054:ff:febc:91fe/64 IP6.GATEWAY: --- Additional comment from Marius Vollmer on 2015-04-13 10:02:52 EDT --- Virtual network XML: $ virsh net-dumpxml default <network connections='4'> <name>default</name> <uuid>60a2a9c4-ccae-1d11-0aff-6fc9f74e3847</uuid> <forward mode='nat'> <nat> <port start='1024' end='65535'/> </nat> </forward> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:16:4f:a7'/> <domain name='mvo.lan'/> <dns> <host ip='192.168.100.3'> <hostname>vm-checkmachine2</hostname> </host> </dns> <ip address='192.168.100.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.100.128' end='192.168.100.254'/> <host mac='52:54:00:d0:03:00' name='f20.mvo.lan' ip='192.168.100.42'/> <host mac='52:54:00:13:e2:98' name='f21.mvo.lan' ip='192.168.100.21'/> <host mac='52:54:00:b2:c5:77' name='f22.mvo.lan' ip='192.168.100.22'/> <host mac='52:54:00:45:7e:db' name='ipa.mvo.lan' ip='192.168.100.2'/> <host mac='52:54:00:95:84:8c' name='collide.mvo.lan' ip='192.168.100.99'/> <host mac='52:54:00:09:33:93' name='vm-checkmachine2' ip='192.168.100.3'/> </dhcp> </ip> </network> --- Additional comment from Marius Vollmer on 2015-04-13 10:14:38 EDT --- I have set the virtual network bridge to stp='off', but that didn't help. --- Additional comment from Marius Vollmer on 2015-04-13 10:16:58 EDT --- Doing the same with a Fedora 22 guest leads to the expected behaviour. --- Additional comment from RHEL Product and Program Management on 2015-04-13 10:17:36 EDT --- Since this bug report was entered in bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release. --- Additional comment from Dominik Perpeet on 2015-04-15 08:48:32 EDT --- I can reproduce the error reliably on a rhel 7.1 guest minimal install (fully updated as of today) as a guest on rhel7 csb. I added second network interface, down. Same commands as Marius: cat /etc/system-release Red Hat Enterprise Linux Server release 7.1 (Maipo) [root@localhost ~]# yum info NetworkManager Loaded plugins: product-id, subscription-manager Installed Packages Name : NetworkManager Arch : x86_64 Epoch : 1 Version : 1.0.0 Release : 14.git20150121.b4ea599c.el7 Size : 8.8 M Repo : installed From repo : anaconda Summary : Network connection manager and user applications URL : http://www.gnome.org/projects/NetworkManager/ License : GPLv2+ Description : NetworkManager is a system service that manages network interfaces and : connections based on user or automatic configuration. It supports : Ethernet, Bridge, Bond, VLAN, Team, InfiniBand, Wi-Fi, mobile broadband : (WWAN), PPPoE and other devices, and supports a variety of different VPN : services. [root@localhost ~]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:12:58:1c brd ff:ff:ff:ff:ff:ff inet 192.168.122.129/24 brd 192.168.122.255 scope global dynamic eth0 valid_lft 2819sec preferred_lft 2819sec inet6 fe80::5054:ff:fe12:581c/64 scope link valid_lft forever preferred_lft forever 3: ens9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:45:ad:d0 brd ff:ff:ff:ff:ff:ff [root@localhost ~]# nmcli d c ens9 Device 'ens9' successfully activated with 'de9e03ca-2688-4e34-90cd-5d6c6905f86d'. Ping while adding connection (note the gap between seq #48 and #97): Wed Apr 15 14:35:32 2015 64 bytes from 192.168.122.129: icmp_seq=30 ttl=64 time=0.568 ms Wed Apr 15 14:35:33 2015 64 bytes from 192.168.122.129: icmp_seq=31 ttl=64 time=0.191 ms Wed Apr 15 14:35:34 2015 64 bytes from 192.168.122.129: icmp_seq=32 ttl=64 time=0.223 ms Wed Apr 15 14:35:35 2015 64 bytes from 192.168.122.129: icmp_seq=33 ttl=64 time=0.234 ms Wed Apr 15 14:35:36 2015 64 bytes from 192.168.122.129: icmp_seq=34 ttl=64 time=0.113 ms Wed Apr 15 14:35:37 2015 64 bytes from 192.168.122.129: icmp_seq=35 ttl=64 time=0.241 ms Wed Apr 15 14:35:38 2015 64 bytes from 192.168.122.129: icmp_seq=36 ttl=64 time=0.344 ms Wed Apr 15 14:35:39 2015 64 bytes from 192.168.122.129: icmp_seq=37 ttl=64 time=0.325 ms Wed Apr 15 14:35:40 2015 64 bytes from 192.168.122.129: icmp_seq=38 ttl=64 time=0.206 ms Wed Apr 15 14:35:41 2015 64 bytes from 192.168.122.129: icmp_seq=39 ttl=64 time=0.141 ms Wed Apr 15 14:35:42 2015 64 bytes from 192.168.122.129: icmp_seq=40 ttl=64 time=0.112 ms Wed Apr 15 14:35:43 2015 64 bytes from 192.168.122.129: icmp_seq=41 ttl=64 time=0.203 ms Wed Apr 15 14:35:44 2015 64 bytes from 192.168.122.129: icmp_seq=42 ttl=64 time=0.142 ms Wed Apr 15 14:35:45 2015 64 bytes from 192.168.122.129: icmp_seq=43 ttl=64 time=0.175 ms Wed Apr 15 14:35:46 2015 64 bytes from 192.168.122.129: icmp_seq=44 ttl=64 time=0.302 ms Wed Apr 15 14:35:47 2015 64 bytes from 192.168.122.129: icmp_seq=45 ttl=64 time=0.192 ms Wed Apr 15 14:35:48 2015 64 bytes from 192.168.122.129: icmp_seq=46 ttl=64 time=0.122 ms Wed Apr 15 14:35:49 2015 64 bytes from 192.168.122.129: icmp_seq=47 ttl=64 time=1.25 ms Wed Apr 15 14:35:50 2015 64 bytes from 192.168.122.129: icmp_seq=48 ttl=64 time=0.306 ms Wed Apr 15 14:36:39 2015 64 bytes from 192.168.122.129: icmp_seq=97 ttl=64 time=0.351 ms Wed Apr 15 14:36:40 2015 64 bytes from 192.168.122.129: icmp_seq=98 ttl=64 time=0.240 ms Wed Apr 15 14:36:41 2015 64 bytes from 192.168.122.129: icmp_seq=99 ttl=64 time=0.244 ms Wed Apr 15 14:36:42 2015 64 bytes from 192.168.122.129: icmp_seq=100 ttl=64 time=0.260 ms --- Additional comment from Marius Vollmer on 2015-05-11 04:45:19 EDT --- I am now seeing similar behavior with NetworkManager 1.0.2 in Fedora 22, but only in our test images, not in my development VM. --- Additional comment from Stef Walter on 2015-05-11 05:35:38 EDT --- This is really bad on servers where the only access to the server is remote. --- Additional comment from Stef Walter on 2015-05-11 05:52:44 EDT --- Nasty work around in Cockpit, that involves breaking cases where an actual outage has occured: https://github.com/cockpit-project/cockpit/pull/2268
I can reproduce this bug with a clean Fedora 22 installation. - Install http://download.fedoraproject.org/pub/fedora/linux/releases/test/22_Beta/Server/x86_64/iso/Fedora-Server-DVD-x86_64-22_Beta.iso into a VM. - Select "minimal install". This ends up with a VM that has NetworkManager 1.0.2-1 and shows the behavior described above.
Fix & explanation attached in bug #1211287 comment #15. We're past the freeze with Fedora 22. Please let me know if you're fine with disabling the strict rp_filter (which should be done in multihomed environments anyway) or you need the fix to be included there.
(In reply to Lubomir Rintel from comment #2) > Fix & explanation attached in bug #1211287 comment #15. Thanks! > We're past the freeze with Fedora 22. Please let me know if you're fine with > disabling the strict rp_filter (which should be done in multihomed > environments anyway) or you need the fix to be included there. I am fine with not fixing this in Fedora 22. The important part is that you know about it and evaluate the impact for the real world use cases. We should now know enough to find a good solution for our test suite as well. Maybe we stop using a multihomed setup (which we only have because of ignorance), maybe we disable rp_filter.
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
this got already fixed on Fedora 22 and newer. There, NMDefaultRouteManager handles adding routes and in case of conflicting metrics, it keeps track which device should get (and keep) a certain route. Closing as fixed.