Bug 448889

Summary: NM enters endless loop, floods /var/log/messages
Product: [Fedora] Fedora Reporter: Rui Matos <tiagomatos>
Component: NetworkManagerAssignee: Dan Williams <dcbw>
Status: CLOSED NEXTRELEASE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: low    
Version: 9CC: dcbw, emmanuel, iny, lamp19, matt, smooge, spider, wtogami
Target Milestone: ---   
Target Release: ---   
Hardware: i386   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-09-28 18:45:40 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Rui Matos 2008-05-29 10:39:43 UTC
On Fedora 8 with these versions:

kernel-2.6.25.3-18.fc9.x86_64
wpa_supplicant-0.6.3-5.fc9.x86_64
NetworkManager-0.7.0-0.9.3.svn3675.fc9.x86_64

on a Latitude D630 with an iwl3945 after a reboot with the killswitch activated
(i.e. wireless off) and subsequent wireless activation with the killswitch, NM
entered an endless loop flooding /var/log/messages with:

May 29 11:04:05 hive NetworkManager:
nm_supplicant_interface_get_connection_state: assertion
`NM_IS_SUPPLICANT_INTERFACE (self)' failed
May 29 11:04:05 hive NetworkManager: nm_supplicant_interface_request_scan:
assertion `NM_IS_SUPPLICANT_INTERFACE (self)' failed

Comment 1 Stephen John Smoogen 2008-06-27 17:46:10 UTC
I am seeing this regularly on a Dell620 with Fedora 9.

[root@localhost log]# rpm -q NetworkManager wpa_supplicant kernel
NetworkManager-0.7.0-0.9.4.svn3675.fc9.i386
wpa_supplicant-0.6.3-6.fc9.i386
kernel-2.6.25.4-30.fc9.i686
kernel-2.6.25.6-55.fc9.i686

System becomes completely irresponsive as rsyslogd takes up 100%+ of the CPU.

The infinite loop starts as:

Jun 27 11:12:57 localhost NetworkManager: <WARN>  killswitch_getpower_reply():
Error getting killswitch power: Did not receive a reply. Possible causes
include: the remote application did not send a reply, t
he message bus security policy blocked the reply, the reply timeout expired, or
the network connection was broken..
Jun 27 11:12:57 localhost kernel: ACPI: PCI Interrupt 0000:0c:00.0[A] -> GSI 17
(level, low) -> IRQ 17
Jun 27 11:12:58 localhost kernel: iwl3945: Radio disabled by HW RF Kill switch
Jun 27 11:12:58 localhost kernel: ACPI: PCI interrupt for device 0000:0c:00.0
disabled
Jun 27 11:12:58 localhost NetworkManager: <info>  Wireless now enabled by radio
killswitch
Jun 27 11:12:58 localhost NetworkManager: <info>  (wlan0): bringing up device.
Jun 27 11:12:58 localhost NetworkManager: <WARN>  nm_device_hw_bring_up():
(wlan0): device not up after timeout!
Jun 27 11:12:58 localhost NetworkManager: <info>  (wlan0): device state change:
2 -> 3
Jun 27 11:12:58 localhost NetworkManager:
nm_supplicant_interface_get_connection_state: assertion
`NM_IS_SUPPLICANT_INTERFACE (self)' failed
Jun 27 11:12:58 localhost NetworkManager: nm_supplicant_interface_request_scan:
assertion `NM_IS_SUPPLICANT_INTERFACE (self)' failed

Wireless is still turned physically off on the system. Problem caused 10,633,997
messages to /var/log/messages in 15 minutes. Similar problem occurred 1 week ago
at FudCon and could only be recovered with a power off as screensaver could not
get any CPU to allow for login. Problem filled / in 40 minutes as I was at lunch.

Problem could be considered potential denial of service. 

Please let me know what I can do to debug.


Comment 2 Rui Matos 2008-06-27 17:54:32 UTC
Hmm, I said Fedora 8 but that was obviously a typo, it's Fedora 9.

It continues to happen to me, now with

kernel-2.6.25.6-55.fc9.x86_64
NetworkManager-0.7.0-0.10.svn3747.fc9.x86_64


Comment 3 Dan Williams 2008-06-28 02:56:54 UTC
Stephen; can you check if there are any AVCs having to do with NM talking to HAL
or HAL running the killswitch power callout?

For various reasons, we can't assume that an error returned by GetPower() means
the radio is disabled.  We'll have to hack around stupid drivers (which are
_why_ we can't assume this) by string-matching the error message and assume the
radio is disabled if the GetPower() call times out.  Additionally, we'll also
handle failures of setting IFF_UP by keeping the device deactivated.

Comment 4 Stephen John Smoogen 2008-06-28 19:41:04 UTC
selinux avc's or some other avc's.. :).

Comment 5 Rui Matos 2008-06-29 11:43:55 UTC
(In reply to comment #3)
> Stephen; can you check if there are any AVCs having to do with NM talking to HAL
> or HAL running the killswitch power callout?

There are no AVCs logged about this. FWIW I currently run fedora with selinux in
permissive mode.

Comment 6 Dan Williams 2008-06-29 16:03:18 UTC
Did this happen for either of you when you booted up the laptop with the radio
killed _at boot_, or did it happen when you were already running and turned the
radio off via the killswitch?  The distinction could be important while fixing
this issue.

Comment 7 Rui Matos 2008-06-29 18:05:24 UTC
(In reply to comment #6)
> Did this happen for either of you when you booted up the laptop with the radio
> killed _at boot_, or did it happen when you were already running and turned the
> radio off via the killswitch?  The distinction could be important while fixing
> this issue.

Never thought about that. But after testing, it only happens when the radio is
killed at boot indeed.

Comment 8 Stephen John Smoogen 2008-06-29 18:12:40 UTC
Same here. I had turned off the kill switch and warm rebooted and after some
time the loop occured.

Comment 9 Dan Williams 2008-07-02 23:20:49 UTC
*** Bug 452945 has been marked as a duplicate of this bug. ***

Comment 10 Fedora Update System 2008-07-24 22:41:41 UTC
NetworkManager-vpnc-0.7.0-0.10.svn3846.fc9,NetworkManager-openvpn-0.7.0-15.svn3846.fc9,NetworkManager-0.7.0-0.11.svn3846.fc9 has been submitted as an update for Fedora 9

Comment 11 Stephen John Smoogen 2008-07-24 22:58:31 UTC
Where can I test this in bodhi? I just came back from a meeting with a 500 MB
/var/log/messages and the poor cpu fan running like no tomorrow. 

Thanks.

Comment 12 Rui Matos 2008-07-24 23:05:58 UTC
It seems to work fine for me with svn3830. Haven't tried with svn3846 yet.


Comment 13 Stephen John Smoogen 2008-07-28 21:12:42 UTC
Tried downloading from koji to test.. package requires newer
NetworkManager-gnome to install.

Comment 14 Stephen John Smoogen 2008-07-28 21:23:19 UTC
Please excuse my idiocy.. didn't see it in bodhi til after I logged into it.. 

Comment 15 Fedora Update System 2008-07-30 19:59:48 UTC
NetworkManager-0.7.0-0.11.svn3846.fc9, NetworkManager-vpnc-0.7.0-0.10.svn3846.fc9, NetworkManager-openvpn-0.7.0-15.svn3846.fc9 has been pushed to the Fedora 9 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager NetworkManager-vpnc NetworkManager-openvpn'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F9/FEDORA-2008-6761

Comment 16 Dan Williams 2008-08-06 13:01:41 UTC
*** Bug 457986 has been marked as a duplicate of this bug. ***

Comment 17 Fedora Update System 2008-08-12 02:29:40 UTC
NetworkManager-openvpn-0.7.0-15.svn3930.fc9,NetworkManager-vpnc-0.7.0-0.10.svn3928.fc9,NetworkManager-0.7.0-0.11.svn3930.fc9 has been submitted as an update for Fedora 9

Comment 18 Fedora Update System 2008-08-12 02:34:27 UTC
NetworkManager-openvpn-0.7.0-15.svn3930.fc8,NetworkManager-vpnc-0.7.0-0.10.svn3928.fc8,NetworkManager-0.7.0-0.11.svn3930.fc8 has been submitted as an update for Fedora 8

Comment 19 Dan Williams 2008-08-25 17:00:58 UTC
*** Bug 460040 has been marked as a duplicate of this bug. ***

Comment 20 Fedora Update System 2008-08-29 14:56:13 UTC
NetworkManager-openvpn-0.7.0-15.svn4027.fc9,NetworkManager-vpnc-0.7.0-0.10.svn4024.fc9,NetworkManager-0.7.0-0.11.svn4022.fc9 has been submitted as an update for Fedora 9.
http://admin.fedoraproject.org/updates/NetworkManager-openvpn-0.7.0-15.svn4027.fc9,NetworkManager-vpnc-0.7.0-0.10.svn4024.fc9,NetworkManager-0.7.0-0.11.svn4022.fc9

Comment 21 Fedora Update System 2008-09-02 20:41:23 UTC
NetworkManager-0.7.0-0.11.svn4022.fc8,NetworkManager-openvpn-0.7.0-15.svn4027.fc8,NetworkManager-vpnc-0.7.0-0.10.svn4024.fc8,NetworkManager-pptp-0.7.0-0.10.svn4027.fc8 has been submitted as an update for Fedora 8.
http://admin.fedoraproject.org/updates/NetworkManager-0.7.0-0.11.svn4022.fc8,NetworkManager-openvpn-0.7.0-15.svn4027.fc8,NetworkManager-vpnc-0.7.0-0.10.svn4024.fc8,NetworkManager-pptp-0.7.0-0.10.svn4027.fc8

Comment 23 Fedora Update System 2008-09-10 06:36:19 UTC
NetworkManager-pptp-0.7.0-0.10.svn4027.fc9, NetworkManager-openvpn-0.7.0-15.svn4027.fc9, NetworkManager-vpnc-0.7.0-0.10.svn4024.fc9, NetworkManager-0.7.0-0.11.svn4022.fc9 has been pushed to the Fedora 9 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager-pptp NetworkManager-openvpn NetworkManager-vpnc NetworkManager'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F9/FEDORA-2008-7375

Comment 24 Fedora Update System 2008-09-10 06:54:03 UTC
NetworkManager-0.7.0-0.11.svn4022.fc8, NetworkManager-openvpn-0.7.0-15.svn4027.fc8, NetworkManager-vpnc-0.7.0-0.10.svn4024.fc8, NetworkManager-pptp-0.7.0-0.10.svn4027.fc8 has been pushed to the Fedora 8 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update NetworkManager NetworkManager-openvpn NetworkManager-vpnc NetworkManager-pptp'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/F8/FEDORA-2008-7484

Comment 25 Fedora Update System 2008-09-28 18:42:19 UTC
NetworkManager-0.7.0-0.11.svn4022.fc8, NetworkManager-openvpn-0.7.0-15.svn4027.fc8, NetworkManager-vpnc-0.7.0-0.10.svn4024.fc8, NetworkManager-pptp-0.7.0-0.10.svn4027.fc8 has been pushed to the Fedora 8 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 26 Fedora Update System 2008-09-28 18:44:28 UTC
NetworkManager-pptp-0.7.0-0.10.svn4027.fc9, NetworkManager-openvpn-0.7.0-15.svn4027.fc9, NetworkManager-vpnc-0.7.0-0.10.svn4024.fc9, NetworkManager-0.7.0-0.11.svn4022.fc9 has been pushed to the Fedora 9 stable repository.  If problems still persist, please make note of it in this bug report.