Bug 735098

Summary: modprobe hangs at 100% CPU usage when changing firewall rules
Product: [Fedora] Fedora Reporter: Juan Orti <jorti>
Component: module-init-toolsAssignee: Jon Masters <jcm>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 15CC: andrea.veri, cacr-support, huangjs, jcm, jonathan, jorti, murple
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-08-07 19:52:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
dmesg output
none
/etc/sysconfig/iptables
none
lsmod output
none
lspci -v -v output
none
ps output none

Description Juan Orti 2011-09-01 13:00:27 UTC
User-Agent:       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0.1) Gecko/20100101 Firefox/6.0.1

I try to open some ports in the firewall using the utility system-config-firewall or in text mode system-config-firewall-tui and when I apply the changes the system goes at 100% CPU usage by the modprobe process.

[root@neon ~]# ps -ef | grep 5602
root      5602     1 99 14:07 ?        00:08:42 modprobe -r xt_state

Meanwhile, I can see the firewall rules have been wiped out:

[root@neon ~]# iptables -L -v
Chain INPUT (policy ACCEPT 8762 packets, 847K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain OUTPUT (policy ACCEPT 8661 packets, 1095K bytes)
 pkts bytes target     prot opt in     out     source               destination

I enter a kill or kill -9 but modprobe don't die!

Reproducible: Always

Steps to Reproduce:
1. Modify the firewall rules with system-config-firewall-tui
2. Apply
Actual Results:  
modprobe consumes 100% CPU

Expected Results:  
new firewall rules applied

Comment 1 Juan Orti 2011-09-01 14:56:25 UTC
Additional info:

I edited firewall rules some months ago when installed Fedora 15 and had no problems. Other processes I see around:

root         2     0  0 15:31 ?        00:00:00 [kthreadd]
root       280     2  0 15:31 ?        00:00:00 [kworker/u:4]
root      2970   280  0 16:34 ?        00:00:00 [kworker/u:4]
root      2824     1 98 16:29 ?        00:13:55 modprobe -r xt_state
root      2969     1  0 16:34 ?        00:00:00 iptables-restore /etc/sysconfig/iptables
root      2971  2970  0 16:34 ?        00:00:01 /sbin/modprobe -q -- ipt_state


Package versions:

kernel-2.6.40.3-0.fc15.x86_64
module-init-tools-3.16-2.fc15.x86_64
iptables-1.4.10-2.fc15.x86_64
iptables-ipv6-1.4.10-2.fc15.x86_64
system-config-firewall-base-1.2.29-4.fc15.noarch
system-config-firewall-tui-1.2.29-4.fc15.noarch
system-config-firewall-1.2.29-4.fc15.noarch

Comment 2 Juan Orti 2011-09-01 14:57:29 UTC
Created attachment 521022 [details]
dmesg output

Comment 3 Juan Orti 2011-09-01 14:59:29 UTC
Created attachment 521023 [details]
/etc/sysconfig/iptables

Comment 4 Juan Orti 2011-09-01 14:59:58 UTC
Created attachment 521024 [details]
lsmod output

Comment 5 Juan Orti 2011-09-01 15:00:27 UTC
Created attachment 521025 [details]
lspci -v -v output

Comment 6 Juan Orti 2011-09-01 15:00:57 UTC
Created attachment 521026 [details]
ps output

Comment 7 Jon Masters 2011-11-29 17:24:54 UTC
Changing to correct email address. Apologies for the delay.

Comment 8 Jon Masters 2011-12-13 11:10:31 UTC
Is this is reproducible regularly? Have you seen it only on one system? Do you see this on any other machines?

The 100% load is likely not modprobe, but instead the kernel-side of the removal. I do wonder why it's removing the modules, since although that ought to work it isn't recommended.

Comment 9 Juan Orti 2011-12-13 11:51:02 UTC
I'm sorry but I can't reproduce it again. It happened to me only in one system, but now it's updated to Fedora 16 and working ok.

Comment 10 Fedora End Of Life 2012-08-07 19:52:10 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 11 Andrea Veri 2013-10-25 17:06:59 UTC
Sadly we triggered this bug on RHEL 5.10 with kernel: 2.6.18-371.1.2.el5. Modprobe went stuck when removing the xt_state module, the CPU went at 100% and the relevant modprobe process was not killable. In addition shutting down iptables became impossible, the only fix we could apply was rebooting the whole machine.

Comment 12 caltech-cacr 2014-01-02 19:38:19 UTC
I've run into this on RHEL 5.10 as well; slightly newer kernel version (2.6.18-371.3.1.el5).  But I've seen it multiple times now; after making a change to /etc/sysconfig/iptables, then doing "/etc/init.d/iptables restart", the "modprobe -r xt_state" sits there forever, cranking up minutes, then tens of minutes, of CPU time; and the "iptables restart" never completes.  It requires a reboot; and sometimes, the system won't even reboot, and a reset (or even power cycle) is needed.  Hope this gets fixed soon.  Even though the status is listed as "CLOSED WONTFIX" because the problem was reported with regard to Fedora 15, which is now EOL'ed, presumably the people responsible for the RHEL5 maintenance are aware of this bug ...

Comment 13 Craig 2014-08-14 19:35:57 UTC
We have this happening on CentOS 6.5, but adding this info here as well as RHEL/CentOS bugzilla since it's surely the same bug. The machine runs KVMs with bridged interfaces... other pages about this issue also mention KVM/VirtualBox and bridges being involved. We've got these kernel stack traces happening, which may help:

Aug 14 14:46:34 head kernel: INFO: task modprobe:9982 blocked for more than 120 seconds.
Aug 14 14:46:34 head kernel:      Not tainted 2.6.32-431.23.3.el6.x86_64 #1
Aug 14 14:46:34 head kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 14 14:46:34 head kernel: modprobe      D 0000000000000009     0  9982   9980 0x00000006
Aug 14 14:46:34 head kernel: ffff880d5ad9de48 0000000000000086 0000000000000000 ffffffff810129de
Aug 14 14:46:34 head kernel: ffff880d5ad9ddd8 ffffffff81123781 ffffffff8100bb8e ffff880d5ad9de48
Aug 14 14:46:34 head kernel: ffff880fded1a638 ffff880d5ad9dfd8 000000000000fbc8 ffff880fded1a638
Aug 14 14:46:34 head kernel: Call Trace:
Aug 14 14:46:34 head kernel: [<ffffffff810129de>] ? copy_user_generic+0xe/0x20
Aug 14 14:46:34 head kernel: [<ffffffff81123781>] ? probe_kernel_write+0x41/0x70
Aug 14 14:46:34 head kernel: [<ffffffff8100bb8e>] ? apic_timer_interrupt+0xe/0x20
Aug 14 14:46:34 head kernel: [<ffffffff8105546b>] ? mutex_spin_on_owner+0x9b/0xc0
Aug 14 14:46:34 head kernel: [<ffffffff8152a36e>] __mutex_lock_slowpath+0x13e/0x180
Aug 14 14:46:34 head kernel: [<ffffffffa0023000>] ? ip_tables_init+0x0/0xb0 [ip_tables]
Aug 14 14:46:34 head kernel: [<ffffffff8152a20b>] mutex_lock+0x2b/0x50
Aug 14 14:46:34 head kernel: [<ffffffffa0023000>] ? ip_tables_init+0x0/0xb0 [ip_tables]
Aug 14 14:46:34 head kernel: [<ffffffff81457e4d>] register_pernet_subsys+0x1d/0x50
Aug 14 14:46:34 head kernel: [<ffffffffa0023015>] ip_tables_init+0x15/0xb0 [ip_tables]
Aug 14 14:46:34 head kernel: [<ffffffff8100204c>] do_one_initcall+0x3c/0x1d0
Aug 14 14:46:34 head kernel: [<ffffffff810bc361>] sys_init_module+0xe1/0x250
Aug 14 14:46:34 head kernel: [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b

Comment 14 Chih-Hsueh "Josh" HUANG 2016-05-12 03:59:30 UTC
Reproduced this on CentOS release 6.6 (Final)