Bug 112630
| Summary: | 2.6: /sbin/service iptables stop hangs on modprobe -r ipt_state | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 5 | Reporter: | Kaj J. Niemi <kajtzu> | ||||||||
| Component: | kernel | Assignee: | Thomas Graf <tgraf> | ||||||||
| Status: | CLOSED DUPLICATE | QA Contact: | Brian Brock <bbrock> | ||||||||
| Severity: | medium | Docs Contact: | |||||||||
| Priority: | medium | ||||||||||
| Version: | 5.0 | CC: | bmillett, bugs.michael, bugzilla, ctm, curtis, fedora, itxx00, jens, jjmckenzie51, jturner, moremellotron, redhat, risantam, rkhan, stefan.hoelldampf, tadej.j | ||||||||
| Target Milestone: | --- | Keywords: | Reopened | ||||||||
| Target Release: | --- | ||||||||||
| Hardware: | All | ||||||||||
| OS: | Linux | ||||||||||
| Whiteboard: | |||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||
| Doc Text: | Story Points: | --- | |||||||||
| Clone Of: | Environment: | ||||||||||
| Last Closed: | 2006-11-22 19:05:04 UTC | Type: | --- | ||||||||
| Regression: | --- | Mount Type: | --- | ||||||||
| Documentation: | --- | CRM: | |||||||||
| Verified Versions: | Category: | --- | |||||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||
| Embargoed: | |||||||||||
| Attachments: |
|
||||||||||
|
Description
Kaj J. Niemi
2003-12-25 01:07:58 UTC
mm.. I can not reproduce your problem with kernel-2.6.0-1.30 iptables-1.2.9-1 What exactly are you doing for this to happen? rmmod of iptables needs to wait until all active state is gone from the kernel, eg all masqueraded connections need to be terminated etc etc etc. That can take some time...... Unable to reproduce on kernel-2.6.2-1.74, iptables-1.2.9-1.2. I'll close for now. This seems to happen intermittently on 2.6.4+ FC2 kernels again. What really happens is that modprobe/rmmod is eating 99% of the CPU. When upgrading iptables and/or restarting them it is almost always necessary to reboot the server(s) which not only is annoying but a bit dangerous too. ;-) rmmod really can't wait until all active state is gone (I believe this didn't happen on the 2.4 kernels) because that might take more than a reasonable while and really have an impact on how the system works. % rpm -q kernel iptables iptables-ipv6 kernel-2.6.6-1.422 iptables-1.2.9-2.3.1 iptables-ipv6-1.2.9-2.3.1 With kernel 2.4 I've added some comments on the lock-up here: https://bugzilla.netfilter.org/cgi-bin/bugzilla/show_bug.cgi?id=91 In particular, 'cat /proc/net/ip_conntrack' is empty, so I fail to see where masqueraded connections would be involved at all. I'm getting the same problems on my 2.4.21-15.0.3EL kernel. I'm using conn_track and ipv4_forwarding. iptables version 1.2.8-12.3 This happens every now and then but I'm unsure how to proceed with this. In my opinion this is a big bug as sometimes /sbin/service iptables restart results in the system being pretty much unusable until a reboot is issued. Rebooting a firewall or a desktop box just because one changed the packet filter things is a major pain in the ass ;-) Fedora Core 2 % rpm -q kernel iptables kernel-2.6.8-1.521 iptables-1.2.9-2.3.1 % service iptables restart Flushing firewall rules: [ OK ] Setting chains to policy ACCEPT: filter [ OK ] Unloading iptables modules: This is where it hangs... is this still a problem with the latest updates ? Happens unfrequently but is still there... Same here... /etc/init.d/iptables restart called inside /sbin/ifdown-local CPU goes 99% and never returns back till reboot Problem still here...
Fedora Core 3
% rpm -q kernel iptables
kernel-2.6.9-1.667
iptables-1.2.11-3.1
% top
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2713 root 25 0 1592 408 1336 R 99.8 0.0 0:30.29 modprobe
1 root 16 0 3196 564 1408 S 0.0 0.1 0:01.01 init
% /sbin/service iptables stop
Flushing firewall rules: [ OK ]
Setting chains to policy ACCEPT: filter [ OK ]
Unloading iptables modules:
This happens on mys system. Kernel 2.6.10-741_FC3smp and iptables-1.2.11-3.2 I've got this too. Exactly as described by the posters above, appears to hang, uses all the cpu etc. This is a new laptop with a fresh install of FC3, and all updates applied. Currently at: kernel-2.6.10-1.741_FC3, iptables-1.2.11-3.1.FC3. It is completely reproduceable for me. ie. It happens any time there is a network configured. If I boot up with no network configured at all it works fine. As soon as I've done 'ifconfig ethX 192.168.1.1' it breaks, whether ethX is the inbuilt wireless or lan. This clearly isn't to do with outstanding connections. I've never seen this on any of my other machines (desktops or laptops), and they all run FC3, mostly with the same versions of kernel/iptables. I see you've marked this as 'NEEDINFO'. What info do you need? I read the other comments & didn't get any hints. For now I stuck 'return 0' at the start of rmmod_r(), not ideal. I also am seeing this. on an /etc/init.d/iptables stop, reload or whatever, modprobe -r ipt_state hangs like a hat on rack. As the machine I am trying to configure is about 6 hours driving time away, I have to quite literally toss it a reboot every time I need to make a change to iptables. What a pain! FC3, fully patched and updated via yum as of 200501311300 EST. Using the FC3 default whatevers kernel 2.6.10-1.741_FC3 stock build, The same thing happens on my laptop (running updated FC3) but only when NetworkManager daemon is running. Relevant versions: kernel-2.6.10-1.737_FC3 kernel-2.6.10-1.741_FC3 kernel-2.6.10-1.753_FC3 iptables-1.2.11-3.1.FC3 NetworkManager-0.3.3-1.cvs20050119.2.fc3 I've tried with all three kernels and it's always the same. I think it might be worth investigating why NetworkManager is causing 'service iptables stop' to hang. NetworkManager may well be it. The only machine I have which has this problem is also the only one running NetworkManager, though it happens regardless of whether NetworkManager configures the interface or I do it manually. Martin, the bug has been around well before NetworkManager was conceived :) I have the same problem with, albeit with a custom 2.6.10 kernel. Symptoms are the same, ie. "modprobe -r ipt_state" is taking 99% CPU if I change the firewall settings. Before I tried doing this, my PC was hanging at system shutdown at "Stopping iptables:" so that I had to do a hard reset. As reported above, it manifests itself only if a network is configured. I think this is quite serious. It seems to have been an on and off problem, e.g. bug 99057. Sorry, didn't have time to include more info in last comment... kill -9 xxxx # kill modprobe process - has no effect This was not a problem with FC1; I skipped FC2; FC3 was a new install iptables-1.2.11-3.1 module-init-tools-3.1-0.pre5.3 kernel - my own 2.6.10 (but derived from Red Hat .config) The PC is a dell inspiron 8600. The only irregularity I can think of is that I installed linuxant's hsfmodem driver right after FC3 installation and new kernel and I know that changed /etc/modprobe.conf (and modules.conf). hsfmodem-7.18.00.02full-1 *** Bug 143731 has been marked as a duplicate of this bug. *** *** Bug 147386 has been marked as a duplicate of this bug. *** Bug 146753 is the same problem but against NetworkManager and I'm not really sure who's at fault so leaving both bugs open. When I originally reported the bug there was no NetworkManager around. The bug works in the same way as the older bug for FC1 kernels. I'm getting this exact same problem too. [root@laptop ~]# rpm -q kernel iptables NetworkManager kernel-2.6.10-1.760_FC3 kernel-2.6.10-1.766_FC3 iptables-1.2.11-3.1.FC3 NetworkManager-0.3.3-1.cvs20050119.2.fc3 But network manager is NOT running for me. I keep up to date with nightly yum updates. Last time I tried a 'service iptables stop' was a week ago and it worked fine. Today it's not working. I have a work-around to saves me from having to restart. I have two network devices: eth0 (wired ethernet) and eth1 (wireless ethernet). I just do: [root@laptop ~]# ifdown eth1 ... and immediately the hanging process finishes giving an OK/success status. I then bring eth1 bank online with [root@laptop ~]# ifup eth1 Having done this once, I then seem able to do [root@laptop ~]# service iptables stop etc. as much as I want without it hanging again. HOWEVER... if I do a fresh system reboot I then have to put eth1 down again to remove the hang. But only once per reboot it would seem. Jake: Could you provide a copy of your iptables rules from this box? Does the box serve as a gateway? Any NAT? What extra iptables modules do you load? Does the problem still exist even if no traffic has gone through the box (i.e. immediately after reboot)? Same symptom here on fresh FC3. I did two things and problem went away. Maybe it is one of these? - disabled ipv6 - boot to 2.6.10-1.770_FC3smp Problem occured here under kernel build 766. And with only the filter table loaded with conntrack and the standard set of matches. Here's my iptables for you Phil. The machine is my laptop on which I develop websites and allow other computers access to them through the firewall. I've also played around with it as a mail server to find out how to get things working well. I'm not sure about immediately after reboot. How would I find out which extra iptables modules I load? I'm very new to Linux still. [root@laptop ~]# service iptables status Table: filter Chain INPUT (policy ACCEPT) target prot opt source destination RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0 Chain FORWARD (policy ACCEPT) target prot opt source destination RH-Firewall-1-INPUT all -- 0.0.0.0/0 0.0.0.0/0 Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain RH-Firewall-1-INPUT (2 references) target prot opt source destination ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 ACCEPT icmp -- 0.0.0.0/0 0.0.0.0/0 icmp type 255 ACCEPT esp -- 0.0.0.0/0 0.0.0.0/0 ACCEPT ah -- 0.0.0.0/0 0.0.0.0/0 ACCEPT udp -- 0.0.0.0/0 224.0.0.251 udp dpt:5353 ACCEPT udp -- 0.0.0.0/0 0.0.0.0/0 udp dpt:631 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:443 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:10000 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:25 ACCEPT tcp -- 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:80 REJECT all -- 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited Created attachment 112005 [details] proposed patch The attached fixes the problem for me. Full explanation of problem and patch available here: http://lists.netfilter.org/pipermail/netfilter-devel/2005-March/018839.html Testing appreciated... Personnaly, stopping iptables takes all CPU whenever eth0 has "Activate device when computer starts" set to false in system-config-network and eth0 is up. So the workaround for me is just to set this flag to true. One strange thing though is that eth0 is not up if I reboot the system. I have to activate it by hand. My numerous tests to have this issue solved may have caused some collateral damages. BTW, I tried both 2.6.10-1.770_FC3 and 2.6.9-1.667 kernels and I got the same misbehavior. Created attachment 112459 [details]
proposed patch
Attached is a better patch which fixes the core problem -- the prior was a
workaround.
I have had similar problems since I have had NetworkManager installed -- occasional hanging during shutdown of iptables. However lately playing around with my firewall settings, I have noticed that if it hangs, I can stop (or restart) my NetworkManager service and modprobe then completes. Got the same problem with 2.6.11-1.1226_FC4smp. Whenever I try to either shutdown iptables or do a restart 'service iptables restart', the process hangs and modprobe takes 99% of the cpu time. Have to reboot for every iptables change. (In reply to comment #35) > Got the same problem with 2.6.11-1.1226_FC4smp. Whenever I try to either > shutdown iptables or do a restart 'service iptables restart', the process hangs > and modprobe takes 99% of the cpu time. Have to reboot for every iptables > change. Installed Turtelfirewall 1.28 (webmin ...wbm.gz) and it is now possible to apply changes without rebooting, while using the init.d/iptables script for stopping/restarting still loops on modprobe. An update has been released for Fedora Core 3 (kernel-2.6.12-1.1372_FC3) which may contain a fix for your problem. Please update to this new kernel, and report whether or not it fixes your problem. If you have updated to Fedora Core 4 since this bug was opened, and the problem still occurs with the latest updates for that release, please change the version field of this bug to 'fc4'. Thank you. I've since updated my laptop to Fedora Core 4 and I'm happy to report that this problem is now gone (for me, at least). This bug has been automatically closed as part of a mass update. It had been in NEEDINFO state since July 2005. If this bug still exists in current errata kernels, please reopen this bug. There are a large number of inactive bugs in the database, and this is the only way to purge them. Thank you. Created attachment 135480 [details]
Updated information/output for this bug
I would still claim that restarting all networking is not really a good workaround for servers with iptables but then again I guess everybody just outsources the packet filtering to a vendor C or vendor J box instead of keeping them on the servers? *** This bug has been marked as a duplicate of 212839 *** the bug which this is a duplicate of is marked private i got this trouble again #uname -r 2.6.32-279.11.1.el6.x86_64 RHEL6.3 x86_64 |