Bug 1101484
| Summary: | Iptables-services hold on xtables lock even it stopped | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Luwen Su <lsu> |
| Component: | firewalld | Assignee: | Thomas Woerner <twoerner> |
| Status: | CLOSED NOTABUG | QA Contact: | qe-baseos-daemons |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.0 | CC: | ajia, dyuan, jpopelka, lsu, mzhan |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2014-05-28 10:23:35 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
The iptables service (also the ip6tables service) are only locking xtables within the ip*tables-save and ip*tables-restore commands, that are used while starting, restarting, stopping and reloading the services. With these services disabled, this is not happening. There is a conflict in the services for firewalld and ip*tables and ebtables. Could it be that there is something else using ip*tables or ebtables commands directly? Like for example docker? Also with libvirtd, there is a possible condition, where firewalld and libvirtd conflicts in xtables locks. This happens only if firewalld was not active while libvirtd started and firewalld gets activated while libvirtd is still running. Then libvirtd is not using firewalld, but ip*tables calls. Restarting libvirtd after starting firewalld in this scenario is fixing this. (In reply to Thomas Woerner from comment #2) > The iptables service (also the ip6tables service) are only locking xtables > within the ip*tables-save and ip*tables-restore commands, that are used > while starting, restarting, stopping and reloading the services. With these > services disabled, this is not happening. There is a conflict in the > services for firewalld and ip*tables and ebtables. > > Could it be that there is something else using ip*tables or ebtables > commands directly? Like for example docker? > > Also with libvirtd, there is a possible condition, where firewalld and > libvirtd conflicts in xtables locks. This happens only if firewalld was not > active while libvirtd started and firewalld gets activated while libvirtd is > still running. Then libvirtd is not using firewalld, but ip*tables calls. > Restarting libvirtd after starting firewalld in this scenario is fixing this. It should be the libvirt lock it in my machine. After stop libvirt, firewalld starts to work.But after restart libvirtd, it says some module cant' be found Module /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so not accessible libvirtd[2908]: Module /usr/lib64/libvirt/connection-driver/libvirt_driver_storage.so not accessible Module /usr/lib64/libvirt/connection-driver/libvirt_driver_nwfilter.so not accessible Module /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so not accessible Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not accessible Is it a problem or just a limit by design which can use the libvirtd and firewalld at the same time. (In reply to time.su from comment #0) > There is a dependency of libvirt > Removing: > iptables-services x86_64 > Removing for dependencies: > libvirt-daemon-driver-lxc x86_64 > libvirt-daemon-driver-network x86_64 > libvirt-daemon-driver-nwfilter x86_64 I can see couple 'Requires: iptables-ipv6' in libvirt.spec. I guess it's there because previously iptables-ipv6 package contained /sbin/ip6tables. But now because iptables-ipv6 is provided ('Provides: %{name}-ipv6 = 1.4.11.1' in iptables.spec) by iptables-services the libvirt-daemon-driver-* have iptables-services as a dependency. If I'm not mistaken libvirtd needs only /sbin/ip[6]tables and nothing from iptables-services so the 'Requires: iptables-ipv6' could be safely removed from libvirt.spec. Do you agree Thomas ? Can I fill a ticket against libvirt ? (In reply to Jiri Popelka from comment #4) > (In reply to time.su from comment #0) > > There is a dependency of libvirt > > Removing: > > iptables-services x86_64 > > Removing for dependencies: > > libvirt-daemon-driver-lxc x86_64 > > libvirt-daemon-driver-network x86_64 > > libvirt-daemon-driver-nwfilter x86_64 > > I can see couple 'Requires: iptables-ipv6' in libvirt.spec. > I guess it's there because previously iptables-ipv6 package contained > /sbin/ip6tables. > But now because iptables-ipv6 is provided ('Provides: %{name}-ipv6 = > 1.4.11.1' in iptables.spec) by iptables-services the libvirt-daemon-driver-* > have iptables-services as a dependency. > If I'm not mistaken libvirtd needs only /sbin/ip[6]tables and nothing from > iptables-services so the 'Requires: iptables-ipv6' could be safely removed > from libvirt.spec. Do you agree Thomas ? Can I fill a ticket against libvirt > ? Yes, I agree. (In reply to time.su from comment #3) > (In reply to Thomas Woerner from comment #2) > > The iptables service (also the ip6tables service) are only locking xtables > > within the ip*tables-save and ip*tables-restore commands, that are used > > while starting, restarting, stopping and reloading the services. With these > > services disabled, this is not happening. There is a conflict in the > > services for firewalld and ip*tables and ebtables. > > > > Could it be that there is something else using ip*tables or ebtables > > commands directly? Like for example docker? > > > > Also with libvirtd, there is a possible condition, where firewalld and > > libvirtd conflicts in xtables locks. This happens only if firewalld was not > > active while libvirtd started and firewalld gets activated while libvirtd is > > still running. Then libvirtd is not using firewalld, but ip*tables calls. > > Restarting libvirtd after starting firewalld in this scenario is fixing this. > It only happens if firewalld was disabled while booting the machine and if it gets activated while libvirt is already running (without using firewalld). libvirtd decides at start if firewalld will be used or now - it is not able to switch to one or the other while running. > It should be the libvirt lock it in my machine. > After stop libvirt, firewalld starts to work.But after restart libvirtd, it > says > some module cant' be found > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so not > accessible > libvirtd[2908]: Module > /usr/lib64/libvirt/connection-driver/libvirt_driver_storage.so not accessible > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_nwfilter.so not > accessible > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so not > accessible > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not > accessible > > > Is it a problem or just a limit by design which can use the libvirtd and > firewalld at the same time. This is a libvirt problem and not related to firewalld. Missing files or mislabled? (In reply to Thomas Woerner from comment #2) > The iptables service (also the ip6tables service) are only locking xtables > within the ip*tables-save and ip*tables-restore commands Not only ip[6]tables-save/restore, every ip[6]tables command does the locking. http://git.netfilter.org/iptables/commit/?id=93587a04d0f2511e108bbc4d87a8b9d28a5c5dd8 (In reply to Thomas Woerner from comment #6) > (In reply to time.su from comment #3) > > (In reply to Thomas Woerner from comment #2) > > > The iptables service (also the ip6tables service) are only locking xtables > > > within the ip*tables-save and ip*tables-restore commands, that are used > > > while starting, restarting, stopping and reloading the services. With these > > > services disabled, this is not happening. There is a conflict in the > > > services for firewalld and ip*tables and ebtables. > > > > > > Could it be that there is something else using ip*tables or ebtables > > > commands directly? Like for example docker? > > > > > > Also with libvirtd, there is a possible condition, where firewalld and > > > libvirtd conflicts in xtables locks. This happens only if firewalld was not > > > active while libvirtd started and firewalld gets activated while libvirtd is > > > still running. Then libvirtd is not using firewalld, but ip*tables calls. > > > Restarting libvirtd after starting firewalld in this scenario is fixing this. > > > It only happens if firewalld was disabled while booting the machine and if > it gets activated while libvirt is already running (without using > firewalld). libvirtd decides at start if firewalld will be used or now - it > is not able to switch to one or the other while running. > > > It should be the libvirt lock it in my machine. > > After stop libvirt, firewalld starts to work.But after restart libvirtd, it > > says > > some module cant' be found > > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so not > > accessible > > libvirtd[2908]: Module > > /usr/lib64/libvirt/connection-driver/libvirt_driver_storage.so not accessible > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_nwfilter.so not > > accessible > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so not > > accessible > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not > > accessible > > > > > > Is it a problem or just a limit by design which can use the libvirtd and > > firewalld at the same time. > This is a libvirt problem and not related to firewalld. Missing files or > mislabled? Ah..sorry for forgetting check, missing files: # ll /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so ls: cannot access /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so: No such file or directory (In reply to Thomas Woerner from comment #5) > (In reply to Jiri Popelka from comment #4) > > Can I fill a ticket against libvirt ? > Yes, I agree. bug #1101510 (In reply to time.su from comment #8) > (In reply to Thomas Woerner from comment #6) > > (In reply to time.su from comment #3) > > > (In reply to Thomas Woerner from comment #2) > > > > The iptables service (also the ip6tables service) are only locking xtables > > > > within the ip*tables-save and ip*tables-restore commands, that are used > > > > while starting, restarting, stopping and reloading the services. With these > > > > services disabled, this is not happening. There is a conflict in the > > > > services for firewalld and ip*tables and ebtables. > > > > > > > > Could it be that there is something else using ip*tables or ebtables > > > > commands directly? Like for example docker? > > > > > > > > Also with libvirtd, there is a possible condition, where firewalld and > > > > libvirtd conflicts in xtables locks. This happens only if firewalld was not > > > > active while libvirtd started and firewalld gets activated while libvirtd is > > > > still running. Then libvirtd is not using firewalld, but ip*tables calls. > > > > Restarting libvirtd after starting firewalld in this scenario is fixing this. > > > > > It only happens if firewalld was disabled while booting the machine and if > > it gets activated while libvirt is already running (without using > > firewalld). libvirtd decides at start if firewalld will be used or now - it > > is not able to switch to one or the other while running. > > > > > It should be the libvirt lock it in my machine. > > > After stop libvirt, firewalld starts to work.But after restart libvirtd, it > > > says > > > some module cant' be found > > > > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so not > > > accessible > > > libvirtd[2908]: Module > > > /usr/lib64/libvirt/connection-driver/libvirt_driver_storage.so not accessible > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_nwfilter.so not > > > accessible > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so not > > > accessible > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not > > > accessible > > > > > > > > > Is it a problem or just a limit by design which can use the libvirtd and > > > firewalld at the same time. > > This is a libvirt problem and not related to firewalld. Missing files or > > mislabled? > > Ah..sorry for forgetting check, missing files: > # ll /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so > ls: cannot access > /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so: No such file > or directory Either your filesystem is failing or there is something strange going on in your system. This is definitely not a firewalld issue. (In reply to Thomas Woerner from comment #10) > (In reply to time.su from comment #8) > > (In reply to Thomas Woerner from comment #6) > > > (In reply to time.su from comment #3) > > > > (In reply to Thomas Woerner from comment #2) > > > > > The iptables service (also the ip6tables service) are only locking xtables > > > > > within the ip*tables-save and ip*tables-restore commands, that are used > > > > > while starting, restarting, stopping and reloading the services. With these > > > > > services disabled, this is not happening. There is a conflict in the > > > > > services for firewalld and ip*tables and ebtables. > > > > > > > > > > Could it be that there is something else using ip*tables or ebtables > > > > > commands directly? Like for example docker? > > > > > > > > > > Also with libvirtd, there is a possible condition, where firewalld and > > > > > libvirtd conflicts in xtables locks. This happens only if firewalld was not > > > > > active while libvirtd started and firewalld gets activated while libvirtd is > > > > > still running. Then libvirtd is not using firewalld, but ip*tables calls. > > > > > Restarting libvirtd after starting firewalld in this scenario is fixing this. > > > > > > > It only happens if firewalld was disabled while booting the machine and if > > > it gets activated while libvirt is already running (without using > > > firewalld). libvirtd decides at start if firewalld will be used or now - it > > > is not able to switch to one or the other while running. > > > > > > > It should be the libvirt lock it in my machine. > > > > After stop libvirt, firewalld starts to work.But after restart libvirtd, it > > > > says > > > > some module cant' be found > > > > > > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so not > > > > accessible > > > > libvirtd[2908]: Module > > > > /usr/lib64/libvirt/connection-driver/libvirt_driver_storage.so not accessible > > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_nwfilter.so not > > > > accessible > > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_qemu.so not > > > > accessible > > > > Module /usr/lib64/libvirt/connection-driver/libvirt_driver_lxc.so not > > > > accessible > > > > > > > > > > > > Is it a problem or just a limit by design which can use the libvirtd and > > > > firewalld at the same time. > > > This is a libvirt problem and not related to firewalld. Missing files or > > > mislabled? > > > > Ah..sorry for forgetting check, missing files: > > # ll /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so > > ls: cannot access > > /usr/lib64/libvirt/connection-driver/libvirt_driver_network.so: No such file > > or directory > > Either your filesystem is failing or there is something strange going on in > your system. This is definitely not a firewalld issue. Yeah, the libvirtd still started after remove iptables-service and some of its driver package. And after re-installed and restarted, there are lots of iptables errors about the rule that libvirtd uses. like firewalld: 2014-05-28 17:33:58 ERROR: COMMAND_FAILED: '/sbin/iptables --table filter --delete INPUT --in-interface virbr0 --protocol tcp --destination-port 67 --jump ACCEPT' failed: iptables: Bad rule (does a matching rule exist in that chain? libvirt tries first to cleanup the rule set it will create in the next step. If there are COMMAND_FAILED messages with --delete only, then you can ignore them. If there are others also, then there is most likely a problem. Check, all are COMMAND_FAILED messages with --delete. So the only thing left is about the libvirt's issue Bug 1101510 - no need to require iptables-ipv6. thanks your help! You are welcome. Closing this as NOT A BUG. |
Description of problem: When install both iptables-services and firewalld, firewalld will fail to get xtables due error "Another app is currently holding the xtables lock" Version-Release number of selected component (if applicable): iptables-1.4.21-13.el7.x86_64 iptables-services-1.4.21-13.el7.x86_64 firewalld-0.3.9-7.el7.noarch kernel-3.10.0-123.el7.x86_64 How reproducible: 100% Steps to Reproduce: 1.Make sure iptables-services installed and not started #systemctl status iptables-services.status iptables.service - IPv4 firewall with iptables Loaded: loaded (/usr/lib/systemd/system/iptables.service; disabled) Active: inactive (dead) 2.Try to start firewalld #firewalld --debug 1 --nofork 2014-05-27 17:22:53 DEBUG1: start() 2014-05-27 17:22:53 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/firewall/server/decorators.py", line 40, in handle_exceptions return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/firewall/server/firewalld.py", line 78, in start return self.fw.start() File "/usr/lib/python2.7/site-packages/firewall/core/fw.py", line 246, in start self._flush() File "/usr/lib/python2.7/site-packages/firewall/core/fw.py", line 536, in _flush self._ip4tables.flush() File "/usr/lib/python2.7/site-packages/firewall/core/ipXtables.py", line 180, in flush self.__run([ "-t", table, flag ]) File "/usr/lib/python2.7/site-packages/firewall/core/ipXtables.py", line 136, in __run " ".join(_args), ret)) ValueError: '/sbin/iptables -t filter -Z' failed: Another app is currently holding the xtables lock. Perhaps you want to use the -w option? 2014-05-27 17:22:53 DEBUG1: zone.addInterface('', 'switch') 2014-05-27 17:22:53 ERROR: INVALID_ZONE 2014-05-27 17:22:53 DEBUG1: zone.addInterface('', 'eno1') 2014-05-27 17:22:53 ERROR: INVALID_ZONE 3.Remove iptables-services and reboot the host There is a dependency of libvirt Removing: iptables-services x86_64 1.4.21-13.el7 @RHEL-Server-7 23 k Removing for dependencies: libvirt-daemon-driver-lxc x86_64 1.1.1-29.el7 @RHEL-Server-7 228 k libvirt-daemon-driver-network x86_64 1.1.1-29.el7 @RHEL-Server-7 135 k libvirt-daemon-driver-nwfilter x86_64 1.1.1-29.el7 @RHEL-Server-7 164 k 4.After reboot Skip the long debug log, firewalld starts to work 2014-05-27 17:28:23 DEBUG1: config.ZoneAdded('work') 2014-05-27 17:28:23 DEBUG1: zone.addInterface('', 'switch') 2014-05-27 17:28:23 DEBUG1: zone.InterfaceAdded('public', 'switch') 2014-05-27 17:28:23 DEBUG1: zone.addInterface('', 'eno1') 2014-05-27 17:28:23 DEBUG1: zone.InterfaceAdded('public', 'eno1') Additional info: