Red Hat Bugzilla – Bug 460410
xm create fails to add vif PHYSDEV match rules for a domU with multiple network interfaces
Last modified: 2010-10-23 00:07:20 EDT
Description of problem:
When a domU is configured to have multiple network interfaces, it may fail to create the iptables rules to allow each of the domU's interfaces to bridge the dom0's firewall.
It seems to be a resources issue with iptables and the Xen scripts for networking - multiple network interfaces means that the iptables command is run in a very quick sequence or even concurrently to add vif PHYSDEV match rules resulting in resource contention.
Version-Release number of selected component (if applicable):
A domU with 2 network interfaces I observed the problem 50% of the time.
A domU with 3 network interfaces I observed the problem 100% of the time
Steps to Reproduce:
1. Set up a domU guest with more than one network interface - for example edit the /etc/xen/domuGuest file and add line :
vif = [ '', '', '' ]
2. Start the domU with xm create
After starting the domU we observe the iptables firewall rules :
ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-in vif1.0
ACCEPT all -- anywhere anywhere PHYSDEV match --physdev-in vif1.1
And in the /var/log/messages we see:
iptables -A FORWARD -m physdev --physdev-in BLAH BLAH -j ACCEPT failed.
If you are using iptables, this may affect networking for guest domains.
If we strace the iptables command itself we see:
5087 setsockopt(3, SOL_IP, 0x40 /* IP_??? */, "filter.., 3840) = -1 EAGAIN (Resource temporarily unavailable)
5087 write(2, "iptables: Unknown error 18446744073709551615n", 45) = 45
5087 exit_group(1) = ?
We should see the xm create command for the domU run completely, create the domain and the PHYSDEV match rules for each vif should be present. Also no errors in /var/log/messages
A patch to resolve the problem is attached. I adds a simple locking mechanism around the iptables command as it is run by the xen network scripts. It ensures that the iptables commands are run one at a time.
Created attachment 315173 [details]
patch to resolve iptables contention in xen network scripts
It's actually a bug in iptables, IMHO. If you run something like
for ((i=0;i<100;i++)); do iptables -A FORWARD -m physdev --physdev-in vif7.$i -j ACCEPT & done
quite a few of the calls will fail. Should we clone this bug for iptables?
Anyway we can easily workaround it by serializing iptables calls...
Created attachment 345650 [details]
Serialize iptables calls in hotplug scripts
Fix built into xen-3.0.3-87.el5
~~ Attention - RHEL 5.4 Beta Released! ~~
RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!
If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.
Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.
Questions can be posted to this bug or your customer or partner representative.
Verified on xen-184.108.40.206.el5 PASS
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.