Bug 460410 - xm create fails to add vif PHYSDEV match rules for a domU with multiple network interfaces
Summary: xm create fails to add vif PHYSDEV match rules for a domU with multiple netw...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: xen
Version: 5.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Jiri Denemark
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 477162
TreeView+ depends on / blocked
 
Reported: 2008-08-28 05:38 UTC by Michael Kearey
Modified: 2018-10-20 03:07 UTC (History)
6 users (show)

Fixed In Version: xen-3.0.3-87.el5
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 503043 (view as bug list)
Environment:
Last Closed: 2009-09-02 10:07:28 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
patch to resolve iptables contention in xen network scripts (3.06 KB, patch)
2008-08-28 05:40 UTC, Michael Kearey
no flags Details | Diff
Serialize iptables calls in hotplug scripts (818 bytes, patch)
2009-05-27 16:42 UTC, Jiri Denemark
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2009:1328 0 normal SHIPPED_LIVE xen bug fix and enhancement update 2009-09-01 10:32:30 UTC

Description Michael Kearey 2008-08-28 05:38:05 UTC
Description of problem:
When a domU is configured to have multiple network interfaces, it may fail to create the iptables rules to allow each of the domU's interfaces to bridge the dom0's firewall.

It seems to be a resources issue with iptables and the Xen scripts for networking - multiple network interfaces means that the iptables command is run in a very quick sequence or even concurrently to add vif PHYSDEV match rules resulting in resource contention.

Version-Release number of selected component (if applicable):


How reproducible:
A domU with 2 network interfaces I observed the problem 50% of the time.
A domU with 3 network interfaces I observed the problem 100% of the time 

Steps to Reproduce:
1. Set up a domU guest with more than one network interface - for example edit the /etc/xen/domuGuest  file and add line :
vif = [ '', '', '' ]

2. Start the domU with xm create 

  
Actual results:

After starting the domU we observe the iptables firewall rules :

ACCEPT     all  --  anywhere             anywhere            PHYSDEV match --physdev-in vif1.0
ACCEPT     all  --  anywhere             anywhere            PHYSDEV match --physdev-in vif1.1

And in the /var/log/messages we see:

iptables -A FORWARD -m physdev --physdev-in BLAH BLAH -j ACCEPT failed.
If you are using iptables, this may affect networking for guest domains.

If we strace the iptables command itself we see:

5087  setsockopt(3, SOL_IP, 0x40 /* IP_??? */, "filter.., 3840) = -1 EAGAIN (Resource temporarily unavailable)
5087  write(2, "iptables: Unknown error 18446744073709551615n", 45) = 45
5087  exit_group(1)                     = ?

Expected results:
We should see the xm create command for the domU run completely, create the domain and the PHYSDEV match rules for each vif should be present. Also no errors in /var/log/messages  

Additional info:

A patch to resolve the problem is attached. I adds a simple locking mechanism around the iptables command as it is run by the xen network scripts. It ensures that the iptables commands are  run one at a time.

Comment 1 Michael Kearey 2008-08-28 05:40:32 UTC
Created attachment 315173 [details]
patch to resolve iptables contention in xen network scripts

Comment 5 Jiri Denemark 2009-05-27 16:34:33 UTC
It's actually a bug in iptables, IMHO. If you run something like

for ((i=0;i<100;i++)); do iptables -A FORWARD -m physdev --physdev-in vif7.$i -j ACCEPT & done

quite a few of the calls will fail. Should we clone this bug for iptables?

Anyway we can easily workaround it by serializing iptables calls...

Comment 6 Jiri Denemark 2009-05-27 16:42:11 UTC
Created attachment 345650 [details]
Serialize iptables calls in hotplug scripts

Comment 7 Michal Novotny 2009-06-08 07:52:42 UTC
Fix built into xen-3.0.3-87.el5

Comment 9 Chris Ward 2009-07-03 18:07:20 UTC
~~ Attention - RHEL 5.4 Beta Released! ~~

RHEL 5.4 Beta has been released! There should be a fix present in the Beta release that addresses this particular request. Please test and report back results here, at your earliest convenience. RHEL 5.4 General Availability release is just around the corner!

If you encounter any issues while testing Beta, please describe the issues you have encountered and set the bug into NEED_INFO. If you encounter new issues, please clone this bug to open a new issue and request it be reviewed for inclusion in RHEL 5.4 or a later update, if it is not of urgent severity.

Please do not flip the bug status to VERIFIED. Only post your verification results, and if available, update Verified field with the appropriate value.

Questions can be posted to this bug or your customer or partner representative.

Comment 10 zhanghaiyan 2009-07-30 07:00:56 UTC
Verified on xen-3.0.3.91.el5 PASS

Comment 12 errata-xmlrpc 2009-09-02 10:07:28 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2009-1328.html


Note You need to log in before you can comment on or make changes to this bug.