Bug 642336

Summary: DHCP broken for "Isolated Network" Guests with virtio network devices
Product: Red Hat Enterprise Linux 6 Reporter: Brad Durrow <brad+rhbz>
Component: qemu-kvmAssignee: Virtualization Maintenance <virt-maint>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: berrange, laine, mkenneth, tburke, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-11-29 13:52:39 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Brad Durrow 2010-10-12 15:52:34 UTC 
Description of problem:  A CentOS 5.5 client on Red Hat Enterprise Linux Server release 6.0 Beta (Santiago) host can not get a DHCP address on an "isolated network" when the client's network adapter is configured as virtio.  I have no issue if I change network adapter type to rtl8139 it does work.  I have not tested with RHEL5.5 guest because I don't want to burn an entitlement.

Version-Release number of selected component (if applicable):
Red Hat Enterprise Linux Server release 6.0 Beta (Santiago)
Linux vdev.foo.priv 2.6.32-44.2.el6.x86_64 #1 SMP Wed Jul 21 12:48:32 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux
qemu-kvm-tools-0.12.1.2-2.90.el6.x86_64
qemu-kvm-0.12.1.2-2.90.el6.x86_64
gpxe-roms-qemu-0.9.7-6.3.el6.noarch
qemu-img-0.12.1.2-2.90.el6.x86_64
dnsmasq-2.48-4.el6.x86_64

How reproducible:

Steps to Reproduce:
1. Add additional interface for a 5.5 guest
2. Choose Isolated Network and virtio
3. Boot 5.5 guest
  
Actual results:
Oct 12 09:29:44 vdev dnsmasq-dhcp[6478]: DHCPDISCOVER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:29:44 vdev dnsmasq-dhcp[6478]: DHCPOFFER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:29:51 vdev dnsmasq-dhcp[6478]: DHCPDISCOVER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:29:51 vdev dnsmasq-dhcp[6478]: DHCPOFFER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:30:02 vdev dnsmasq-dhcp[6478]: DHCPDISCOVER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:30:02 vdev dnsmasq-dhcp[6478]: DHCPOFFER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:30:16 vdev dnsmasq-dhcp[6478]: DHCPDISCOVER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:30:16 vdev dnsmasq-dhcp[6478]: DHCPOFFER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:30:35 vdev dnsmasq-dhcp[6478]: DHCPDISCOVER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:30:35 vdev dnsmasq-dhcp[6478]: DHCPOFFER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 

Expected results:
Oct 12 09:38:09 vdev dnsmasq-dhcp[6478]: DHCPDISCOVER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:38:09 vdev dnsmasq-dhcp[6478]: DHCPOFFER(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:38:09 vdev dnsmasq-dhcp[6478]: DHCPREQUEST(virbr1) 172.23.29.154 52:54:00:9e:d8:87 
Oct 12 09:38:09 vdev dnsmasq-dhcp[6478]: DHCPACK(virbr1) 172.23.29.154 52:54:00:9e:d8:87 

Additional info:
Comment 2 Daniel Berrangé 2010-10-22 10:41:01 UTC 
Please provide details on what libvirt RPM version you have installed, and also the output of

 # iptables -L -n -v
 # iptables -t nat -L -n -v
Comment 4 Laine Stump 2010-10-22 16:21:39 UTC 
The version of the kernel and iptables rpms are also relevant - specifically any iptables prior to iptables-1.4.7-3.el6 would result in this problem (if the vhost-net module is loaded - check to see if /dev/vhost-net exists)
Comment 5 Brad Durrow 2010-11-28 19:15:06 UTC 
From Host

[root@vdev ~]# rpm -qa | fgrep libvirt
libvirt-python-0.8.1-13.el6.x86_64
libvirt-0.8.1-13.el6.x86_64
libvirt-client-0.8.1-13.el6.x86_64

**NOTE: I redacted some IPs and SUBNETS, they are marked with **
**NOTE** none of the above rules match 172.23.29.0/24 the subnet on virbr1

iptables -L -n -v
Chain INPUT (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:53 
    0     0 ACCEPT     udp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           udp dpt:67 
    0     0 ACCEPT     tcp  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:67 
    0     0 ACCEPT     udp  --  virbr1 *       0.0.0.0/0            0.0.0.0/0           udp dpt:53 
    0     0 ACCEPT     tcp  --  virbr1 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:53 
    0     0 ACCEPT     udp  --  virbr1 *       0.0.0.0/0            0.0.0.0/0           udp dpt:67 
    0     0 ACCEPT     tcp  --  virbr1 *       0.0.0.0/0            0.0.0.0/0           tcp dpt:67 
 6300 4669K ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PUBLICIP1**        0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PUBLICNET1**/26       0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PUBLICNET2**/29      0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PUBLICNET3**/26    0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PUBLICNET4**/29    0.0.0.0/0           
   42 14738 ACCEPT     all  --  *      *       **PUBLICNET5**/27     0.0.0.0/0           
 3009  567K ACCEPT     all  --  *      *       **PUBLICNET6**/25     0.0.0.0/0           
  462 34030 ACCEPT     all  --  *      *       **PRIVATENET1**/21       0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PRIVATENET2**/24       0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PRIVATENET3**/24      0.0.0.0/0           
    0     0 ACCEPT     all  --  *      *       **PRIVATENET4**/24     0.0.0.0/0           
    0     0 ACCEPT     udp  --  *      *       **PUBLICIP2*          0.0.0.0/0           udp spt:53 
    0     0 DROP       tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport sports 137,138,139,445 
    0     0 DROP       udp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport sports 137,138,139,445 
   20  6560 DROP       all  --  *      *       0.0.0.0/0            255.255.255.255     
    0     0 DROP       all  --  *      *       0.0.0.0/0            **PRIVATENET2_BROADCAST**       
    0     0 DROP       all  --  *      *       0.0.0.0/0            **PRIVATENET3_BROADCAST**       
    0     0 DROP       all  --  *      *       0.0.0.0/0            **PRIVATENET4_BROADCAST**      
  226 19630 DROP       all  --  *      *       0.0.0.0/0            224.0.0.0/8         
 164K   11M log_accept  tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp dpt:22 
   28  2208 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           state RELATED,ESTABLISHED 
    0     0 log_drop_banned  all  --  *      *       0.0.0.0/0            0.0.0.0/0           recent: UPDATE seconds: 3600 hit_count: 4 name: BANME side: source 
    0     0 ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           multiport dports 80,443 
    0     0 log_drop_portscan  all  --  *      *       0.0.0.0/0            0.0.0.0/0           recent: SET name: BANME side: source 

Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     all  --  *      virbr0  0.0.0.0/0            192.168.122.0/24    state RELATED,ESTABLISHED 
    0     0 ACCEPT     all  --  virbr0 *       192.168.122.0/24     0.0.0.0/0           
    0     0 ACCEPT     all  --  virbr0 virbr0  0.0.0.0/0            0.0.0.0/0           
    0     0 REJECT     all  --  *      virbr0  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
    0     0 REJECT     all  --  virbr0 *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
    0     0 ACCEPT     all  --  virbr1 virbr1  0.0.0.0/0            0.0.0.0/0           
    0     0 REJECT     all  --  *      virbr1  0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 
    0     0 REJECT     all  --  virbr1 *       0.0.0.0/0            0.0.0.0/0           reject-with icmp-port-unreachable 

Chain OUTPUT (policy ACCEPT 3094 packets, 14M bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain log_accept (1 references)
 pkts bytes target     prot opt in     out     source               destination         
 164K   11M ACCEPT     tcp  --  *      *       0.0.0.0/0            0.0.0.0/0           tcp flags:!0x17/0x02 
    3   192 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           LOG flags 0 level 4 prefix `ACCEPTED:' 
    3   192 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain log_drop_banned (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           LOG flags 0 level 4 prefix `BANNED:' 
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain log_drop_portscan (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           LOG flags 0 level 4 prefix `PORTSCAN:' 
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain log_drop_ssh (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0           LOG flags 0 level 4 prefix `DROPPED SSH:' 
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0      

         
[root@vdev work]# iptables -t nat -L -n -v
Chain PREROUTING (policy ACCEPT 2462 packets, 146K bytes)
 pkts bytes target     prot opt in     out     source               destination         

Chain POSTROUTING (policy ACCEPT 24 packets, 2021 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    1   347 MASQUERADE  all  --  *      *       192.168.122.0/24    !192.168.122.0/24    

Chain OUTPUT (policy ACCEPT 25 packets, 2368 bytes)
 pkts bytes target     prot opt in     out     source               destination         


[root@vdev ~]# rpm -qa | fgrep iptables
iptables-1.4.7-2.el6.x86_64
iptables-ipv6-1.4.7-2.el6.x86_64

[root@vdev ~]# ls -lda /dev/vhost-net
crw-rw----. 1 root root 10, 233 Nov  6 11:06 /dev/vhost-net

If iptables-1.4.7-3.el6 is my solution here do I get it.  Yum reports that my iptables is up to date.  And when I  tried to build from ftp://mirror.switch.ch/pool/2/mirror/redhat/linux/enterprise/6Server/en/os/SRPMS/iptables-1.4.7-3.el6.src.rpm I got the following error:

libxt_CHECKSUM.c:19:41: warning: linux/netfilter/xt_CHECKSUM.h: No such file or directory
libxt_CHECKSUM.c: In function 'CHECKSUM_parse':
libxt_CHECKSUM.c:44: error: dereferencing pointer to incomplete type
libxt_CHECKSUM.c:44: error: 'XT_CHECKSUM_OP_FILL' undeclared (first use in this function)
libxt_CHECKSUM.c:44: error: (Each undeclared identifier is reported only once
libxt_CHECKSUM.c:44: error: for each function it appears in.)
libxt_CHECKSUM.c: In function 'CHECKSUM_print':
libxt_CHECKSUM.c:69: error: dereferencing pointer to incomplete type
libxt_CHECKSUM.c:69: error: 'XT_CHECKSUM_OP_FILL' undeclared (first use in this function)
libxt_CHECKSUM.c: In function 'CHECKSUM_save':
libxt_CHECKSUM.c:78: error: dereferencing pointer to incomplete type
libxt_CHECKSUM.c:78: error: 'XT_CHECKSUM_OP_FILL' undeclared (first use in this function)
libxt_CHECKSUM.c: At top level:


I believe this means I need a newer kernel and I am not willing to build and deploy an new kernel on this machine.  Specifically I need a newer version of kernel-headers, I have (kernel-headers-2.6.32-44.2.el6.x86_64).
Comment 6 Daniel Berrangé 2010-11-29 13:52:39 UTC 
Both your kernel & iptables packages are too old. Those are pre-release  Beta versions, and not supported. Upgrade to the official RHEL6 release packages (kernel-2.6.32-71.el6 and iptables-1.4.7-3.el6) to get the fixes.

*** This bug has been marked as a duplicate of bug 612587 ***