Bug 711275

Summary: Packets dropped by the rx ring buffer in the bridge configuration
Product: Red Hat Enterprise Linux 6 Reporter: IBM Bug Proxy <bugproxy>
Component: qemu-kvmAssignee: jason wang <jasowang>
Status: CLOSED DUPLICATE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: high    
Version: 6.1CC: akong, bsarathy, ddutile, jkachuck, juzhang, mkenneth, rhod, rprice, tburke, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: All   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-03-05 08:01:01 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On:    
Bug Blocks: 562808, 580953, 684953, 711169, 756082, 804141    
Attachments:
Description Flags
file contains the ifconfig eth0 output when the file transfer is going on
none
Packet loss in RX buffer on guest for simultaneous large file trfr. none

Description IBM Bug Proxy 2011-06-07 05:00:19 UTC
---Problem Description---
In the Bridge configuration :

   <interface type='bridge'>
      <mac address='de:ad:be:ef:43:42'/>
      <source bridge='virbr0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

packets are dropped in the rx ring buffer when transferring large files from host-guest and vice-versa (No packets are dropped in tx ring buffer)

 
---uname output---
Linux localhost.localdomain 2.6.9-89.EL #1 Mon Apr 20 10:23:08 EDT 2009 i686 i686 i386 GNU/Linux
 
Machine Type = x3650 
 
---Steps to Reproduce---
 create 5GB files on guest and host : 
dd if=/dev/zero of=5GBGBfile.guest bs=1024k count=5000 on guest 
dd if=/dev/zero of=5GBGBfile.host bs=1024k count=5000 on host

start copying the files : 

5GBGBfile.guest file - from guest to host 
5GBGBfile.host file - from host to guest 

using scp utility 

run the following one-liner on the guest 
while true ; do date >> rxtx.log  ; ifconfig eth0 | egrep "RX bytes|RX packets|TX packets"  >> rxtx.log  ; sleep 1 ; done 

this will continuously monitor the RX/TX ring buffers and  write the information to rxtx.log file - during the analysis of the log file it is observed that approx 2000 packets are dropped by rx ring buffer.
 
Log file 
---------
file contains the ifconfig eth0 output when the file transfer is going on


General information :
--------------------------
Kernel version: Linux mx3650a.in.ibm.com 2.6.32-128.el6.x86_64 #1 SMP Mon Mar 28 21:55:33 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

Qemu version: 
root@mx3650a ping_test_data]# rpm -qa | grep kvm
qemu-kvm-0.12.1.2-2.153.el6.x86_64
qemu-kvm-tools-0.12.1.2-2.153.el6.x86_64
[root@mx3650a ping_test_data]# rpm -qa|grep qemu
qemu-kvm-0.12.1.2-2.153.el6.x86_64
gpxe-roms-qemu-0.9.7-6.7.el6.noarch
qemu-img-0.12.1.2-2.153.el6.x86_64
qemu-kvm-tools-0.12.1.2-2.153.el6.x86_64

Guest OS: 64 bit

Guest OS Image storage type: local file

Host Machine Type: x3650

Target Machine Type (for migration): -NA-

Test Type: manual 

Qemu Command Line: 
 /usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 1024 -smp 4,sockets=4,cores=1,threads=1 -name vm00-RHEL48-SMP -uuid e9e158b9-0b9b-79e4-53e6-8229b72a28ad -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vm00-RHEL48-SMP.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -boot c -drive file=/home/storage/onnm/vm00-RHEL48-SMP.img,if=none,id=drive-ide0-0-0,format=raw,cache=none,aio=threads -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,fd=32,id=hostnet0 -device rtl8139,netdev=hostnet0,id=net0,mac=de:ad:be:ef:48:46,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 0.0.0.0:54,password -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

Using libvirt or not?
Using libvirt , but not relevant to this test

============ Networking details ====================

Device details :
Adapter : Qemu Emulated Realtek-8139 
device script :
 <interface type='bridge'>
      <mac address='de:ad:be:ef:48:46'/>
      <source bridge='virbr0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

Default network :
Network Script :
<network>
  <name>default</name>
  <uuid>32612766-4b8e-4cd1-8eb9-5f9350a1b094</uuid>
  <forward mode='nat'/>
  <bridge name='virbr0' stp='on' delay='0' />
  <mac address='52:54:00:30:7B:9A'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254' />
    </dhcp>
  </ip>
</network>



============= Actual test ========================
** Note : 
As far as "Data Transmission Receving test" 
(http://fedoraproject.org/wiki/QA:Testcase_Virtualization_Data_Transmission_Receving)
is concerned - I have already PASSed it. 

But I am concerned about the packt loss because of this 
(http://en.wikipedia.org/wiki/Packet_loss), so here are the detailed steps :

(1) create a 2.5GB File on host:
    dd if=/dev/zero of=2.5GGFile.host bs=1024k count=2500
(2) take it chksum :
    cksum 2.5GBFile.host > cksum.host
(3) Repeate the steps (1) & (2) on guest : 
    dd if=/dev/zero of=2.5GGFile.guest bs=1024k count=2500
    cksum 2.5GBFile.guest > cksum.guest
(4) Copy the cksum.host file to guest 
    So, now the there are 3 files on guest:
    	[root@localhost files]# ls -l # on guest 
	total 2562512
	-rw-r--r--  1 root root 2621440000 Apr 13 12:42 2.5GBFile.guest
	-rw-r--r--  1 root root         38 Apr 14 05:16 cksum.guest
	-rw-r--r--  1 root root         37 Apr 14 05:19 cksum.host
	[root@localhost files]# cat cksum.*
	2930258761 2621440000 2.5GBFile.guest 
	2930258761 2621440000 2.5GBFile.host
(5) Copy the cksum.guest file to host 
    So, now the there are 3 files on host:
    	[root@mx3650a files]# ls -l
	total 2560012
	-rw-r--r--. 1 root root 2621440000 Apr 13 18:12 2.5GBFile.host
	-rw-r--r--. 1 root root         38 Apr 14 10:52 cksum.guest
	-rw-r--r--. 1 root root         37 Apr 14 10:42 cksum.host
	[root@mx3650a files]# cat cksum.*
	2930258761 2621440000 2.5GBFile.guest
	2930258761 2621440000 2.5GBFile.host
(6) reboot the guest 
(7) Check for the state of the network interface on _guest_:
    [root@localhost ~]# ifconfig eth0 
eth0      Link encap:Ethernet  HWaddr DE:AD:BE:EF:48:46  
          inet addr:192.168.122.164  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::dcad:beff:feef:4846/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:68 errors:0 dropped:0 overruns:0 frame:0 <---no packets dropped !!
          TX packets:65 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:6982 (6.8 KiB)  TX bytes:8811 (8.6 KiB)
          Interrupt:10 
(8) Check for the state of the network interface on _host_:
[root@mx3650a files]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:1A:64:63:94:D4  
          inet addr:9.126.89.21  Bcast:9.126.89.255  Mask:255.255.255.0
          inet6 addr: fe80::21a:64ff:fe63:94d4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3229397 errors:0 dropped:0 overruns:0 frame:0 <---no packets dropped !!
          TX packets:1775444 errors:19283 dropped:0 overruns:0 carrier:19283
          collisions:203734 txqueuelen:1000 
          RX bytes:2978279305 (2.7 GiB)  TX bytes:868705632 (828.4 MiB)
          Interrupt:16 Memory:ce000000-ce012800
(9) Start transfering 2.5 GB Files from host-to-guest and guest-to-host simultaneously:
    From host give these two commands from different terminals :
    [root@mx3650a files]# scp root.122.164:/root/files/2.5GBFile.guest . 
    [root@mx3650a files]# scp 2.5GBFile.host  root.122.164:/root/files/

    and wait for the transfer to complete !!

After transfer is complete on both sides .....

(10) Check for the state of the network interface on _host_:
[root@mx3650a files]# ifconfig eth0
eth0      Link encap:Ethernet  HWaddr 00:1A:64:63:94:D4  
          inet addr:9.126.89.21  Bcast:9.126.89.255  Mask:255.255.255.0
          inet6 addr: fe80::21a:64ff:fe63:94d4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3243345 errors:0 dropped:0 overruns:0 frame:0 <---no packets dropped !!
          TX packets:1790267 errors:19954 dropped:0 overruns:0 carrier:19954
          collisions:204442 txqueuelen:1000 
          RX bytes:2979433111 (2.7 GiB)  TX bytes:883134757 (842.2 MiB)
          Interrupt:16 Memory:ce000000-ce012800 

(11) Check for the state of the network interface on _guest_:
[root@localhost files]# ifconfig eth0 
eth0      Link encap:Ethernet  HWaddr DE:AD:BE:EF:48:46  
          inet addr:192.168.122.164  Bcast:192.168.122.255  Mask:255.255.255.0
          inet6 addr: fe80::dcad:beff:feef:4846/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2739049 errors:0 dropped:1675 overruns:0 frame:0 <--- PACKETS DROPPED !!
          TX packets:2010431 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2772622210 (2.5 GiB)  TX bytes:2760062357 (2.5 GiB)
          Interrupt:10 

(12) Cksum for guest file :
[root@localhost files]# cksum 2.5GBFile.guest > cksum.guest.latest
[root@localhost files]# diff cksum.guest cksum.guest.latest

seems to be OK !

(13) cksum for host file :
 [root@localhost files]# cksum 2.5GBFile.host > cksum.host.latest
[root@localhost files]# diff cksum.host.latest cksum.host

seems to be OK !

But then , is packet loss here insignificant ??

** Note ** 

With virtio adapter - file transfer is clean with 0 packet loss, 
both with vhost=on|off

Comment 1 IBM Bug Proxy 2011-06-07 05:00:27 UTC
Created attachment 503378 [details]
file contains the ifconfig eth0 output when the file transfer is going on

Comment 4 Ronen Hod 2011-06-08 09:45:21 UTC
Did you measure the total transmission time? Was the transfer to the guest any slower?

Comment 7 IBM Bug Proxy 2011-06-20 06:21:41 UTC
------- Comment From onmahaja.com 2011-06-20 02:10 EDT-------
Following transfer rates were observed :

Guest to Host
2000MB   2.8MB/s   11:56

Host to Guest
2000MB   5.0MB/s   06:44

H/w :
0:03.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 20)

-- Onkar Mahajan

Comment 9 IBM Bug Proxy 2011-10-07 07:10:33 UTC
------- Comment From vahegde1.ibm.com 2011-10-07 03:01 EDT-------
Hi RedHat,

This problem is observed in RHEL 6.2 guest running on RHEL 6.2 host. Please see
the attachment for more details.

Please observe that file transfer from guest->host is 60% slower than from
host->guest for same file sizes ; chsum of the files transferred remain the
same , however - no data loss is observed.

-- Onkar

Comment 10 IBM Bug Proxy 2011-10-07 07:10:45 UTC
Created attachment 526838 [details]
Packet loss in RX buffer on guest for simultaneous large file trfr.

Comment 12 jason wang 2011-12-11 03:41:13 UTC
*** Bug 580345 has been marked as a duplicate of this bug. ***

Comment 14 jason wang 2012-03-05 08:01:01 UTC

*** This bug has been marked as a duplicate of bug 743513 ***