Bug 692770

Summary: windows installation is slow
Product: Red Hat Enterprise Linux 5 Reporter: Suqin Huang <shuang>
Component: kvmAssignee: Kevin Wolf <kwolf>
Status: CLOSED WONTFIX QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.7CC: gcosta, juzhang, michen, mkenneth, virt-maint, xwei
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2011-07-28 14:54:41 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 580948    

Description Suqin Huang 2011-04-01 05:16:54 UTC
Description of problem:
put win7-64 iso,image on local machine, and install it, 32mins only expend 20% files

Version-Release number of selected component (if applicable):
kvm-83-229.el5

How reproducible:
100%

Steps to Reproduce:
1. cmd:
/usr/libexec/qemu-kvm -monitor stdio -drive file='/home/Auto/autotest/client/tests/kvm/images/win7-64-virtio.qcow2',index=0,if=virtio,media=disk,cache=none,boot=on,format=qcow2 -net nic,vlan=0,model=virtio,macaddr='9a:47:be:0e:ae:c9' -net tap,vlan=0,ifname='t0-192228-k20o',script='/home/scripts/qemu-ifup-switch',downscript='no' -m 2048 -smp 2,cores=1,threads=1,sockets=2 -drive file='/root/en_windows_7_ultimate_x64_dvd_x15-65922.iso',media=cdrom,index=1 -drive file='/home/Auto/autotest/client/tests/kvm/isos/windows/winutils.iso',media=cdrom,index=2 -cpu qemu64,+sse2 -soundhw ac97 -fda '/usr/share/virtio-win/virtio-drivers.vfd' -vnc :0 -rtc-td-hack -M rhel5.6.0 -boot d  -usbdevice tablet

2.
3.
  
Actual results:
expand 20% files take 32mins

Expected results:


Additional info:
1. host
processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 44
model name	: Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
stepping	: 2
cpu MHz		: 2394.067
cache size	: 12288 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 4
apicid		: 21
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx pdpe1gb rdtscp lm constant_tsc ida nonstop_tsc arat pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4788.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

2. guest
win7-64

3. top
9062 root      15   0 8592m 380m 4168 S  3.3  3.2   2:25.92 -         qemu                                                                                                      
 9070 root      15   0 8592m 380m 4168 S  0.0  3.2   0:00.00 -         qemu                                                                                                      
 9072 root      15   0 8592m 380m 4168 S  0.0  3.2   0:03.83 kvm_vcpu_ qemu                                                                                                      
 9073 root      15   0 8592m 380m 4168 S  0.0  3.2   0:00.71 kvm_vcpu_ qemu                                                                                                      
 9074 root      15   0 8592m 380m 4168 S  0.0  3.2   0:02.03 kvm_vcpu_ qemu                                                                                                      
 9075 root      15   0 8592m 380m 4168 S  0.0  3.2   0:01.11 kvm_vcpu_ qemu                                                                                                      
 9082 root      15   0 8592m 380m 4168 S  0.0  3.2   0:00.00 -         qemu                                                                                                      
 9086 root      15   0 8592m 380m 4168 S  0.0  3.2   0:00.27 -         qemu  

4. kvm_stat

  kvm statistics

 efer_reload              11992       0
 exits                  6443889   23411
 fpu_reload             1546829   10109
 halt_exits              435371    1960
 halt_wakeup             155979     759
 host_state_reload	2585411   10180
 hypercalls                   0       0
 insn_emulation         2674191   10799
 insn_emulation_fail          0       0
 invlpg                       0       0
 io_exits               1693311    9338
 irq_exits               342406     890
 irq_injections          598602    2492
 irq_window               79207    1188
 kvm_request_irq              0       0
 largepages                   0       0
 mmio_exits              440318       5
 mmu_cache_miss           45301       0
 mmu_flooded                  0       0
 mmu_pde_zapped               0       0
 mmu_pte_updated              0       0
 mmu_pte_write          1946633    8672
 mmu_recycled                 0       0
 mmu_shadow_zapped        51985       0
 mmu_unsync                   0       0
 mmu_unsync_global            0       0
 nmi_injections               0       0
 nmi_window                   0       0
 pf_fixed               1327247     473
 pf_guest                     0       0
 remote_tlb_flush          8077       2
 request_nmi                  0       0
 signal_exits                 2       0
 tlb_flush                67327     644

Comment 1 Glauber Costa 2011-05-16 18:15:09 UTC
Do you notice this slowness only on this kind of guest ? 
Is the guest also slow on other parts of the installation other than file-copy ? 
My first hunch is that we have a qcow2 slowness problem here.

Comment 2 Xiaoqing Wei 2011-06-09 11:06:48 UTC
(In reply to comment #1)
> Do you notice this slowness only on this kind of guest ? 
> Is the guest also slow on other parts of the installation other than file-copy
> ? 
> My first hunch is that we have a qcow2 slowness problem here.


Hi Glauber,

I tried several times,
 when installing with qcow2, it's much slower than raw format.


if with memory overcommitment (boot guest with max(host mem)):
 raw image can finish installation within 120 mins 
 qcow2 image even cannot finished file expanding(approx 80% when 120 mins past)

not memory overcommitment :
 raw image can finish installation within 40 mins (mostly 20+ mins)
 qcow2 image mostly can finish installtion within 60 mins


cmd to start installation (just image format differs : qcow2/raw): 
qemu-kvm  -name '08r2-ins' -monitor stdio \
-serial unix:'/tmp/serial \
-drive file='/win2008r2-64-virtio.qcow2',index=0,if=virtio,media=disk,cache=none,boot=on,format=qcow2 -net nic,vlan=0,model=virtio,macaddr='9a:6b:1e:67:cf:66' \
-net tap,vlan=0,ifname='t08r2',script='/qemu-ifup-switch',downscript='no' \
\
-m 12003 \
\
-smp 16,cores=1,threads=1,sockets=16 \
-drive file='/2008_r2_with_sp1_x64.iso',media=cdrom,index=1 \
-drive file='/winutils.iso',media=cdrom,index=2 \
-drive file='/virtio-win.iso.el5',media=cdrom,index=3 \
-cpu qemu64,+sse2 -soundhw ac97 \
-fda '/answer.vfd'  \
-vnc :0 -rtc-td-hack -M rhel5.6.0 -boot d  -usbdevice tablet


host info:

free -m
             total       used       free     shared    buffers     cached
Mem:         12003        343      11660          0          3        117
-/+ buffers/cache:        222      11781
Swap:        14015        160      13855

cpu info (16 processors in total)

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 44
model name	: Intel(R) Xeon(R) CPU           E5620  @ 2.40GHz
stepping	: 2
cpu MHz		: 2394.069
cache size	: 12288 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 4
apicid		: 21
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx pdpe1gb rdtscp lm constant_tsc ida nonstop_tsc arat pni monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 4787.94
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


Best Regards,
Xiaoqing.

Comment 4 Xiaoqing Wei 2011-06-10 15:08:08 UTC
 > (In reply to comment #2)
 > > (In reply to comment #1)
 > > > Do you notice this slowness only on this kind of guest ? 
 > > > Is the guest also slow on other parts of the installation other than file-copy
 > > > ? 
 > > > My first hunch is that we have a qcow2 slowness problem here.
 > > 
 > > 
 > > Hi Glauber,
 > > 
 > > I tried several times,
 > >  when installing with qcow2, it's much slower than raw format.
 > > 
 > > 
 > > if with memory overcommitment (boot guest with max(host mem)):
 > >  raw image can finish installation within 120 mins 
 > >  qcow2 image even cannot finished file expanding(approx 80% when 120 mins past)
 > 
 > Does it swap to disk? Or is KSM around and makes some room?
 > If it swaps to disk, then the speed sounds OK to me.
 > 
 > > 
 > > not memory overcommitment :
 > >  raw image can finish installation within 40 mins (mostly 20+ mins)
 > 
 > So, is that a regression from 5.6?
 no, can reproduce on rhel5.6
 2.6.18-238.el5
 kvm-83-224.el5
 > 
 > >  qcow2 image mostly can finish installtion within 60 mins
 > > 
 > > 
 > > cmd to start installation (just image format differs : qcow2/raw): 
 > > qemu-kvm  -name '08r2-ins' -monitor stdio \
 > > -serial unix:'/tmp/serial \
 > > -drive
 > > file='/win2008r2-64-virtio.qcow2',index=0,if=virtio,media=disk,cache=none,boot=on,format=qcow2
 > > -net nic,vlan=0,model=virtio,macaddr='9a:6b:1e:67:cf:66' \
 > > -net tap,vlan=0,ifname='t08r2',script='/qemu-ifup-switch',downscript='no' \
 > > \
 > > -m 12003 \
 > > \
 > > -smp 16,cores=1,threads=1,sockets=16 \
 > 
 > Can you try with less vCPUs? Say, 2?
 yes, also slow when install with 2 cpu & 4G mem
 Intel e5620 (16cpu & 12Gmem)
 1 hour only expend 60%
 others,
 can not reproduce on  AMD Phenom(tm) 9600B
 finish installation within 40 mins

Comment 5 Kevin Wolf 2011-07-28 14:54:41 UTC
Known problem, qcow2 is slow on RHEL 5 during cluster allocation. We won't be able to make further optimisations for RHEL 5, it would require very invasive changes.