750421 – reboot windows2003 guest will hang after change balloon size

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 750421 - reboot windows2003 guest will hang after change balloon size

Summary: reboot windows2003 guest will hang after change balloon size

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	virtio-win
Sub Component:
Version:	6.2
Hardware:	Unspecified
OS:	Linux
Priority:	high
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Vadim Rozenfeld
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2011-11-01 03:30 UTC by langfang
Modified:	2013-02-21 10:37 UTC (History)
CC List:	11 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	In the previous versions WS2K3 hangs during reboot followed after changing balloon size. This situation happens due to lack of free memory, the system needs to start shutdown process. With the following fix we release all the inflated memory to the guest before starting to process power management request.
Clone Of:
Environment:
Last Closed:	2013-02-21 10:37:43 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
Crash dump file on win2003-64 guest (16.65 MB, application/x-gzip) 2012-02-27 10:23 UTC, Xiaomei Gao	no flags	Details
console output (13.33 KB, text/plain) 2012-04-26 07:03 UTC, Xiaoqing Wei	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2013:0441	0	normal	SHIPPED_LIVE	virtio-win bug fix and enhancement update	2013-02-20 20:48:13 UTC

Description langfang 2011-11-01 03:30:26 UTC

Description of problem:
window 2003 guest will hang when set balloon size too small


Version-Release number of selected component (if applicable):
kernel:2.6.32-214.el6.x86_64
qemu-kvm-0.12.1.2-2.206.el6.x86_64
virtio-win:virtio-win-1.4.0-1.el6.noarch.rpm

How reproducible:
100%

CLI:/usr/libexec/qemu-kvm -m 2G -smp 2 -cpu cpu64-rhel6,+x2apic -usb -device usb-tablet -rtc base=localtime,clock=host,driftfix=slew -drive file=/home/win2003R2sp2_x64.qcow2.bak,format=qcow2,if=none,id=drive-ide0-0-0,werror=stop,rerror=stop,cache=none -device virtio-blk-pci,drive=drive-ide0-0-0,id=ide0-0-0 -netdev tap,id=hostnet0,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,mac=00:10:20:2c:45:23,bus=pci.0,addr=0x4,id=net0 -boot order=dcn -uuid d2b0ab4c-d828-468e-8e30-ef5da1754db1 -rtc-td-hack -no-kvm-pit-reinjection -monitor stdio -name win2k3sp2-64 -drive file=/usr/share/virtio-win/virtio-win.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -device virtio-balloon-pci,bus=pci.0,id=balloon0 -spice port=5830,disable-ticketing -vga qxl


Steps to Reproduce:
1.boot a windows2003 guest with balloon device(....-device virtio-balloon-pci,bus=pci.0,id=balloon0)
2.info balloon 
balloon: actual=2048
3.balloon 600
 info balloon
balloon: actual=600
4.restart guest
  
Actual results: the guest will  hang


Expected results:the guest will not hang,and work well.


Additional info:

Comment 2 langfang 2011-11-01 06:10:03 UTC

attention:
1) the hang guest is windows2003(64bit)
2) restart guest will hang after the balloon size changed,for example after set balloon 1024 or balloon 1400,and then restart guest,guest will hang too.

other important thing:when change guest windows7(64bit) balloon size,and then restart guest,will not hang .

Comment 3 Xiaomei Gao 2011-12-12 12:07:07 UTC

if QXL driver is installed in win2003-64, the guest restarts normally when balloon size is changed and guest is restarted. Also, guest shows black screen on my test machine if QXL driver is not installed. so the issue is relevant to QXL driver.

Comment 5 Vadim Rozenfeld 2012-02-13 15:26:17 UTC

Hi,

How about w2k3-32? Can you reproduce this problem on 32-bit platform?
Could you also enable NMI crash dump inside the guest, and generate 
a crash dump file when hanging?

Thank you,
Vadim.

Comment 6 Xiaomei Gao 2012-02-15 02:01:58 UTC

(In reply to comment #5)
> Hi,
> 
> How about w2k3-32? Can you reproduce this problem on 32-bit platform?
> Could you also enable NMI crash dump inside the guest, and generate 
> a crash dump file when hanging?
> 
> Thank you,
> Vadim.

Hi,Vadim
    This problem can not be reproduced on 32-bit platform, so only win2003-64 platform can hit the issue. Crash dump file is in the attachment. Please check it.

Best Regards
xigao

Comment 7 Xiaomei Gao 2012-02-15 02:50:27 UTC

Created attachment 562105 [details]
Crash dump file on win2003-64 guest

Comment 8 Vadim Rozenfeld 2012-02-26 07:24:41 UTC

(In reply to comment #7)
> Created attachment 562105 [details]
> Crash dump file on win2003-64 guest

Hi Xiaomei,

Unfortunately, the above dump file 
seems to be corrupted. 
Could you please generate another 
crash dump file for me?

Thank you.
Kind regards,
Vadim.

Comment 9 Xiaomei Gao 2012-02-27 10:23:58 UTC

Created attachment 566009 [details]
Crash dump file on win2003-64 guest

Comment 10 Xiaomei Gao 2012-02-27 10:40:43 UTC

(In reply to comment #8)
> (In reply to comment #7)
> > Created attachment 562105 [details]
> > Crash dump file on win2003-64 guest
> 
> Hi Xiaomei,
> 
> Unfortunately, the above dump file 
> seems to be corrupted. 
> Could you please generate another 
> crash dump file for me?
> 
> Thank you.
> Kind regards,
> Vadim.

hi, Vadim
    Please check the attachment 566009 [details]. The file size is a little big, so you can contract me if you have any need.

Best Regards
xigao

Comment 11 Vadim Rozenfeld 2012-02-27 11:46:56 UTC

(In reply to comment #10)
> hi, Vadim
>     Please check the attachment 566009 [details]. The file size is a little 

Many thanks.
This one is good.

Kind regards,
Vadim.

big, so you
> can contract me if you have any need.
> 
> Best Regards
> xigao

Comment 15 Xiaoqing Wei 2012-04-26 07:01:09 UTC

reproduce this on rhel-5.8.ga guest, 
host:
kernel-2.6.32-220.13.1.el6.x86_64
qemu-kvm-0.12.1.2-2.209.el6_2.5.x86_64

when guest hangs
(qemu) info registers
info registers
EAX=00000000 EBX=00000000 ECX=c0416d00 EDX=00000005
ESI=c063cfa3 EDI=cb044620 EBP=00000020 ESP=c070ef94
EIP=c0416d2f EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=1
ES =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
CS =0060 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
SS =0068 00000000 ffffffff 00c09300 DPL=0 DS   [-WA]
DS =007b 00000000 ffffffff 00c0f300 DPL=3 DS   [-WA]
FS =0000 00000000 ffffffff 00000000
GS =0000 00000000 ffffffff 00000000
LDT=0088 c0751020 00000027 00008200 DPL=0 LDT
TR =0080 cb00f400 00002073 00008b00 DPL=0 TSS32-busy
GDT=     cb047000 000000ff
IDT=     c0700000 000007ff
CR0=8005003b CR2=bfb02fb0 CR3=0074d000 CR4=000006d0
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=8000000000000000 4000 FPR7=8000000000000000 3fff
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000


# top -p 19749 -H

top - 14:32:18 up 3 days,  3:07,  3 users,  load average: 1.00, 1.24, 1.37
Tasks:  10 total,   1 running,   9 sleeping,   0 stopped,   0 zombie
Cpu(s): 12.5%us,  2.3%sy,  0.0%ni, 85.2%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   7872976k total,   825148k used,  7047828k free,     7020k buffers
Swap: 33554416k total,        0k used, 33554416k free,    42888k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                  
19755 root      20   0 9305m 312m 5044 R 99.5  4.1  12:54.17 qemu                                     
19749 root      20   0 9305m 312m 5044 S 16.6  4.1   5:48.24 qemu                                     
19780 root      20   0 9305m 312m 5044 S  0.3  4.1   0:06.22 qemu                                     
19754 root      20   0 9305m 312m 5044 S  0.0  4.1   5:28.93 qemu                                     
19756 root      20   0 9305m 312m 5044 S  0.0  4.1   5:13.45 qemu                                     
19759 root      20   0 9305m 312m 5044 S  0.0  4.1   5:17.19 qemu                                     
19760 root      20   0 9305m 312m 5044 S  0.0  4.1   4:29.46 qemu                                     
19761 root      20   0 9305m 312m 5044 S  0.0  4.1   5:15.49 qemu                                     
19762 root      20   0 9305m 312m 5044 S  0.0  4.1   5:15.65 qemu                                     
19763 root      20   0 9305m 312m 5044 S  0.0  4.1   8:21.44 qemu                                     


guest console output during reboot will be attahched.

Comment 16 Xiaoqing Wei 2012-04-26 07:03:10 UTC

Created attachment 580378 [details]
console output

/root/staf-kvm/autotest/client/tests/kvm/qemu -name vm1 -chardev socket,id=human_monitor_id_humanmonitor1,path=/tmp/monitor-humanmonitor1-20120426-111129-IlrO,server,nowait -mon chardev=human_monitor_id_humanmonitor1,mode=readline -chardev socket,id=serial_id_20120426-111129-IlrO,path=/tmp/serial-20120426-111129-IlrO,server,nowait -device isa-serial,chardev=serial_id_20120426-111129-IlrO -device ich9-usb-uhci1,id=usb1,multifunction=off,bus=pci.0,addr=0x4 -drive file=/root/staf-kvm/autotest/client/tests/kvm/images/RHEL-Server-5.8-32-virtio.qcow2,index=0,if=none,id=drive-virtio-disk1,media=disk,cache=none,format=qcow2,aio=native -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1 -device virtio-net-pci,netdev=idCr0paF,mac=9a:f0:54:6e:2f:d4,id=ndev00idCr0paF,bus=pci.0,addr=0x3 -netdev tap,id=idCr0paF,vhost=on,fd=19 -m 8192 -smp 8,cores=4,threads=1,sockets=2 -cpu Westmere -device usb-tablet,id=usb-tablet1,bus=usb1.0 -spice port=8000,disable-ticketing -vga qxl -rtc base=utc,clock=host,driftfix=slew -M rhel6.2.0 -boot order=cdn,once=c,menu=off -device virtio-balloon-pci -no-shutdown -no-kvm-pit-reinjection -enable-kvm


then balloon to 4096, and it will actually evict to 4914

(qemu) info balloon
info balloon
balloon: actual=4914

Comment 17 Amit Shah 2012-04-27 05:33:56 UTC

To isolate the issues, please get us details of the following:

1. RHEL6 guests behaviour with steps as in comment 0
2. RHEL6 guests behaviour with and without qxl
3. RHEL5 guests, both similar to points 1 and 2
4. In each case, please detail the steps you take that reproduces or does not reproduce the problem.

Comment 18 Xiaoqing Wei 2012-04-27 05:53:39 UTC

(In reply to comment #17)

> 3. RHEL5 guests, both similar to points 1 and 2

1) When I met this issue, rhel5 guest didn't configured to use the qxl driver
2) After configured rhel5 guest to use the qxl driver and reboot to take effect, didn't hit this issue in 5 attempts.

Comment 19 langfang 2012-06-07 05:43:47 UTC

(In reply to comment #17)
> To isolate the issues, please get us details of the following:
> 
> 1. RHEL6 guests behaviour with steps as in comment 0
> 2. RHEL6 guests behaviour with and without qxl
> 3. RHEL5 guests, both similar to points 1 and 2
> 4. In each case, please detail the steps you take that reproduces or does
> not reproduce the problem.

very sorry for the too late comments:

test 2 senarios for this bug.
version:
host:
# uname -r
2.6.32-274.el6.x86_64
 # rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.295.el6.x86_64

senario 1)

guest: kernel:2.6.32-274.el6.x86_64

test on rhel6.3 guest with -vga qxl-->change balloon size ---> guest work well
test on rhel6.3 guest without -vga qxl-->change balloon size --->guest work well

senario 2)

guest:kernel
2.6.18-300.el5
test on rhel5.8 guest with -vga qxl-->change balloon size --->guest work well  

test on rhel5.8 guest with -vga qxl update use qxl driver-->change balloon size ---> guest work well
test on rhel5.8 guest without -vga qxl-->change balloon size --->guest work well

Comment 20 Amit Shah 2012-06-07 06:30:51 UTC

(In reply to comment #19)

> test on rhel6.3 guest with -vga qxl-->change balloon size ---> guest work
> well

You also restarted the guest in all cases as in orig. description, right?

Comment 21 langfang 2012-06-07 07:33:29 UTC

(In reply to comment #20)
> (In reply to comment #19)
> 
> > test on rhel6.3 guest with -vga qxl-->change balloon size ---> guest work
> > well
> 
> You also restarted the guest in all cases as in orig. description, right?

Hi! Amit,please ignore the comment 19 .

test 2 senarios for this bug.
version:
host:
# uname -r
2.6.32-274.el6.x86_64
 # rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.295.el6.x86_64

senario 1)

guest: kernel:2.6.32-274.el6.x86_64

test on rhel6.3 guest with -vga qxl(use qxl driver)-->change balloon size -->reboot guest---> gest work well
test on rhel6.3 guest with -vga qxl(not use qxl driver)-->change balloon size-->reboot guest-->guest work well
test on rhel6.3 guest without -vga qxl-->change balloon size --->reboot guest-->guest work well

senario 2)

guest:kernel
2.6.18-300.el5
test on rhel5.8 guest with -vga qxl-->change balloon size --->reboot guest-->guest work well  

test on rhel5.8 guest with -vga qxl (update use qxl driver)-->reboot guest-->change balloon size ---> guest work well
test on rhel5.8 guest without -vga qxl-->change balloon size --->reboot guest-->guest work well


Amit,if need me test other cases ,please directly ping me~,thanks

Comment 22 Amit Shah 2012-06-07 09:25:02 UTC

(In reply to comment #21)

> test 2 senarios for this bug.
> version:
> host:
> # uname -r
> 2.6.32-274.el6.x86_64
>  # rpm -q qemu-kvm
> qemu-kvm-0.12.1.2-2.295.el6.x86_64

OK, so looks like it doesn't affect Linux/RHEL guests.  Thanks.

Comment 24 Ronen Hod 2012-07-10 09:00:04 UTC

There were several related fixes in the virtio-win balloon drivers. Please retest with build 30.

Thanks.

Comment 25 langfang 2012-07-11 05:15:11 UTC

reproduce version:
# uname -r
2.6.32-272.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.295.el6.x86_64
# rpm -q virtio-win
virtio-win-1.4.0-1.el6.noarch

senario 1: boot guest with -vga qxl(not use qxl driver)-->change balloon size -->reboot guest-->guest show black screen

senario 2: boot guest with -vga qxl(use qxl driver)-->change balloon size -->reboot guest-->balloon size changed and guest work well



test use virtio-win-prewhql-0.1-30

# uname -r
2.6.32-272.el6.x86_64
# rpm -q qemu-kvm
qemu-kvm-0.12.1.2-2.295.el6.x86_64
# rpm -q virtio-win

senario 1):boot guest with -vga qxl(not use qxl driver)-->change balloon size -->reboot guest-->after boot successfully,the balloon size not changed,guest work well

boot guest with -vga qxl(use qxl driver)-->change balloon size -->reboot guest-->after boot successfully,the balloon size not changed,guest work well
senario 2):



addtion info :
when i use 'virtio-win-1.4.0-1.el6.noarch' update balloon driver-->then change balloon size from 2048 to 2000-->update balloon driver use 'virtio-win-prewhql-0.1-30' --->reboot guest--->balloon size change to 2048.  

i will report a bug to track this issue

Comment 26 Ronen Hod 2012-07-11 08:31:52 UTC

Fixed in build 30 (or earlier). Marking as MODIFIED.
The other issue was reported to BZ#839143

Comment 28 Mike Cao 2012-08-03 02:42:49 UTC

According to comment #25 ,this bug has been fixed already ,move status to verified.

Comment 29 errata-xmlrpc 2013-02-21 10:37:43 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0441.html

Note You need to log in before you can comment on or make changes to this bug.