Bug 1263998

For steps4, modified as “In destination, shutdown the guest with "system_powerdown" in qemu-kvm after migrate completed”

Is this a regression from RHEL 7.1? 
Does this affect other Windows OS versions?
Does the guest shutdown properly if it is idle?
Can you shutdown the guest from the guest OS?
What is the guest doing when it is hung? Can you get a dump?

Hi Jing,

Could you reply comment3?

Best Regards,
Junyi

I have reproduced this issue with following option on RHEL7.2
  - delete network device and dataplane
1. boot guest in src with cli;
/usr/libexec/qemu-kvm \
-M pc \
-cpu IvyBridge \
-nodefaults -rtc base=utc -no-hpet \
-m 2G \
-smp 4,sockets=2,cores=2,threads=1 \
-enable-kvm \
-name rhel7.2 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-monitor stdio \
-qmp tcp:0:6661,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-serial unix:/tmp/serial0,server,nowait \
-vga cirrus \
-vnc :0  \
-chardev socket,id=seabios,path=/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-usb \
-device usb-tablet,id=tablet0 \
-drive file=/home/win8r2/win8r2.raw,if=none,id=drive-virtio-disk0,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=pci.0,id=drive-virtio-disk0,drive=drive-virtio-disk0,bootindex=1 \
-nodefaults
2.Boot vm with -incoming tcp:0:5800 in destination
3.migrate in the src host
4.shutdown from guest OS or "system_powerdown" after migration

The same issue can be reproduced.

Reproduced this issue with option(delete dataplane, keep network) on RHEL7.2
1. boot guest in src with cli;
-M pc \
-cpu SandyBridge \
-nodefaults -rtc base=utc -no-hpet \
-m 2G \
-smp 4,sockets=2,cores=2,threads=1 \
-enable-kvm \
-name rhel7.2 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-monitor stdio \
-qmp tcp:0:6661,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-serial unix:/tmp/serial0,server,nowait \
-vga cirrus \
-vnc :0  \
-netdev tap,id=hostnet0,vhost=on,queues=4 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=42:55:4a:69:c6:e3,bus=pci.0,addr=0x3,vectors=10,mq=on \
-chardev socket,id=seabios,path=/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-usb \
-device usb-tablet,id=tablet0 \
-drive file=/home/win8r2/win8r2.raw,if=none,id=drive-virtio-disk0,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=pci.0,id=drive-virtio-disk0,drive=drive-virtio-disk0,bootindex=1 \

2.Boot vm with -incoming tcp:0:5800 in destination
3.migrate in the src host
4.shutdown from guest OS or "system_powerdown" after migration

Verified with 7.1 and the issue didn't reproduced.
Guest: win2008 r2
Host kernel: 3.10.0-229.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7.x86_64.rpm
1. deleted dataplane and keep network, boot cli with follow cli:
1. boot guest in src with cli;
-M pc \
-cpu SandyBridge \
-nodefaults -rtc base=utc -no-hpet \
-m 2G \
-smp 4,sockets=2,cores=2,threads=1 \
-enable-kvm \
-name rhel7.2 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-monitor stdio \
-qmp tcp:0:6661,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-serial unix:/tmp/serial0,server,nowait \
-vga cirrus \
-vnc :0  \
-netdev tap,id=hostnet0,vhost=on,queues=4 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=42:55:4a:69:c6:e3,bus=pci.0,addr=0x3,vectors=10,mq=on \
-chardev socket,id=seabios,path=/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-usb \
-device usb-tablet,id=tablet0 \
-drive file=/home/win8r2/win8r2.raw,if=none,id=drive-virtio-disk0,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=pci.0,id=drive-virtio-disk0,drive=drive-virtio-disk0,bootindex=1 \

2. Boot vm with -incoming tcp:0:5800
3. Copying some files 
4. migrated in the src host
5. Shutdown from guest or "system_powerdown" after migration 
---Guest OS shutdown successfully and qemu exit

Also verified it with delete network device option, guest also can be shutdown through guestOS shutdown or "system_powerdown"

Is the problem reproducible with ide device instead of virtio block device?
Can we try enabling NMICrashDump ( https://technet.microsoft.com/en-us/library/cc783271%28v=ws.10%29.aspx?f=255&MSPPError=-2147217396 )
generating a crash dump file with NMI qemu command after the system is hung?

Thanks,
Vadim.

Hi Jin,

Could you reply comment16?

Best Regards,
Junyi

Tested it and didn't reproduce the issue on 7.3.z version

[root@hp-z800-01 home]# uname -r
3.10.0-514.6.1.el7.x86_64
[root@hp-z800-01 home]# rpm -qa |grep qemu-kvm-rhev
qemu-kvm-rhev-2.6.0-28.el7_3.2.x86_64

Test scenario
1. Boot guest with command [1]
2. copy large files in guest
3. migrate during copying step
4. "system_powerdown" in hmp after migrate finished.

BTW: also test it with following test scenario 
a. without network and dataplane
b. with network but without dataplane

[1]
/usr/libexec/qemu-kvm \
-M pc \
-cpu Nehalem \
-nodefaults -rtc base=utc -no-hpet \
-m 2G \
-smp 4,sockets=2,cores=2,threads=1 \
-enable-kvm \
-name rhel7.2 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-monitor stdio \
-qmp tcp:0:6661,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-serial unix:/tmp/serial0,server,nowait \
-vga cirrus \
-vnc :0  \
-netdev tap,id=hostnet0,vhost=on,queues=4 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=1e:55:60:41:0c:37,bus=pci.0,addr=0x3,vectors=10,mq=on \
-chardev socket,id=seabios,path=/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-usb \
-device usb-tablet,id=tablet0 \
-object iothread,id=iothread0 \
-drive file=/mnt/win8r2.raw,if=none,id=drive-virtio-disk0,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=pci.0,id=drive-virtio-disk0,drive=drive-virtio-disk0,bootindex=1,iothread=iothread0 \
-object iothread,id=iothread1 \
-drive file=/mnt/en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_with_sp1_x64_dvd_617601.iso,if=none,media=cdrom,id=drive-ide0,format=raw \
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=0 \
-drive file=/usr/share/virtio-win/virtio-win-1.9.0.iso,if=none,media=cdrom,id=drive-ide1,format=raw \
-device ide-drive,bus=ide.0,unit=1,drive=drive-ide1,id=ide1 \


Thanks
Jing Zhao

Can we please check if it works on rhel 7.4?

Thanks,
Vadim.

(In reply to Vadim Rozenfeld from comment #20)
> Can we please check if it works on rhel 7.4?
> 
> Thanks,
> Vadim.

Hi Jing,

Could you have a try by using latest qemu-kvm-rhev? 

Best Regards,
Junyi

Tested it with following version and didn't reproduce the bz

Host:
  kernel-3.10.0-552.el7.x86_64
  qemu-kvm-rhev-2.8.0-3.el7.x86_64

Guest:
 en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_with_sp1_x64_dvd_617601.iso

Test Steps:

  1. Boot guest with qemu command [1]
  2. copy large files in guest during migration
  3. "system_powerdown" on hmp after migration 

[1]
/usr/libexec/qemu-kvm \
-M pc \
-cpu IvyBridge \
-nodefaults -rtc base=utc -no-hpet \
-m 2G \
-smp 4,sockets=2,cores=2,threads=1 \
-enable-kvm \
-name rhel7.4 \
-uuid 990ea161-6b67-47b2-b803-19fb01d30d12 \
-smbios type=1,manufacturer='Red Hat',product='RHEV Hypervisor',version=el6,serial=koTUXQrb,uuid=feebc8fd-f8b0-4e75-abc3-e63fcdb67170 \
-k en-us \
-monitor stdio \
-qmp tcp:0:6661,server,nowait \
-boot menu=on \
-bios /usr/share/seabios/bios.bin \
-serial unix:/tmp/serial0,server,nowait \
-vga cirrus \
-vnc :0  \
-netdev tap,id=hostnet0,vhost=on,queues=4 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=1e:55:60:41:0c:37,bus=pci.0,addr=0x3,vectors=10,mq=on \
-chardev socket,id=seabios,path=/tmp/seabios,server,nowait \
-device isa-debugcon,chardev=seabios,iobase=0x402 \
-usb \
-device usb-tablet,id=tablet0 \
-object iothread,id=iothread0 \
-drive file=/mnt/test/bz1263998/win8r2.raw,if=none,id=drive-virtio-disk0,format=raw,cache=none,werror=stop,rerror=stop \
-device virtio-blk-pci,bus=pci.0,id=drive-virtio-disk0,drive=drive-virtio-disk0,bootindex=1,iothread=iothread0 \
-object iothread,id=iothread1 \
-drive file=/mnt/en_windows_server_2008_r2_standard_enterprise_datacenter_and_web_with_sp1_x64_dvd_617601.iso,if=none,media=cdrom,id=drive-ide0,format=raw \
-device ide-drive,bus=ide.0,unit=0,drive=drive-ide0,id=ide0,bootindex=0 \
-drive file=/usr/share/virtio-win/virtio-win-1.9.0.iso,if=none,media=cdrom,id=drive-ide1,format=raw \
-device ide-drive,bus=ide.0,unit=1,drive=drive-ide1,id=ide1 \


Thanks
Jing Zhao