Bug 1755340
| Summary: | Live storage migration fails with: TimeoutError: Timed out during operation: cannot acquire state change lock (held by monitor=remoteDispatchConnectGetAllDomainStats) and the VM becomes 'Not Responding' [rhel-7.7.z] | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | RAD team bot copy to z-stream <autobot-eus-copy> |
| Component: | qemu-kvm-rhev | Assignee: | Sergio Lopez <slopezpa> |
| Status: | CLOSED ERRATA | QA Contact: | aihua liang <aliang> |
| Severity: | urgent | Docs Contact: | |
| Priority: | urgent | ||
| Version: | 7.6 | CC: | aliang, aoconnor, bugs, bzlotnik, chayang, coli, ebenahar, frolland, jferlan, jinzhao, jsuchane, juzhang, mprivozn, mtessun, ngu, pkrempa, qzhang, sfishbai, slopezpa, tnisan, virt-maint |
| Target Milestone: | rc | Keywords: | Automation, Reopened, ZStream |
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | qemu-kvm-rhev-2.12.0-33.el7_7.5 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1665256 | Environment: | |
| Last Closed: | 2020-02-10 09:17:47 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1665256 | ||
| Bug Blocks: | |||
|
Description
RAD team bot copy to z-stream
2019-09-25 09:27:48 UTC
Verify in qemu-kvm-rhev-2.12.0-33.el7_7.5.x86_64, the problem has been resolved, set its status to "Verified".
Test Steps:
1.Create an empty image.
#qemu-img create -f qcow2 mirror.qcow2 20G
2.Start dst vm with mirror.qcow2(virtio-blk-pci, data_plane enable)
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-machine pc \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190624-022456-Uc0cnuxt,server,nowait \
-mon chardev=qmp_id_qmpmonitor1,mode=control \
-chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190624-022456-Uc0cnuxs,server,nowait \
-mon chardev=qmp_id_catch_monitor,mode=control \
-device pvpanic,ioport=0x505,id=idOSU1la \
-chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190624-022456-Uc0cnuxs,server,nowait \
-device isa-serial,chardev=serial_id_serial0 \
-chardev socket,id=seabioslog_id_20190624-022456-Uc0cnuxs,path=/var/tmp/seabios-20190624-022456-Uc0cnuxs,server,nowait \
-device isa-debugcon,chardev=seabioslog_id_20190624-022456-Uc0cnuxs,iobase=0x402 \
-device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
-object iothread,id=iothread0 \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,file=/home/mirror.qcow2 \
-device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=1,bus=pci.0,iothread=iothread0 \
-device virtio-net-pci,mac=9a:01:02:03:04:05,id=idk78PtS,vectors=4,netdev=idd7g5ik,bus=pci.0,addr=0x5 \
-netdev tap,id=idd7g5ik,vhost=on \
-m 5120 \
-smp 2,maxcpus=2,cores=1,threads=1,sockets=2 \
-cpu 'Nehalem',+kvm_pv_unhalt \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-vnc :1 \
-rtc base=utc,clock=host,driftfix=slew \
-boot menu=off,strict=off,order=cdn,once=c \
-enable-kvm \
-monitor stdio \
-incoming tcp:0:5000 \
3.Start src guest with virtio_blk device, data_plane enable.
/usr/libexec/qemu-kvm \
-name 'avocado-vt-vm1' \
-machine pc \
-nodefaults \
-device VGA,bus=pci.0,addr=0x2 \
-chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20190624-022456-Uc0cnuxs,server,nowait \
-mon chardev=qmp_id_qmpmonitor1,mode=control \
-chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20190624-022456-Uc0cnuxs,server,nowait \
-mon chardev=qmp_id_catch_monitor,mode=control \
-device pvpanic,ioport=0x505,id=idOSU1la \
-chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20190624-022456-Uc0cnuxs,server,nowait \
-device isa-serial,chardev=serial_id_serial0 \
-chardev socket,id=seabioslog_id_20190624-022456-Uc0cnuxs,path=/var/tmp/seabios-20190624-022456-Uc0cnuxs,server,nowait \
-device isa-debugcon,chardev=seabioslog_id_20190624-022456-Uc0cnuxs,iobase=0x402 \
-device nec-usb-xhci,id=usb1,bus=pci.0,addr=0x3 \
-object iothread,id=iothread0 \
-drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,file=/home/kvm_autotest_root/images/rhel77-64-virtio-scsi.qcow2 \
-device virtio-blk-pci,id=image1,drive=drive_image1,bootindex=1,bus=pci.0,iothread=iothread0 \
-device virtio-net-pci,mac=9a:01:02:03:04:05,id=idk78PtS,vectors=4,netdev=idd7g5ik,bus=pci.0,addr=0x5 \
-netdev tap,id=idd7g5ik,vhost=on \
-m 5120 \
-smp 2,maxcpus=2,cores=1,threads=1,sockets=2 \
-cpu 'Nehalem',+kvm_pv_unhalt \
-device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
-vnc :0 \
-rtc base=utc,clock=host,driftfix=slew \
-boot menu=off,strict=off,order=cdn,once=c \
-enable-kvm \
-monitor stdio \
4.Do mirror from src to dst.
{ "execute": "drive-mirror", "arguments": { "device": "drive_image1","target":"/home/mirror.qcow2","sync":"full","mode":"existing"}}
5.Set migration capabilities.
{"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"pause-before-switchover","state":true}]}}
6.Migrate from src to dst.
{ "execute": "migrate", "arguments": { "uri": "tcp:localhost:5000"}}
{"return": {}}
7.Wait some minutes, check migrate status
{"execute":"qury-migrate"}
{"return": {"expected-downtime": 300, "status": "active", "setup-time": 328, "total-time": 20992, "ram": {"total": 7533830144, "postcopy-requests": 0, "dirty-sync-count": 1, "page-size": 4096, "remaining": 1263505408, "mbps": 268.5672, "transferred": 689773714, "duplicate": 1365763, "dirty-pages-rate": 0, "skipped": 0, "normal-bytes": 676159488, "normal": 165078}}}
{"timestamp": {"seconds": 1569389952, "microseconds": 920203}, "event": "JOB_STATUS_CHANGE", "data": {"status": "standby", "id": "drive_image1"}}
{"timestamp": {"seconds": 1569389952, "microseconds": 920339}, "event": "JOB_STATUS_CHANGE", "data": {"status": "ready", "id": "drive_image1"}}
{"timestamp": {"seconds": 1569389952, "microseconds": 920563}, "event": "STOP"}
8. Continue migration
{"execute":"migrate-continue","arguments":{"state":"pre-switchover"}}
9. Check src vm status
(qemu) info status
VM status: paused (postmigrate)
10. Check dst vm status
(qemu) info status
VM status: running
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2020:0457 |