Bug 1130483
| Summary: | qemu failed to reopen target image after drive-mirror | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | Xiaoqing Wei <xwei> |
| Component: | qemu-kvm | Assignee: | Jeff Cody <jcody> |
| Status: | CLOSED WONTFIX | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 6.6 | CC: | ailan, chayang, juzhang, mkenneth, mzhan, qzhang, rbalakri, rpacheco, virt-maint |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-12-06 10:21:39 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1359965 | ||
Red Hat Enterprise Linux 6 is in the Production 3 Phase. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available. The official life cycle policy can be reviewed here: http://redhat.com/rhel/lifecycle This issue does not meet the inclusion criteria for the Production 3 Phase and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Note that a strong business justification will be required for re-evaluation. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL: https://access.redhat.com/ |
Description of problem: qemu failed to reopen target image after drive-mirror Version-Release number of selected component (if applicable): qemu-kvm-rhev-tools-0.12.1.2-2.438.el6.x86_64 qemu-kvm-rhev-0.12.1.2-2.438.el6.x86_64 qemu-kvm-rhev-debuginfo-0.12.1.2-2.438.el6.x86_64 qemu-img-rhev-0.12.1.2-2.438.el6.x86_64 also tested 6.5.z(_5.14) and 6.5 GA How reproducible: 100% Steps to Reproduce: 1. create LVMs, 10GB * 1, 32GB * 1 # lvscan ACTIVE '/dev/VolGroup/LogVol_swap' [12.00 GiB] inherit ACTIVE '/dev/VolGroup/LogVol_root' [409.25 GiB] inherit ACTIVE '/dev/VolGroup/lv10g' [10.00 GiB] inherit ACTIVE '/dev/VolGroup/raw30g' [32.00 GiB] inherit 2. having a guest img, virtual size 20G, actual size < 10G # qemu-img info /mnt/linux/RHEL-Server-6.5-64-virtio.qcow2 image: /mnt/linux/RHEL-Server-6.5-64-virtio.qcow2 file format: qcow2 virtual size: 20G (21474836480 bytes) disk size: 3.6G cluster_size: 65536 3. convert img from file based to lvm based. #qemu-img convert -f qcow2 -O qcow2 /mnt/linux/RHEL-Server-6.5-64-virtio.qcow2 /dev/VolGroup/lv10g -p 4. boot w/ lvm based /usr/bin/qemu-kvm -monitor stdio \ -S \ -name 'virt-tests-vm1' \ -nodefaults \ -vga std \ -M rhel6.6.0 \ -chardev socket,id=qmp_id_qmp1,path=/tmp/monitor-qmp1-20140812-133800-bf4OXeCV,server,nowait \ -mon chardev=qmp_id_qmp1,mode=control \ -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140812-133800-bf4OXeCV,server,nowait \ -device isa-serial,chardev=serial_id_serial0 \ -chardev socket,id=seabioslog_id_20140812-133800-bf4OXeCV,path=/tmp/seabios-20140812-133800-bf4OXeCV,server,nowait \ -device isa-debugcon,chardev=seabioslog_id_20140812-133800-bf4OXeCV,iobase=0x402 \ -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=04 \ \ \ -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,werror=stop,rerror=stop,format=qcow2,file='/dev/VolGroup/lv10g' \ \ \ -device ide-drive,id=image1,drive=drive_image1,bus=ide.0,unit=0 \ -device virtio-net-pci,mac=9a:9b:9c:9d:9e:9f,id=idokVqY6,vectors=4,netdev=idcsqU1f,bus=pci.0,addr=05 \ -netdev tap,id=idcsqU1f,vhost=on \ -m 4096 \ -smp 4,maxcpus=4,cores=2,threads=1,sockets=2 \ -cpu 'SandyBridge' \ -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso \ -device ide-drive,id=cd1,drive=drive_cd1,bus=ide.0,unit=1 \ -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \ -vnc :0 \ -rtc base=localtime,clock=host,driftfix=slew \ -boot order=cdn,once=c,menu=off \ -enable-kvm 5. consume more space, make LVM out of space, and qemu pauses VM(io-error) # dd if=/dev/urandom of=/root/4GB.img bs=1M count=4096 6. mirror LVM to the one has enough space. # nc -U /tmp/monitor-qmp1-20140812-133800-bf4OXeCV {"QMP": {"version": {"qemu": {"micro": 1, "minor": 12, "major": 0}, "package": "(qemu-kvm-rhev-0.12.1.2-2.438.el6)"}, "capabilities": []}} {'execute': 'qmp_capabilities', 'id': 'XNISVhf3'} {"return": {}, "id": "XNISVhf3"} {"execute": "query-block-jobs", "id": "K4RBuwiZ"} {"return": [], "id": "K4RBuwiZ"} {"execute": "__com.redhat_drive-mirror", "arguments": {"device": "drive_image1", "format": "qcow2", "full": true, "target": "/dev/VolGroup/raw30g", "mode": "absolute-paths"}, "id": "BH2iL2Kl"} {"id": "BH2iL2Kl", "error": {"class": "GenericError", "desc": "/dev/VolGroup/raw30g: error while creating qcow2: Invalid argument", "data": {"message": "/dev/VolGroup/raw30g: error while creating qcow2: Invalid argument"}}} {"execute": "__com.redhat_drive-mirror", "arguments": {"device": "drive_image1", "format": "raw", "full": true, "target": "/dev/VolGroup/raw30g", "mode": "absolute-paths"}, "id": "BH2iL2Kl"} {"return": {}, "id": "BH2iL2Kl"} 7. wait till mirror done (qemu) block I/O error in device 'drive_image1': No space left on device (28) (qemu) (qemu) Formatting '/dev/VolGroup/raw30g', fmt=qcow2 size=21474836480 encryption=off cluster_size=65536 (qemu) (qemu) info block-jobs No active jobs (qemu) Formatting '/dev/VolGroup/raw30g', fmt=raw size=21474836480 info block-jobs Type mirror, device drive_image1: Completed 10972233728 of 21474836480 bytes, speed limit 0 bytes/s (qemu) info status VM status: paused (io-error) (qemu) info block-jobs Type mirror, device drive_image1: Completed 21474836480 of 21474836480 bytes, speed limit 0 bytes/s (qemu) info block-jobs Type mirror, device drive_image1: Completed 21474836480 of 21474836480 bytes, speed limit 0 bytes/s 8. switch VM storage to target LVM(or called reopen) {"execute": "__com.redhat_drive-reopen", "arguments": {"device": "drive_image1", "new-image-file": "/dev/VolGroup/raw30g", "format": "raw"}} {"timestamp": {"seconds": 1408098876, "microseconds": 761571}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 21474836480, "offset": 21474836480, "speed": 0, "type": "mirror"}} {"error": {"class": "IOError", "desc": "An IO error has occurred", "data": {}}} 9. recheck VM storage after 8. raise error (qemu) info block-jobs No active jobs (qemu) info block drive_image1: removable=0 io-status=nospace file=/dev/VolGroup/lv10g ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0 drive_cd1: removable=1 locked=1 tray-open=0 io-status=ok file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso ro=1 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0 (qemu) info status VM status: paused (io-error) Actual results: as 8. outputs, qemu still io-error and 9. the underlying storage didn't switched. Expected results: 9. should shown underlying storage switched, and 8. should not error happen. Additional info: identical test performed and error on 6.5.Z 6.5 GA 6.4 GA identical test performed and GOOD on RHEL 7.1 W/ RHEV 7.1 pkg libcacard-tools-rhev-2.1.0-1.el7.x86_64 qemu-img-rhev-2.1.0-1.el7.x86_64 qemu-kvm-common-rhev-2.1.0-1.el7.x86_64 libcacard-devel-rhev-2.1.0-1.el7.x86_64 libcacard-rhev-2.1.0-1.el7.x86_64 qemu-kvm-rhev-2.1.0-1.el7.x86_64 qemu-kvm-tools-rhev-2.1.0-1.el7.x86_64 qemu-kvm-rhev-debuginfo-2.1.0-1.el7.x86_64 (qemu) info status VM status: running (qemu) block I/O error in device 'drive_image1': No space left on device (28) Formatting '/dev/rhel/raw30g', fmt=qcow2 size=32212254720 encryption=off cluster_size=65536 lazy_refcounts=off info block drive_image1: /dev/rhel/lv10g (qcow2) I/O status: nospace drive_cd1: /root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso (raw, read-only) Removable device: not locked, tray closed (qemu) info block-jobs Type mirror, device drive_image1: Completed 21831286784 of 32212254720 bytes, speed limit 0 bytes/s # nc -U /tmp/monitor-qmpmonitor1-20140815-110559-kPOHZHpD {"QMP": {"version": {"qemu": {"micro": 0, "minor": 1, "major": 2}, "package": " (qemu-kvm-2.1.0-1.el7)"}, "capabilities": []}} {'execute': 'qmp_capabilities', 'id': 'XNISVhf3'} {"return": {}, "id": "XNISVhf3"} {"execute": "drive-mirror", "arguments": {"device": "drive_image1", "mode": "absolute-paths", "format": "qcow2", "target": "/dev/rhel/raw30g", "sync": "full"}, "id": "rUUcP4XX"} {"return": {}, "id": "rUUcP4XX"} {"timestamp": {"seconds": 1408092522, "microseconds": 227620}, "event": "BLOCK_JOB_READY", "data": {"device": "drive_image1", "len": 32212254720, "offset": 32212254720, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": 1408092535, "microseconds": 154378}, "event": "RESUME"} {"timestamp": {"seconds": 1408092535, "microseconds": 166620}, "event": "BLOCK_IO_ERROR", "data": {"device": "drive_image1", "__com.redhat_debug_info": {"message": "No space left on device", "errno": 28}, "__com.redhat_reason": "enospc", "operation": "write", "action": "stop"}} {"timestamp": {"seconds": 1408092535, "microseconds": 166835}, "event": "STOP"} {"execute": "block-job-complete", "arguments": { "device": "drive_image1"} } {"return": {}} {"timestamp": {"seconds": 1408093092, "microseconds": 102092}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 32212254720, "offset": 32212254720, "speed": 0, "type": "mirror"}} {"timestamp": {"seconds": 1408093105, "microseconds": 762376}, "event": "RESUME"} (qemu) info block drive_image1: /dev/rhel/raw30g (qcow2) drive_cd1: /root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso (raw, read-only) Removable device: not locked, tray closed (qemu) info status VM status: running (qemu)