RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1130483 - qemu failed to reopen target image after drive-mirror
Summary: qemu failed to reopen target image after drive-mirror
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm
Version: 6.6
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Jeff Cody
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks: 1359965
TreeView+ depends on / blocked
 
Reported: 2014-08-15 10:58 UTC by Xiaoqing Wei
Modified: 2017-12-06 10:21 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-06 10:21:39 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Xiaoqing Wei 2014-08-15 10:58:50 UTC 
Description of problem:
qemu failed to reopen target image after drive-mirror

Version-Release number of selected component (if applicable):
qemu-kvm-rhev-tools-0.12.1.2-2.438.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.438.el6.x86_64
qemu-kvm-rhev-debuginfo-0.12.1.2-2.438.el6.x86_64
qemu-img-rhev-0.12.1.2-2.438.el6.x86_64
also tested 6.5.z(_5.14)
and 6.5 GA

How reproducible:
100%

Steps to Reproduce:
1. create LVMs, 10GB * 1, 32GB * 1
# lvscan 
  ACTIVE            '/dev/VolGroup/LogVol_swap' [12.00 GiB] inherit
  ACTIVE            '/dev/VolGroup/LogVol_root' [409.25 GiB] inherit
  ACTIVE            '/dev/VolGroup/lv10g' [10.00 GiB] inherit
  ACTIVE            '/dev/VolGroup/raw30g' [32.00 GiB] inherit

2. having a guest img, virtual size 20G, actual size < 10G
# qemu-img info /mnt/linux/RHEL-Server-6.5-64-virtio.qcow2 
image: /mnt/linux/RHEL-Server-6.5-64-virtio.qcow2
file format: qcow2
virtual size: 20G (21474836480 bytes)
disk size: 3.6G
cluster_size: 65536

3. convert img from file based to lvm based.
#qemu-img convert -f qcow2 -O qcow2 /mnt/linux/RHEL-Server-6.5-64-virtio.qcow2 /dev/VolGroup/lv10g -p

4. boot w/ lvm based
/usr/bin/qemu-kvm -monitor stdio \
    -S  \
    -name 'virt-tests-vm1' \
    -nodefaults  \
    -vga std \
    -M rhel6.6.0  \
    -chardev socket,id=qmp_id_qmp1,path=/tmp/monitor-qmp1-20140812-133800-bf4OXeCV,server,nowait \
    -mon chardev=qmp_id_qmp1,mode=control  \
    -chardev socket,id=serial_id_serial0,path=/tmp/serial-serial0-20140812-133800-bf4OXeCV,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20140812-133800-bf4OXeCV,path=/tmp/seabios-20140812-133800-bf4OXeCV,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20140812-133800-bf4OXeCV,iobase=0x402 \
    -device ich9-usb-uhci1,id=usb1,bus=pci.0,addr=04 \
\
\
    -drive id=drive_image1,if=none,cache=none,snapshot=off,aio=native,werror=stop,rerror=stop,format=qcow2,file='/dev/VolGroup/lv10g' \
\
\
    -device ide-drive,id=image1,drive=drive_image1,bus=ide.0,unit=0 \
    -device virtio-net-pci,mac=9a:9b:9c:9d:9e:9f,id=idokVqY6,vectors=4,netdev=idcsqU1f,bus=pci.0,addr=05  \
    -netdev tap,id=idcsqU1f,vhost=on \
    -m 4096  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'SandyBridge' \
    -drive id=drive_cd1,if=none,snapshot=off,aio=native,media=cdrom,file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso \
    -device ide-drive,id=cd1,drive=drive_cd1,bus=ide.0,unit=1 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off \
    -enable-kvm

5. consume more space, make LVM out of space, and qemu pauses VM(io-error)
# dd if=/dev/urandom of=/root/4GB.img bs=1M count=4096

6. mirror LVM to the one has enough space.
# nc -U /tmp/monitor-qmp1-20140812-133800-bf4OXeCV 
{"QMP": {"version": {"qemu": {"micro": 1, "minor": 12, "major": 0}, "package": "(qemu-kvm-rhev-0.12.1.2-2.438.el6)"}, "capabilities": []}}
{'execute': 'qmp_capabilities', 'id': 'XNISVhf3'}
{"return": {}, "id": "XNISVhf3"}
{"execute": "query-block-jobs", "id": "K4RBuwiZ"}
{"return": [], "id": "K4RBuwiZ"}
{"execute": "__com.redhat_drive-mirror", "arguments": {"device": "drive_image1", "format": "qcow2", "full": true, "target": "/dev/VolGroup/raw30g", "mode": "absolute-paths"}, "id": "BH2iL2Kl"}
{"id": "BH2iL2Kl", "error": {"class": "GenericError", "desc": "/dev/VolGroup/raw30g: error while creating qcow2: Invalid argument", "data": {"message": "/dev/VolGroup/raw30g: error while creating qcow2: Invalid argument"}}}
{"execute": "__com.redhat_drive-mirror", "arguments": {"device": "drive_image1", "format": "raw", "full": true, "target": "/dev/VolGroup/raw30g", "mode": "absolute-paths"}, "id": "BH2iL2Kl"}
{"return": {}, "id": "BH2iL2Kl"}

7. wait till mirror done

(qemu) block I/O error in device 'drive_image1': No space left on device (28)

(qemu) 
(qemu) Formatting '/dev/VolGroup/raw30g', fmt=qcow2 size=21474836480 encryption=off cluster_size=65536 

(qemu) 
(qemu) info block-jobs 
No active jobs
(qemu) Formatting '/dev/VolGroup/raw30g', fmt=raw size=21474836480 
info block-jobs 
Type mirror, device drive_image1: Completed 10972233728 of 21474836480 bytes, speed limit 0 bytes/s
(qemu) info status 
VM status: paused (io-error)
(qemu) info block-jobs 
Type mirror, device drive_image1: Completed 21474836480 of 21474836480 bytes, speed limit 0 bytes/s
(qemu) info block-jobs 
Type mirror, device drive_image1: Completed 21474836480 of 21474836480 bytes, speed limit 0 bytes/s

8. switch VM storage to target LVM(or called reopen)
{"execute": "__com.redhat_drive-reopen", "arguments": {"device": "drive_image1", "new-image-file": "/dev/VolGroup/raw30g", "format": "raw"}}
{"timestamp": {"seconds": 1408098876, "microseconds": 761571}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 21474836480, "offset": 21474836480, "speed": 0, "type": "mirror"}}
{"error": {"class": "IOError", "desc": "An IO error has occurred", "data": {}}}

9. recheck VM storage after 8. raise error
(qemu) info block-jobs 
No active jobs
(qemu) info block
drive_image1: removable=0 io-status=nospace file=/dev/VolGroup/lv10g ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
drive_cd1: removable=1 locked=1 tray-open=0 io-status=ok file=/root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso ro=1 drv=raw encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
(qemu) info status 
VM status: paused (io-error)


Actual results:
as 8. outputs, qemu still io-error
and 9. the underlying storage didn't switched.

Expected results:
9. should shown underlying storage switched, and
8. should not error happen.

Additional info:
identical test performed and error on
6.5.Z
6.5 GA
6.4 GA


identical test performed and GOOD on
RHEL 7.1 W/ RHEV 7.1 pkg
libcacard-tools-rhev-2.1.0-1.el7.x86_64
qemu-img-rhev-2.1.0-1.el7.x86_64
qemu-kvm-common-rhev-2.1.0-1.el7.x86_64
libcacard-devel-rhev-2.1.0-1.el7.x86_64
libcacard-rhev-2.1.0-1.el7.x86_64
qemu-kvm-rhev-2.1.0-1.el7.x86_64
qemu-kvm-tools-rhev-2.1.0-1.el7.x86_64
qemu-kvm-rhev-debuginfo-2.1.0-1.el7.x86_64


(qemu) info status 
VM status: running
(qemu) block I/O error in device 'drive_image1': No space left on device (28)
Formatting '/dev/rhel/raw30g', fmt=qcow2 size=32212254720 encryption=off cluster_size=65536 lazy_refcounts=off 
info block
drive_image1: /dev/rhel/lv10g (qcow2)
    I/O status:       nospace

drive_cd1: /root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso (raw, read-only)
    Removable device: not locked, tray closed
(qemu) info block-jobs 
Type mirror, device drive_image1: Completed 21831286784 of 32212254720 bytes, speed limit 0 bytes/s



# nc -U /tmp/monitor-qmpmonitor1-20140815-110559-kPOHZHpD 
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 1, "major": 2}, "package": " (qemu-kvm-2.1.0-1.el7)"}, "capabilities": []}}
{'execute': 'qmp_capabilities', 'id': 'XNISVhf3'}
{"return": {}, "id": "XNISVhf3"}
{"execute": "drive-mirror", "arguments": {"device": "drive_image1", "mode": "absolute-paths", "format": "qcow2", "target": "/dev/rhel/raw30g", "sync": "full"}, "id": "rUUcP4XX"}
{"return": {}, "id": "rUUcP4XX"}
{"timestamp": {"seconds": 1408092522, "microseconds": 227620}, "event": "BLOCK_JOB_READY", "data": {"device": "drive_image1", "len": 32212254720, "offset": 32212254720, "speed": 0, "type": "mirror"}}
{"timestamp": {"seconds": 1408092535, "microseconds": 154378}, "event": "RESUME"}
{"timestamp": {"seconds": 1408092535, "microseconds": 166620}, "event": "BLOCK_IO_ERROR", "data": {"device": "drive_image1", "__com.redhat_debug_info": {"message": "No space left on device", "errno": 28}, "__com.redhat_reason": "enospc", "operation": "write", "action": "stop"}}
{"timestamp": {"seconds": 1408092535, "microseconds": 166835}, "event": "STOP"}
{"execute": "block-job-complete", "arguments": { "device": "drive_image1"} }
{"return": {}}
{"timestamp": {"seconds": 1408093092, "microseconds": 102092}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 32212254720, "offset": 32212254720, "speed": 0, "type": "mirror"}}
{"timestamp": {"seconds": 1408093105, "microseconds": 762376}, "event": "RESUME"}


(qemu) info block
drive_image1: /dev/rhel/raw30g (qcow2)

drive_cd1: /root/staf-kvm-devel/autotest-devel/client/tests/virt/shared/data/isos/windows/winutils.iso (raw, read-only)
    Removable device: not locked, tray closed
(qemu) info status 
VM status: running
(qemu)
Comment 7 Jan Kurik 2017-12-06 10:21:39 UTC 
Red Hat Enterprise Linux 6 is in the Production 3 Phase. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available.

The official life cycle policy can be reviewed here:

http://redhat.com/rhel/lifecycle

This issue does not meet the inclusion criteria for the Production 3 Phase and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Note that a strong business justification will be required for re-evaluation. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL:

https://access.redhat.com/
Note You need to log in before you can comment on or make changes to this bug.