Bug 1029338 - fail to boot guest with a vmdk data image which converted by system image from qcow2 to vmdk
Summary: fail to boot guest with a vmdk data image which converted by system image fro...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm
Version: 7.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: rc
: ---
Assignee: Fam Zheng
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-11-12 08:16 UTC by Sibiao Luo
Modified: 2013-11-27 11:05 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-11-27 11:05:47 UTC
Target Upstream Version:

Attachments (Terms of Use)
guest-kernel-boot-logs.txt (24.40 KB, text/plain)
2013-11-12 08:17 UTC, Sibiao Luo 		no flags Details
View All Add an attachment (proposed patch, testcase, etc.)

Description Sibiao Luo 2013-11-12 08:16:28 UTC 
Description of problem:
install a qcow2 system image via RHEL-7.0-20131111.1 tree and covert it to vmdk format, then boot guest with this vmdk image attached but fail to boot up, if remove the vmdk disk which can boot up successfully.

Version-Release number of selected component (if applicable):
host info:
3.10.0-48.el7.x86_64
qemu-kvm-1.5.3-19.el7.x86_64
guest info:
OS tree: RHEL-7.0-20131111.1
3.10.0-48.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1.install a qcow2 system image and covert it to vmdk format.
# qemu-img info RHEL-7.0-20131111.1-Server-x86_64.qcow2
image: RHEL-7.0-20131111.1-Server-x86_64.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 3.7G
cluster_size: 65536
Format specific information:
    compat: 0.10
# qemu-img convert -p -f qcow2 RHEL-7.0-20131111.1-Server-x86_64.qcow2 -O vmdk RHEL-7.0-20131111.1-Server-x86_64.vmdk
    (100.00/100%)
# qemu-img info RHEL-7.0-20131111.1-Server-x86_64.vmdk
image: RHEL-7.0-20131111.1-Server-x86_64.vmdk
file format: vmdk
virtual size: 10G (10737418240 bytes)
disk size: 3.7G
Format specific information:
    cid: 1384241839
    parent cid: 4294967295
    create type: monolithicSparse
    extents:
        [0]:
            virtual size: 10737418240
            filename: RHEL-7.0-20131111.1-Server-x86_64.vmdk
            cluster size: 65536
            format: 
2.boot guest with this vmdk image attached and specify readonly=on for it.
e.g:...-drive file=/home/RHEL-7.0-20131111.1-Server-x86_64.vmdk,if=none,id=drive-data-disk,*format=vmdk*,cache=none,werror=stop,rerror=stop,*readonly=on* -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk

Actual results:
after step 2, guest will fail to boot up and VM status: paused (io-error). I will attach the guest kernel boot logs.
(qemu) c
(qemu) info block
drive-disk: removable=0 io-status=ok file=/home/RHEL-7.0-20131111.1-Server-x86_64.qcow2 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
drive-data-disk: removable=0 io-status=ok file=/home/RHEL-7.0-20131111.1-Server-x86_64.vmdk ro=1 drv=vmdk encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
floppy0: removable=1 locked=0 tray-open=0 [not inserted]
sd0: removable=1 locked=0 tray-open=0 [not inserted]
(qemu) 
(qemu) info status 
VM status: paused (io-error)
(qemu) info block
drive-disk: removable=0 io-status=ok file=/home/RHEL-7.0-20131111.1-Server-x86_64.qcow2 ro=0 drv=qcow2 encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
drive-data-disk: removable=0 io-status=failed file=/home/RHEL-7.0-20131111.1-Server-x86_64.vmdk ro=1 drv=vmdk encrypted=0 bps=0 bps_rd=0 bps_wr=0 iops=0 iops_rd=0 iops_wr=0
floppy0: removable=1 locked=0 tray-open=0 [not inserted]
sd0: removable=1 locked=0 tray-open=0 [not inserted]

Expected results:
It should boot up succcessfully without any problem.

Additional info:
# /usr/libexec/qemu-kvm -M pc -S -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -drive file=/home/RHEL-7.0-20131111.1-Server-x86_64.qcow2,if=none,id=drive-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device ide-drive,bus=ide.0,unit=0,drive=drive-disk,id=system-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,id=e1000-net-pci0,mac=00:01:02:B6:40:21,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x6 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -drive file=/home/RHEL-7.0-20131111.1-Server-x86_64.vmdk,if=none,id=drive-data-disk,format=vmdk,cache=none,werror=stop,rerror=stop,readonly=on -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :1 -spice disable-ticketing,port=5931 -monitor stdio
Comment 1 Sibiao Luo 2013-11-12 08:17:43 UTC 
Created attachment 822777 [details]
guest-kernel-boot-logs.txt
Comment 2 Fam Zheng 2013-11-12 08:31:08 UTC 
If I read it clearly, the guest is booted from attached virtio-blk-pci,drive=drive-data-disk,id=data-disk, which is read only.

You should make sure it boots from the qcow2 disk.

Fam
Comment 3 Sibiao Luo 2013-11-12 08:44:36 UTC 
(In reply to Fam Zheng from comment #2)
> If I read it clearly, the guest is booted from attached
> virtio-blk-pci,drive=drive-data-disk,id=data-disk, which is read only.
No, it boot from the system disk via ide interface, I can select it from seabios.
> You should make sure it boots from the qcow2 disk.
> 
Also tried to use qemu-img to create vmdk image and do the same testing as comment#0 that did not meet such issue.

# qemu-img create -f vmdk my-data-disk.vmdk 20G
Formatting 'my-data-disk.vmdk', fmt=vmdk size=21474836480 compat6=off zeroed_grain=off 

# qemu-img info my-data-disk.vmdk
image: my-data-disk.vmdk
file format: vmdk
virtual size: 20G (21474836480 bytes)
disk size: 16K
Format specific information:
    cid: 1384244787
    parent cid: 4294967295
    create type: monolithicSparse
    extents:
        [0]:
            virtual size: 21474836480
            filename: my-data-disk.vmdk
            cluster size: 65536
            format: 
e.g:...-drive file=/home/my-data-disk.vmdk,if=none,id=drive-data-disk,format=vmdk,cache=none,werror=stop,rerror=stop,readonly=on -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk

Best regards,
sluo
Comment 4 Fam Zheng 2013-11-12 09:08:25 UTC 
(In reply to Sibiao Luo from comment #3)
> (In reply to Fam Zheng from comment #2)
> > If I read it clearly, the guest is booted from attached
> > virtio-blk-pci,drive=drive-data-disk,id=data-disk, which is read only.
> No, it boot from the system disk via ide interface, I can select it from
> seabios.

No. I think your mirrored disks confuse the boot procedure.

You selected the right MBR from seabios, so the GRUB starts. But kernel probably looks for the root mount point by UUID, as you have mirrored disks, the filesystems on both have the same UUID and in this case it uses the VMDK disk. Could you try delete the original qcow2 that you used in conversion and install the guest again, and then attach the vmdk with the same step?

Fam

> > You should make sure it boots from the qcow2 disk.
> > 
> Also tried to use qemu-img to create vmdk image and do the same testing as
> comment#0 that did not meet such issue.
> 
> # qemu-img create -f vmdk my-data-disk.vmdk 20G
> Formatting 'my-data-disk.vmdk', fmt=vmdk size=21474836480 compat6=off
> zeroed_grain=off 
> 
> # qemu-img info my-data-disk.vmdk
> image: my-data-disk.vmdk
> file format: vmdk
> virtual size: 20G (21474836480 bytes)
> disk size: 16K
> Format specific information:
>     cid: 1384244787
>     parent cid: 4294967295
>     create type: monolithicSparse
>     extents:
>         [0]:
>             virtual size: 21474836480
>             filename: my-data-disk.vmdk
>             cluster size: 65536
>             format: 
> e.g:...-drive
> file=/home/my-data-disk.vmdk,if=none,id=drive-data-disk,format=vmdk,
> cache=none,werror=stop,rerror=stop,readonly=on -device
> virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk
Comment 5 Fam Zheng 2013-11-27 00:43:37 UTC 
Sibiao,

Any idea on comment 4?

Fam
Comment 6 Sibiao Luo 2013-11-27 03:05:48 UTC 
(In reply to Fam Zheng from comment #4)
> (In reply to Sibiao Luo from comment #3)
> > (In reply to Fam Zheng from comment #2)
> > > If I read it clearly, the guest is booted from attached
> > > virtio-blk-pci,drive=drive-data-disk,id=data-disk, which is read only.
> > No, it boot from the system disk via ide interface, I can select it from
> > seabios.
> 
> No. I think your mirrored disks confuse the boot procedure.
> 
> You selected the right MBR from seabios, so the GRUB starts. But kernel
> probably looks for the root mount point by UUID, as you have mirrored disks,
> the filesystems on both have the same UUID and in this case it uses the VMDK
> disk.
Maybe just as your said, but this just a problem, could we provide a method to prevent such thing which influence our KVM ?
> Could you try delete the original qcow2 that you used in conversion
> and install the guest again, and then attach the vmdk with the same step?
> 
Hit the same issue, and wait a moment, there will flood with "block I/O error in device 'drive-data-disk': Permission denied (13)" and VM was in paused (io-error) status.
# qemu-img convert -p -f qcow2 -O vmdk RHEL6.5-20131019.1_Server_x86_64.qcow2bk RHEL6.5-20131019.1_Server_x86_64.vmdk
    (100.00/100%)
# /usr/libexec/qemu-kvm -M pc -S -cpu host -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -no-kvm-pit-reinjection -usb -device usb-tablet,id=input0 -name sluo -uuid 990ea161-6b67-47b2-b803-19fb01d30d30 -rtc base=localtime,clock=host,driftfix=slew -device virtio-serial-pci,id=virtio-serial0,max_ports=16,vectors=0,bus=pci.0,addr=0x3 -chardev socket,id=channel1,path=/tmp/helloworld1,server,nowait -device virtserialport,chardev=channel1,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port1 -chardev socket,id=channel2,path=/tmp/helloworld2,server,nowait -device virtserialport,chardev=channel2,name=com.redhat.rhevm.vdsm,bus=virtio-serial0.0,id=port2 -drive file=/home/RHEL6.5-20131019.1_Server_x86_64.qcow2,if=none,id=drive-disk,format=qcow2,cache=none,aio=native,werror=stop,rerror=stop -device ide-drive,bus=ide.0,unit=0,drive=drive-disk,id=system-disk,bootindex=1 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device e1000,netdev=hostnet0,id=e1000-net-pci0,mac=00:01:02:B6:40:21,bus=pci.0,addr=0x5 -device virtio-balloon-pci,id=ballooning,bus=pci.0,addr=0x6 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -drive file=/home/RHEL6.5-20131019.1_Server_x86_64.vmdk,if=none,id=drive-data-disk,format=vmdk,cache=none,werror=stop,rerror=stop,readonly=on -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-data-disk,id=data-disk -k en-us -boot menu=on -qmp tcp:0:4444,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :1 -spice disable-ticketing,port=5931 -monitor stdio
QEMU 0.12.1 monitor - type 'help' for more information
(qemu) c
(qemu) block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)
block I/O error in device 'drive-data-disk': Permission denied (13)

(qemu) info status 
VM status: paused (io-error)

The guest kernel log:
...
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
virtio-pci 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, high) -> IRQ 10
usb 1-1: configuration #1 chosen from 1 choice
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
virtio-pci 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 10 (level, high) -> IRQ 10
virtio-pci 0000:00:07.0: PCI INT A -> Link[LNKC] -> GSI 10 (level, high) -> IRQ 10
input: QEMU 0.12.1 QEMU USB Tablet as /devices/pci0000:00/0000:00:01.2/usb1/1-1/1-1:1.0/input/input3
generic-usb 0003:0627:0001.0001: input,hidraw0: USB HID v0.01 Pointer [QEMU 0.12.1 QEMU USB Tablet] on usb-0000:00:01.2-1/input0
sd 0:0:0:0: [sda] 20971520 512-byte logical blocks: (10.7 GB/10.0 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
 vda: vda1 vda2
Refined TSC clocksource calibration: 3392.301 MHz.
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input4
dracut: Scanning devices sda2 vda2  for LVM logical volumes vg_dhcp8144/lv_swap vg_dhcp8144/lv_root 
dracut: Found duplicate PV XggZO1loCfI3XmZUC15yS7ZM8RP1faFK: using /dev/vda2 not /dev/sda2
dracut: inactive '/dev/vg_dhcp8144/lv_root' [8.51 GiB] inherit
dracut: inactive '/dev/vg_dhcp8144/lv_swap' [1.00 GiB] inherit
dracut: Found duplicate PV XggZO1loCfI3XmZUC15yS7ZM8RP1faFK: using /dev/vda2 not /dev/sda2
EXT4-fs (dm-0): INFO: recovery required on readonly filesystem
EXT4-fs (dm-0): write access will be enabled during recovery

Best Regards,
sluo
Comment 7 Sibiao Luo 2013-11-27 03:30:03 UTC 
Tried to install separately OS image or clone it from another image independently, just to make sure different UUID for the system image and VMDK data disk image. We can boot up guest successfully.
e.g:
RHEL7-64bit.qcow2   -------- system qcow2 image
RHEL6.5_x86_64.vmdk -------- VMDK image convert from a rhel6.5 qcow2 system image

Best Regards,
sluo
Note You need to log in before you can comment on or make changes to this bug.