RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 914802 - Support backup vendors in qemu to access qcow disk readonly (qemu-img metadata dump)
Summary: Support backup vendors in qemu to access qcow disk readonly (qemu-img metadat...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: qemu-kvm
Version: 6.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: ---
Assignee: Paolo Bonzini
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 914801 1010610
Blocks: 883503 883504 896690 906042 960054 963405 977088 989646 1000882
TreeView+ depends on / blocked
 
Reported: 2013-02-22 19:30 UTC by Bhavna Sarathy
Modified: 2015-06-11 01:10 UTC (History)
28 users (show)

Fixed In Version: qemu-kvm-0.12.1.2-2.403.el6
Doc Type: Release Note
Doc Text:
Support for Dumping Metadata of Virtual Disks This low-level feature introduces a new command option "qemu-img map" to dump all the information that is required for qcow2 images to be mapped to a block device via LVM. As a result, virtual machine images (with the virtual machine shutdown) can be accessed as block devices (or directly) without knowing the details of the qcow2 image format. This is useful to let backup applications read the contents of qcow2 guest images.
Clone Of: 906042
: 977088 989646 (view as bug list)
Environment:
Last Closed: 2013-11-21 06:38:10 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
testing tool (13.64 KB, application/x-gzip)
2013-09-19 12:38 UTC, Paolo Bonzini 		no flags Details
testing tool v2 (14.44 KB, application/gzip)
2013-10-09 19:11 UTC, Paolo Bonzini 		no flags Details
Show Obsolete (1) View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2013:1553 0 normal SHIPPED_LIVE Important: qemu-kvm security, bug fix, and enhancement update 2013-11-20 21:40:29 UTC

Comment 2 RHEL Program Management 2013-03-01 06:47:19 UTC 
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.
Comment 4 Paolo Bonzini 2013-05-22 08:50:09 UTC 
Yes, I'll add instructions once the patches are closer to being ready.
Comment 14 Xu Tian 2013-07-05 08:04:27 UTC 
Hi Paolo,

What's status of this feature?

Thanks,
Xu
Comment 25 Xu Tian 2013-09-18 07:27:41 UTC 
Hi Paolo,

I'd like to verify this feature on build qemu-kvm-0.12.1.2-2.403.el6, would you give us a steps about how to verify this feature?

Thanks,
Xu
Comment 27 Paolo Bonzini 2013-09-19 12:38:55 UTC 
Created attachment 799911 [details]
testing tool

The attached program can be used to test "qemu-img map".  It will convert an image to raw, similar to "qemu-img convert" but using the output of "qemu-img map".  You can test its result against "qemu-img convert".

I suggest testing the following cases:

(1) raw image

(2) qcow2 image

(3) qcow2 image with raw backing file

(4) qcow2 image with qcow2 backing file

(5) qcow2 image with raw backing file, after doing block_resize of the image

(6) qcow2 image with qcow2 backing file, after doing block_resize of the image
Comment 28 Xu Tian 2013-09-22 02:53:02 UTC 
(In reply to Paolo Bonzini from comment #27)
> Created attachment 799911 [details]
> testing tool
> 
> The attached program can be used to test "qemu-img map".  It will convert an
> image to raw, similar to "qemu-img convert" but using the output of
> "qemu-img map".  You can test its result against "qemu-img convert".
> 
> I suggest testing the following cases:
> 
> (1) raw image
> 
> (2) qcow2 image
> 
> (3) qcow2 image with raw backing file
> 
> (4) qcow2 image with qcow2 backing file
> 
> (5) qcow2 image with raw backing file, after doing block_resize of the image
> 
> (6) qcow2 image with qcow2 backing file, after doing block_resize of the
> image

Hi Paolo,

it looks tool in attachment doesn't for qemu-img-0.12.1.2-2.403.el6.x86_64 package;

[root@localhost qemu-to-raw]# ./qemu-to-raw.py  RHEL-Server-6.4-64-virtio.qcow2  cvt_6.4.raw
info: invalid option -- '-'

check source code found that  "qemu-img info --output=json --backing-chain RHEL-Server-6.4-64-virtio.qcow2" doesn't work because  options '--output' and '--backing-chain' not support by qemu-img-0.12.1.2-2.403.el6.x86_64;

would you like to update the test tool,

Thanks very much;

Xu
Comment 29 Xu Tian 2013-10-08 05:58:34 UTC 
Hi Paolo,

Verfiy this feature on qemu-img-0.12.1.2-2.409.el6.x86_64 with your tool in attachment, always hang at get qemu-img map results, would you help to update the tool.

yajl version 2.0.

[root@localhost qemu-to-raw]# ps aux |grep qemu-to-raw.py
root      6868  0.0  0.0 148344  6372 pts/0    S    21:33   0:00 python qemu-to-raw.py RHEL-Server-6.4-64-virtio.qcow2 target.raw

[root@localhost qemu-to-raw]# strace -p 6868
Process 6868 attached - interrupt to quit
wait4(6873, 

[root@localhost qemu-to-raw]# ps aux |grep 6873
root      6873  0.0  0.0  51976  2368 pts/0    S    21:33   0:00 qemu-img map --output=json RHEL-Server-6.4-64-virtio.qcow2

execute "qemu-img map --output=json RHEL-Server-6.4-64-virtio.qcow2" I can get json strings like:
[{ "start": 0, "length": 720896, "depth": 0, "zero": false, "data": true, "offset": 327680},
{ "start": 720896, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 2162688},
{ "start": 1769472, "length": 196608, "depth": 0, "zero": false, "data": true, "offset": 4063232},
....
{ "start": 21464350720, "length": 9437184, "depth": 0, "zero": true, "data": false},
{ "start": 21473787904, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": 1114112}]

thanks,
Xu
Comment 30 Xu Tian 2013-10-08 05:59:33 UTC 
(In reply to xu from comment #29)
> Hi Paolo,
> 
> Verfiy this feature on qemu-img-0.12.1.2-2.409.el6.x86_64 with your tool in
> attachment, always hang at get qemu-img map results, would you help to
> update the tool.
> 
> yajl version 2.0.
yajl version 2.0.3
> 
> [root@localhost qemu-to-raw]# ps aux |grep qemu-to-raw.py
> root      6868  0.0  0.0 148344  6372 pts/0    S    21:33   0:00 python
> qemu-to-raw.py RHEL-Server-6.4-64-virtio.qcow2 target.raw
> 
> [root@localhost qemu-to-raw]# strace -p 6868
> Process 6868 attached - interrupt to quit
> wait4(6873, 
> 
> [root@localhost qemu-to-raw]# ps aux |grep 6873
> root      6873  0.0  0.0  51976  2368 pts/0    S    21:33   0:00 qemu-img
> map --output=json RHEL-Server-6.4-64-virtio.qcow2
> 
> execute "qemu-img map --output=json RHEL-Server-6.4-64-virtio.qcow2" I can
> get json strings like:
> [{ "start": 0, "length": 720896, "depth": 0, "zero": false, "data": true,
> "offset": 327680},
> { "start": 720896, "length": 1048576, "depth": 0, "zero": false, "data":
> true, "offset": 2162688},
> { "start": 1769472, "length": 196608, "depth": 0, "zero": false, "data":
> true, "offset": 4063232},
> ....
> { "start": 21464350720, "length": 9437184, "depth": 0, "zero": true, "data":
> false},
> { "start": 21473787904, "length": 1048576, "depth": 0, "zero": false,
> "data": true, "offset": 1114112}]
> 
> thanks,
> Xu
Comment 31 Paolo Bonzini 2013-10-09 19:11:21 UTC 
Created attachment 810122 [details]
testing tool v2

Testing tool, now working with RHEL6's older Python.  Also yajl is not required anymore.
Comment 32 Xu Tian 2013-10-10 03:26:04 UTC 
(In reply to Paolo Bonzini from comment #31)
> Created attachment 810122 [details]
> testing tool v2
> 
> Testing tool, now working with RHEL6's older Python.  Also yajl is not
> required anymore.

Hi Paolo,

thanks for your qucik fix,  I will verify this feature today;

Thanks,
Xu
Comment 33 Xu Tian 2013-10-10 11:26:34 UTC 
Test below scenarios both qcow2 and raw image file
scenarios 1：

test pass when image format both qcow2 and raw:

1. boot guest with raw/qcow2 image, then create 400M file in "/root" folder;
   
/usr/libexec/qemu-kvm \
-drive file=./rhel7.0_x86_64.qcow2,if=none,cache=none,format=qcow2,media=disk,id=d1\
-device virtio-blk-pci,drive=d1,id=hd1 \
-smp 2 -m 1024 -vnc :1 \
-enable-kvm \
-qmp tcp:0:1234,server,nowait \
-monitor stdio

login guest, create big file
dd if=/dev/urandom of=/root/test.img bs=4M count=100; sync; 

recording md5sum of the file 

cd /root/ && md5sum test.img > test.md5

2. convert image to raw
    python  qemu-to-raw/qemu-to-raw.py ./rhel7.0_x86_64.qcow2 target.raw
3. map image target.raw to logcial volume
    losetup `losetup -f` target.raw
    losetup -j target.raw (get loop device is /dev/loop0)
    kpartx -av /dev/loop0
    pvscan; vgscan; lvscan
4. active  the logical volume
    lvchange -ay /dev/rhel/root (lv name)
5. check md5sum not changed
    mount /dev/rhel/root /mnt
    cd /mnt/root && md5sum -c test.md5 
    md5sum check OK

scenarios 2：
test pass when image has qcow2/raw backingfile;

1. create snapshot for rhel7.0_x86_64.qcow2
    qemu-img create -f qcow2 -b rhel7.0_x86_64.qcow2  sn1.qcow2
2.  boot guest with sn1.qcow2
    /usr/libexec/qemu-kvm \
-drive file=./sn1.qcow2,if=none,cache=none,format=qcow2,media=disk,id=d1\
-device virtio-blk-pci,drive=d1,id=hd1 \
-smp 2 -m 1024 -vnc :1 \
-enable-kvm \
-qmp tcp:0:1234,server,nowait \
-monitor stdio
3.  create big file in guest
   dd if=/dev/urandom of=/root/test2.img bs=4M count=100; sync; 

recording md5sum of the file 

cd /root/ && md5sum test2.img > test2.md5

4. convert image to raw
    python  qemu-to-raw/qemu-to-raw.py ./sn1.qcow2 target.raw
5. map image target.raw to logcial volume
    losetup `losetup -f` target.raw
    losetup -j target.raw (get loop device is /dev/loop0)
    kpartx -av /dev/loop0
    pvscan; vgscan; lvscan
6. active  the logical volume
    lvchange -ay /dev/rhel/root (lv name)
7. check md5sum not changed
    mount /dev/rhel/root /mnt
    cd /mnt/root && md5sum -c test2.md5 
    md5sum check OK


scenarios 3：
  base on scenarios2 steps 1 - 3,

add step,  do block_resize in monitor, before covert to raw

  block_reszie  d1  40G

then do qemu-img info sn1.qcow2 in another terminal, show virtual disk size changed to 40G 

then, do step4 - step 7 in scenarios 2：

md5sum check OK too

Base on above test results,  this bug has fixed;

Thanks,
Xu
Comment 34 Xu Tian 2013-10-11 03:10:08 UTC 
(In reply to xu from comment #33)
> Test below scenarios both qcow2 and raw image file
> scenarios 1：
> 
> test pass when image format both qcow2 and raw:
> 
> 1. boot guest with raw/qcow2 image, then create 400M file in "/root" folder;
>    
> /usr/libexec/qemu-kvm \
> -drive
> file=./rhel7.0_x86_64.qcow2,if=none,cache=none,format=qcow2,media=disk,id=d1\
> -device virtio-blk-pci,drive=d1,id=hd1 \
> -smp 2 -m 1024 -vnc :1 \
> -enable-kvm \
> -qmp tcp:0:1234,server,nowait \
> -monitor stdio
> 
> login guest, create big file
> dd if=/dev/urandom of=/root/test.img bs=4M count=100; sync; 
> 
> recording md5sum of the file 
> 
> cd /root/ && md5sum test.img > test.md5
> 
> 2. convert image to raw
>     python  qemu-to-raw/qemu-to-raw.py ./rhel7.0_x86_64.qcow2 target.raw
> 3. map image target.raw to logcial volume
>     losetup `losetup -f` target.raw
>     losetup -j target.raw (get loop device is /dev/loop0)
>     kpartx -av /dev/loop0
>     pvscan; vgscan; lvscan
> 4. active  the logical volume
>     lvchange -ay /dev/rhel/root (lv name)
> 5. check md5sum not changed
>     mount /dev/rhel/root /mnt
>     cd /mnt/root && md5sum -c test.md5 
>     md5sum check OK
> 
> scenarios 2：
> test pass when image has qcow2/raw backingfile;
> 
> 1. create snapshot for rhel7.0_x86_64.qcow2
>     qemu-img create -f qcow2 -b rhel7.0_x86_64.qcow2  sn1.qcow2
> 2.  boot guest with sn1.qcow2
>     /usr/libexec/qemu-kvm \
> -drive file=./sn1.qcow2,if=none,cache=none,format=qcow2,media=disk,id=d1\
> -device virtio-blk-pci,drive=d1,id=hd1 \
> -smp 2 -m 1024 -vnc :1 \
> -enable-kvm \
> -qmp tcp:0:1234,server,nowait \
> -monitor stdio
> 3.  create big file in guest
>    dd if=/dev/urandom of=/root/test2.img bs=4M count=100; sync; 
> 
> recording md5sum of the file 
> 
> cd /root/ && md5sum test2.img > test2.md5
> 
> 4. convert image to raw
>     python  qemu-to-raw/qemu-to-raw.py ./sn1.qcow2 target.raw
> 5. map image target.raw to logcial volume
>     losetup `losetup -f` target.raw
>     losetup -j target.raw (get loop device is /dev/loop0)
>     kpartx -av /dev/loop0
>     pvscan; vgscan; lvscan
> 6. active  the logical volume
>     lvchange -ay /dev/rhel/root (lv name)
> 7. check md5sum not changed
>     mount /dev/rhel/root /mnt
>     cd /mnt/root && md5sum -c test2.md5 
>     md5sum check OK
> 
> 
> scenarios 3：
>   base on scenarios2 steps 1 - 3,
> 
> add step,  do block_resize in monitor, before covert to raw
> 
>   block_reszie  d1  40G
> 
> then do qemu-img info sn1.qcow2 in another terminal, show virtual disk size
> changed to 40G 
> 
> then, do step4 - step 7 in scenarios 2：
> 
> md5sum check OK too
> 
> Base on above test results,  this bug has fixed;
> 
> Thanks,
> Xu

verification pass on packages qemu-img-0.12.1.2-2.410.el6.x86_64
Comment 35 Qunfang Zhang 2013-10-11 05:19:28 UTC 
Hi, Paolo

Could you help have a look at comment 33? We tested with qemu-img-0.12.1.2-2.410.el6.x86_64. Is this enough to call the bug verified pass? Any other test need to implement please let us know.

Thanks,
Qunfang
Comment 36 Paolo Bonzini 2013-10-11 11:33:22 UTC 
Yes, it should be enough.
Comment 39 errata-xmlrpc 2013-11-21 06:38:10 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2013-1553.html
Note You need to log in before you can comment on or make changes to this bug.