Bug 582475

I expect a PRD with the full requirements before starting to work on it

The use cases here are:
1. Want to move the VM from low performing LUNs to high performing LUNs
2. Want to move from qcow to raw to improve performance
3. Want to free a LUN
4. Want to move a VM running on local storage to shared storage
...

Note that the requirement is to be able to migrate only the disk images without migrating the VM (VM will go on running on same host).

Also note that requirement is to move only a subset of the disks (passed in as params) and not necessarily all disks.

Ayal,

Can we close this as a dupe of https://bugzilla.redhat.com/show_bug.cgi?id=545198 ?  (or the other way around?)

(In reply to comment #3)
> Ayal,
> 
> Can we close this as a dupe of
> https://bugzilla.redhat.com/show_bug.cgi?id=545198 ?  (or the other way
> around?)

Keep this (582475) open and close 545198 as it is less specific as to the requirements.

*** Bug 545198 has been marked as a duplicate of this bug. ***

This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated
in the current release, Red Hat is unfortunately unable to
address this request at this time. Red Hat invites you to
ask your support representative to propose this request, if
appropriate and relevant, in the next release of Red Hat
Enterprise Linux. If you would like it considered as an
exception in the current release, please ask your support
representative.

This request was erroneously denied for the current release of Red Hat
Enterprise Linux.  The error has been fixed and this request has been
re-proposed for the current release.

*** Bug 637706 has been marked as a duplicate of this bug. ***

*** Bug 666336 has been marked as a duplicate of this bug. ***

Testing instructions for HMP:

1) install guest;

2) qemu-img create -f qcow2 -o backing_file=test.img src.qcow2 

3) start QEMU on src.qcow2

4) perhaps generate some load to grow src.qcow2 (a hibernation is enough)

5) drive_mirror ide0-hd0 dest.qcow2 qcow2

6) generate some more I/O load in the VM

7) block_stream ide0-hd0 test.img
   (second argument as in the backing_file above)

8) when info block-jobs signal completion, exit QEMU.  Instead of poweroff, another hibernation is a good test that everything worked fine.

9) test that src.qcow2 and dest.qcow2 are almost or exactly the same size;

10) delete src.qcow2, restart QEMU, test that the guest resumes fine from hibernation.

Of course a more basic step can be done by omitting steps 4 and 6.

Paolo's instruction includes mirroring, which is a test blocker:
https://bugzilla.redhat.com/show_bug.cgi?id=802284#c10

QE are running a test against block_stream only now.

In order for libvirt to tell the difference between RHEL 6.2 (where block_job_cancel is synchronous) and RHEL 6.3 (where job cancellation is asynchronous), we should backport this additional upstream patch before the RHEL 6.3 release:
https://lists.gnu.org/archive/html/qemu-devel/2012-04/msg01225.html

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
No Documentation Needed

Hi All,

For verifying this feature on RHEL6.3 Snap1 release(k.v- 2.6.32-268.el6), the "drive_mirror" and "block_stream" commands are missing in the qemu monitor.

Thanks,
Santwana

IBM,

note that the commands are block-stream and drive-mirror, the latter being only available via QMP.  Is your package named qemu-kvm or qemu-kvm-rhev?

(In reply to comment #11)
> IBM,
>
> note that the commands are block-stream and drive-mirror, the latter being
> only available via QMP.  Is your package named qemu-kvm or qemu-kvm-rhev?

Hello Redhat,

I tried the commands block-stream and drive-mirror using QMP, but it doesn't recognise the command, stating  "unknown command".
Package of qemu-kvm installed is:
qemu-kvm-0.12.1.2-2.286.el6.x86_64

Thanks,
Santwana

(In reply to comment #12)
> (In reply to comment #11)
> > IBM,
> >
> > note that the commands are block-stream and drive-mirror, the latter being
> > only available via QMP.  Is your package named qemu-kvm or qemu-kvm-rhev?
>
> Hello Redhat,
>
> I tried the commands block-stream and drive-mirror using QMP, but it doesn't
> recognise the command, stating  "unknown command".
> Package of qemu-kvm installed is:
> qemu-kvm-0.12.1.2-2.286.el6.x86_64
>
> Thanks,
> Santwana

Hello Redhat,

Please ignore my previous message, I posted the results from qemu monitor instead of QMP.
However, after connecting my guest using the QMP, the block-stream and drive-mirror commands were not found in the query-commands.
Below is the output for the same:
{"QMP": {"version": {"qemu": {"micro": 1, "minor": 12, "major": 0}, "package": "(qemu-kvm-0.12.1.2)"}, "capabilities": []}}
{ "execute": "query-commands" }
{"return": [{"name": "quit"}, {"name": "eject"}, {"name": "__com.redhat_drive_del"}, {"name": "change"}, {"name": "screendump"}, {"name": "__com.redhat_qxl_screendump"}, {"name": "stop"}, {"name": "cont"}, {"name": "system_wakeup"}, {"name": "system_reset"}, {"name": "system_powerdown"}, {"name": "device_add"}, {"name": "device_del"}, {"name": "cpu"}, {"name": "memsave"}, {"name": "pmemsave"}, {"name": "inject-nmi"}, {"name": "migrate"}, {"name": "migrate_cancel"}, {"name": "migrate_set_speed"}, {"name": "client_migrate_info"}, {"name": "migrate_set_downtime"}, {"name": "block_resize"}, {"name": "netdev_add"}, {"name": "netdev_del"}, {"name": "balloon"}, {"name": "set_link"}, {"name": "getfd"}, {"name": "closefd"}, {"name": "block_passwd"}, {"name": "set_password"}, {"name": "expire_password"}, {"name": "__com.redhat_set_password"}, {"name": "__com.redhat_spice_migrate_info"}, {"name": "qmp_capabilities"}, {"name": "human-monitor-command"}, {"name": "__com.redhat_drive_add"}, {"name": "block-job-set-speed"}, {"name": "block-job-cancel"}, {"name": "query-version"}, {"name": "query-commands"}, {"name": "query-chardev"}, {"name": "query-block"}, {"name": "query-blockstats"}, {"name": "query-cpus"}, {"name": "query-kvm"}, {"name": "query-status"}, {"name": "query-mice"}, {"name": "query-vnc"}, {"name": "query-spice"}, {"name": "query-name"}, {"name": "query-uuid"}, {"name": "query-migrate"}, {"name": "query-balloon"}, {"name": "query-block-jobs"}]}

Thanks,
Santwana

(In reply to comment #30)
> (In reply to comment #12)
> > (In reply to comment #11)
> > > IBM,
> > >
> > > note that the commands are block-stream and drive-mirror, the latter being
> > > only available via QMP.  Is your package named qemu-kvm or qemu-kvm-rhev?
> >
> > Hello Redhat,
> >
> > I tried the commands block-stream and drive-mirror using QMP, but it doesn't
> > recognise the command, stating  "unknown command".
> > Package of qemu-kvm installed is:
> > qemu-kvm-0.12.1.2-2.286.el6.x86_64
> >
> > Thanks,
> > Santwana
> 
> Hello Redhat,
> 
> Please ignore my previous message, I posted the results from qemu monitor
> instead of QMP.
> However, after connecting my guest using the QMP, the block-stream and
> drive-mirror commands were not found in the query-commands.
> Below is the output for the same:
> {"QMP": {"version": {"qemu": {"micro": 1, "minor": 12, "major": 0}, "package":
> "(qemu-kvm-0.12.1.2)"}, "capabilities": []}}
> { "execute": "query-commands" }
> {"return": [{"name": "quit"}, {"name": "eject"}, {"name":
> "__com.redhat_drive_del"}, {"name": "change"}, {"name": "screendump"}, {"name":
> "__com.redhat_qxl_screendump"}, {"name": "stop"}, {"name": "cont"}, {"name":
> "system_wakeup"}, {"name": "system_reset"}, {"name": "system_powerdown"},
> {"name": "device_add"}, {"name": "device_del"}, {"name": "cpu"}, {"name":
> "memsave"}, {"name": "pmemsave"}, {"name": "inject-nmi"}, {"name": "migrate"},
> {"name": "migrate_cancel"}, {"name": "migrate_set_speed"}, {"name":
> "client_migrate_info"}, {"name": "migrate_set_downtime"}, {"name":
> "block_resize"}, {"name": "netdev_add"}, {"name": "netdev_del"}, {"name":
> "balloon"}, {"name": "set_link"}, {"name": "getfd"}, {"name": "closefd"},
> {"name": "block_passwd"}, {"name": "set_password"}, {"name":
> "expire_password"}, {"name": "__com.redhat_set_password"}, {"name":
> "__com.redhat_spice_migrate_info"}, {"name": "qmp_capabilities"}, {"name":
> "human-monitor-command"}, {"name": "__com.redhat_drive_add"}, {"name":
> "block-job-set-speed"}, {"name": "block-job-cancel"}, {"name":
> "query-version"}, {"name": "query-commands"}, {"name": "query-chardev"},
> {"name": "query-block"}, {"name": "query-blockstats"}, {"name": "query-cpus"},
> {"name": "query-kvm"}, {"name": "query-status"}, {"name": "query-mice"},
> {"name": "query-vnc"}, {"name": "query-spice"}, {"name": "query-name"},
> {"name": "query-uuid"}, {"name": "query-migrate"}, {"name": "query-balloon"},
> {"name": "query-block-jobs"}]}
> 
> Thanks,
> Santwana
Hi,IBM

Please use qemu-kvm-rhev instead of qemu-kvm.  block-stream,
drive-mirror  and live snapshot commands are disabled in qemu-kvm.

(In reply to comment #31)

> Please use qemu-kvm-rhev instead of qemu-kvm.  block-stream,
> drive-mirror  and live snapshot commands are disabled in qemu-kvm.

Also, in qemu-kvm-rhev, it is named __com.redhat_drive-mirror, not drive-mirror.

Test run for qemu-kvm-rhev-0.12.1.2-2.292.el6 - live block copy(stream) - rhel6.3-64

This test run has just been finished, there are two existing bugs but both are proposed to 6.4:
Bug 806325 - block stream speed limit is not accurate
Bug 808114 - streaming with no backing file should not do anything

And bug 783950 and bug 818876 which this bug depends on have been verified, so i am willing to verify this bug.

Based on the discussion with Stefan tested again with modified steps and find the below results:

Scenario 1:

1. install a image(rhel63-64.qcow2)
# qemu-img info rhel63-64.qcow2
image: rhel63-64.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 2.1G
cluster_size: 65536

2. create a src.qow2 with a backing file rhel63-64.qcow2
#  qemu-img create -f qcow2 -b rhel63-64.qcow2 -F qcow2 src.qcow2
# qemu-img info src.qcow2
image: src.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 11M
cluster_size: 65536
backing file: rhel63-64.qcow2

3. start the guest in qemu cmd line with src.qcow2
/usr/libexec/qemu-kvm -name 'streaming' -nodefaults -monitor stdio -drive file='/home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/src.qcow2',index=0,if=ide,cache=none -m 4028 -smp 8 -vnc :12 -vga std  -enable-kvm  -qmp tcp:localhost:4444,server
-------->Guest can be loggedin--OK

4. qemu)block-stream ide0-hd0
---->pulls all the data from rhel63-64.qcow2---------------------------------------------OK
---->Output from QMP monitor flaged "BLOCK_JOB_COMPLETED" after the streaming completed---OK

5. # qemu-img info src.qcow2
image: src.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 10G
cluster_size: 65536

---->qemu-img info of src.qcow2 does not have the backing file(rhel63-64.qcow2) anymore------OK
---->The size is 10G where in size of the backing file itself 2.1G--------------------------NOK--needs to raise separate bug.

Note:
If we create src.qcow2 with -o compact 1:1 option, the qemu-command line fails with the below error

# qemu-img create -f qcow2 -o compat=1.1 -b rhel63-64.qcow2 src.qcow2
Formatting 'src.qcow2', fmt=qcow2 size=10737418240 compat='1.1' backing_file='rhel63-64.qcow2' encryption=off cluster_size=65536

# /usr/libexec/qemu-kvm -name 'streaming' -nodefaults -monitor stdio -drive file='/home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/src.qcow2',index=0,if=ide,cache=none -m 4028 -smp 8 -vnc :12 -vga std  -enable-kvm  -qmp tcp:localhost:4444,server
QEMU waiting for connection on: tcp:::1:4444,server

qemu-kvm: -drive file=/home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/src.qcow2,index=0,if=ide,cache=none: 'ide0-hd0' uses a qcow2 feature which is not supported by this qemu version: QCOW version 3
qemu-kvm: -drive file=/home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/src.qcow2,index=0,if=ide,cache=none: could not open disk image /home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/src.qcow2: Operation not supported

Scenario 2:

1. install a image(rhel63-64.qcow2)
2. create a src.qow2 with a backing file rhel63-64.qcow2
3. start the guest in qemu cmd line with src.qcow2
4. create a dest.qcow2 with src.qcow2 as backing file
#qemu-img info dest1.qcow2
image: dest1.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 200K
cluster_size: 65536
backing file: src.qcow2

5. add the dest.qcow2 as drive ide0-hd1 and mirror ide0-hd0(src.qcow2) to dest.qcow2
qemu)__com.redhat_drive_add file=/home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/dest.qcow2,format=qcow2,id=ide0-hd1
qemu)__com.redhat_drive-mirror ide0-hd0 /home/sath/kvm_storage/Satheesh/Rhel6.3-IT1-ide/dest.qcow2 existing qcow2
qemu)info block-jobs
Type mirror, device ide0-hd0: Completed 144703488 of 10737418240 bytes, speed limit 0 bytes/s

6. qemu)block-stream ide0-hd1
---->Output from QMP monitor flaged "BLOCK_JOB_COMPLETED" after the streaming completed---OK
---->pulls all data from rhel63.64.qcow2

7. #qemu-img info src1.qcow2
image: src1.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 13M
cluster_size: 65536
backing file: rhel63-64.qcow2

# qemu-img info dest.qcow2
image: dest.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 10G
cluster_size: 65536

----qemu-img info of dest.qcow2 does not have the backing file(src.qcow2) anymore--OK
----qemu-img info of src.qcow2 have the backing file(rhel63-64.qcow2) and the size is also same as before, though it is mirrored---????
-----The size of dest.qcow2 is 10G where in size of the backing file itself 2.1G--------------NOK

Scenario 3:

1. install a image(rhel63-64.qcow2)
2. create a src.qow2 with a backing file rhel63-64.qcow2
3. start the guest in qemu cmd line with src.qcow2
4. create a dest.qcow2 with src.qcow2 as backing file
5  qemu)block-stream ide0-hd1
---->pulls all data from rhel63.64.qcow2
---->Output from QMP monitor flaged "BLOCK_JOB_COMPLETED" after the streaming completed---OK
---->Guest can be started with dest.qcow2------------------------------------OK

6. # qemu-img info dest.qcow2
image: dest.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 10G
cluster_size: 65536
# qemu-img info src.qcow2
image: src.qcow2
file format: qcow2
virtual size: 10G (10737418240 bytes)
disk size: 11M
cluster_size: 65536
backing file: rhel63-64.qcow2

---->qemu-img info of dest.qcow2 does not have the backing file(src.qcow2) anymore-----------OK
----->The size of dest.qcow2 is 10G where in size of the backing file itself 2.1G------------NOK

P:S:- from scenraio 2 and 3 results seems to be same though there is an extra step of mirroring the drive.---?

Regards,
-Satheesh

Scenario 2 is not supported; you are opening the same image twice (for the mirroring destination and a device in the guest).

It's not clear exactly what you would like to do in that case.  You should only need to use drive-mirror, without either device-add or block-stream.


Regarding the size of the mirroring destination, this was done by design in upstream QEMU and backported this way to RHEL.  We changed it before the 1.1 release in upstream QEMU, but it was too late for RHEL6.3.  For now, the QED data format will have the expected result and will not "explode" the size of dest.qcow2.  I opened bug 832336 for RHEL6.4 to fix this problem.

Thanks for testing.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-1017.html