Bug 913410

Created attachment 700393 [details]
debug log for issue b

debug log for issue b attached

For issue a) I've proposed patch upstream:

https://www.redhat.com/archives/libvir-list/2013-February/msg01481.html

For issue b) I think it's actually qemu bug:

2013-02-21 08:13:24.962+0000: 9552: error : qemuMonitorIORead:513 : Unable to read from monitor: Connection reset by peer
2013-02-21 08:13:24.962+0000: 9552: debug : qemuMonitorIO:646 : Error on monitor Unable to read from monitor: Connection reset by peer
2013-02-21 08:13:24.962+0000: 9552: debug : virEventPollUpdateHandle:146 : EVENT_POLL_UPDATE_HANDLE: watch=40 events=12
2013-02-21 08:13:24.962+0000: 9552: debug : virEventPollInterruptLocked:697 : Skip interrupt, 1 -747411360
2013-02-21 08:13:24.962+0000: 9552: debug : virObjectUnref:135 : OBJECT_UNREF: obj=0x7f4ab801a640
2013-02-21 08:13:24.962+0000: 9552: debug : qemuMonitorIO:680 : Triggering error callback
2013-02-21 08:13:24.962+0000: 9552: debug : qemuProcessHandleMonitorError:342 : Received error on 0x7f4ab810bae0 'rhel6.3'

Qemu simply dies right after 'migrate' command is isued to it.

For issue c) I don't think this acutally is a problem - if your data has been copied I don't see any problem in using incremental mode.

For issue c , There are the following problems about it.

1) The guest image doesn't has the shared base image (backing file)

# qemu-img info /var/lib/libvirt/images/rhel63-new.img
image: /var/lib/libvirt/images/rhel63-new.img
file format: raw
virtual size: 8.0G (8589934592 bytes)
disk size: 8.0G
  
2) The format of guest image is raw. If a image has backing file , its format should be qcow2.

for example 
# qemu-img create -f raw -o backing_file=/var/base.img /var/images/increment.img
Unknown option 'backing_file'
Invalid options for file format 'raw'.

qemu-img create -f qcow2 -o backing_file=/var/base.img /var/increment.img
Formatting '/var/increment.img', fmt=qcow2 size=8589934592 backing_file='/var/base.img' encryption=off cluster_size=65536


There isn't any precondition for migration with copy-storage-inc.

--copy-storage-inc  migration with non-shared storage with incremental copy (same base image shared between source and destination)


3)Although libvirt display migration is successfully finished, it is actually failed to migrate.

On source host 

qemu-img info /var/lib/libvirt/images/rhel63-new.img
image: /var/lib/libvirt/images/rhel63-new.img
file format: raw
virtual size: 8.0G (8589934592 bytes)
disk size: 8.0G


On target host 
After migration is finished . check the guest img .
 
# qemu-img info /var/lib/libvirt/images/rhel63-new.img
image: /var/lib/libvirt/images/rhel63-new.img
file format: qcow2
virtual size: 8.0G (8589934592 bytes)
disk size: 140K
cluster_size: 65536

Login guest. the attached error occurs.

Created attachment 703283 [details]
error in guest

For issue b , File the following bug to qemu-kvm
Bug 916067 - when cancel the migration with ctrl+c during block migration(full disk copy or incremental disk copy), then migration again will cause domain destroyed

(In reply to comment #3)
> For issue c , There are the following problems about it.
> 
> 1) The guest image doesn't has the shared base image (backing file)
> 
> # qemu-img info /var/lib/libvirt/images/rhel63-new.img
> image: /var/lib/libvirt/images/rhel63-new.img
> file format: raw
> virtual size: 8.0G (8589934592 bytes)
> disk size: 8.0G
>   
> 2) The format of guest image is raw. If a image has backing file , its
> format should be qcow2.
> 
> for example 
> # qemu-img create -f raw -o backing_file=/var/base.img
> /var/images/increment.img
> Unknown option 'backing_file'
> Invalid options for file format 'raw'.
> 
> qemu-img create -f qcow2 -o backing_file=/var/base.img /var/increment.img
> Formatting '/var/increment.img', fmt=qcow2 size=8589934592
> backing_file='/var/base.img' encryption=off cluster_size=65536
> 
> 
> There isn't any precondition for migration with copy-storage-inc.
> 
> --copy-storage-inc  migration with non-shared storage with incremental copy
> (same base image shared between source and destination)
> 
> 
> 3)Although libvirt display migration is successfully finished, it is
> actually failed to migrate.
> 
> On source host 
> 
> qemu-img info /var/lib/libvirt/images/rhel63-new.img
> image: /var/lib/libvirt/images/rhel63-new.img
> file format: raw
> virtual size: 8.0G (8589934592 bytes)
> disk size: 8.0G
> 
> 
> On target host 
> After migration is finished . check the guest img .
>  
> # qemu-img info /var/lib/libvirt/images/rhel63-new.img
> image: /var/lib/libvirt/images/rhel63-new.img
> file format: qcow2
> virtual size: 8.0G (8589934592 bytes)
> disk size: 140K
> cluster_size: 65536
> 
> Login guest. the attached error occurs.

Can yo attach libvirt debug logs from both source and destination? You know one of the assumption is you have pre-created storage manually, right?

Yes , If don't pre-created storage manulally , the migration can't start.

For issue c . Add more detailed steps. And attached debug log.

On source host

# virsh start rhel63-new
Domain rhel63-new started

# virsh dumpxml rhel63-new|grep "source file"
      <source file='/var/lib/libvirt/images/rhel63-new.img'/>

# qemu-img info /var/lib/libvirt/images/rhel63-new.img
image: /var/lib/libvirt/images/rhel63-new.img
file format: raw
virtual size: 8.0G (8589934592 bytes)
disk size: 8.0G

# virsh migrate --verbose --live --persistent rhel63-new qemu+ssh://10.66.6.76/system 
root.6.76's password: 
Migration: [100 %]
===============================================================================
On target host
Before migration start 

# qemu-img create -f qcow2 rhel63-new.img 8G
Formatting 'rhel63-new.img', fmt=qcow2 size=8589934592 encryption=off cluster_size=65536 


After migration
# virsh list 
 Id    Name                           State
----------------------------------------------------
 18    rhel63-new                     running

Created attachment 704090 [details]
debug log for source and target host

Sorry, about migration step should be the following . I made a type when do cope -paste.

# virsh migrate --verbose --live --persistent rhel63-new qemu+ssh://10.66.6.76/system --copy-storage-inc
root.6.76's password: 
Migration: [100 %]

From the log I'd say it's not a libvirt issue. I mean, qemu is supposed to fall back to full copy even if you start migration with incremental one. In fact, that's the only reasonable thing to do when we are not allowed to mark only a subset of disks which we want to migrate. So, in case where you have both raw and qcow2 images, qemu should copy qcow2 incrementally and do a full copy on raw. Or in your case - just do a full copy. If it is not working - I'd say it's a qemu bug. NB in upstream we have reworked this to support NBD migration which is turned on every time we detect qemu supports it.

Moving to POST:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2013-July/msg00061.html

We decided not to rebase libvirt in RHEL 6.5 to avoid stability issues
we faced in 6.4. This bug has already been trivially fixed upstream but
it is not considered important enough to be backported to RHEL 6.5.

Thus I'm pushing this bug to RHEL 6.6 (and setting Upstream keyword to
indicate we have patches upstream) for now. If you don't agree with
this resolution, please, give us reasons which you think are strong
enough for us to reevaluate the decision not to backport patches for
this bug.

Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.