Bug 1497120

This seems to have settled out OK upstream; see todays:

  [PATCH v4 0/7] migration: pause-before-switchover

upstream commits 93fbd0314ec060ffaf90..0331c8cabf6168aa263a

Fix included in qemu-kvm-rhev-2.10.0-4.el7

Hi, Dave,
I have found some information about your patch, that in this link: http://patchwork.ozlabs.org/cover/827760/
there is two work flow as said in this link,
The precopy flow is:
active->pre-switchover->device->completed

The postcopy flow is:
active->pre-switchover->postcopy-active->completed

But I didn't know the detail test steps for this bug, could you give me the detail test steps and expected results for these two scenarios? thanks

Hi, Dave and kwolf,
according to my understand and referring to http://patchwork.ozlabs.org/cover/827760/, the following is my test steps for this scenario, could you help to review whether this bug could be verified as following steps, thanks:
version:
Host:
3.10.0-776.el7.ppc64le
qemu-kvm-rhev-2.10.0-5.el7.ppc64le
SLOF-20170724-2.git89f519f.el7.noarch

Guest:
3.10.0-776.el7.ppc64

steps:
1.Boot a guest on src host with drive:
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/RHEL74_BE.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,bus=virtio_scsi_pci0.0,lun=0,bootindex=0 \
    -qmp tcp:0:8881,server,nowait \
2.Launch listening mode on dst host, with same qemu cli as src end but different disk:
    -drive id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,file=/home/mount_point/block.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,bus=virtio_scsi_pci0.0,lun=0,bootindex=0 \
3.On src host, do drive-mirror job via qmp:
{"execute":"qmp_capabilities"}
{ "execute": "drive-mirror", "arguments": { "device": "drive_image1", "target": "/home/block.qcow2", "format": "qcow2", "mode": "absolute-paths", "sync": "full", "on-source-error": "stop", "on-target-error": "stop"} }
after a while, block job becomes ready automatically:
{"timestamp": {"seconds": 1510641851, "microseconds": 178028}, "event": "BLOCK_JOB_READY", "data": {"device": "drive_image1", "len": 10142023680, "offset": 10142023680, "speed": 0, "type": "mirror"}}
4.In guest, execute "stress":
# stress --cpu 4 --io 8 --vm 2 --vm-bytes 128M
5.On src host, do live migration with enabling "pause-before-switchover":
(qemu) migrate_set_capability pause-before-switchover on
(qemu) info migrate_capabilities 
xbzrle: off
rdma-pin-all: off
auto-converge: off
zero-blocks: off
compress: off
events: off
postcopy-ram: off
x-colo: off
release-ram: off
return-path: off
pause-before-switchover: on
(qemu) migrate -d tcp:10.16.42.48:5801
(qemu) info migrate
Migration status: active
in order to make "migrate status" to "pre-switchover" quickly, I enlarge the value of "downtime" and "speed"
(qemu) migrate_set_downtime 10
(qemu) migrate_set_speed 1G
(qemu) info migrate
Migration status: pre-switchover
(qemu) info status 
VM status: paused (finish-migrate)
6.On src host, do block-job-complete
{"execute":"block-job-complete","arguments":{"device":"drive_image1"}}
{"timestamp": {"seconds": 1510642464, "microseconds": 566489}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len": 10163388416, "offset": 10163388416, "speed": 0, "type": "mirror"}}
7.On src host
(qemu) migrate_continue pre-switchover

Result:
Block job and migration completed and vm works well.
On src host:
(qemu) info migrate
globals: store-global-state=1, only_migratable=0, send-configuration=1, send-section-footer=1
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off postcopy-ram: off x-colo: off release-ram: off return-path: off pause-before-switchover: on 
Migration status: completed
(qemu) info status 
VM status: paused (postmigrate)

On dst host:
(qemu) info status 
VM status: running

1)Due to after "block-job-complete", data will only write to target disk while live migration is in process, I don't know whether this steps are right, I just wonder if we don't execute "block-job-complete" before "migrate_continue pre-switchover", because once live migration complete and quit qemu of src end, block job would complete automatically.

2)For cancelling migration scenarios, which phases could cancel live migration?
at least, during post-copy mode, migration can't be cancelled.

Thanks

(In reply to xianwang from comment #7)
> Hi, Dave and kwolf,
> according to my understand and referring to
> http://patchwork.ozlabs.org/cover/827760/, the following is my test steps
> for this scenario, could you help to review whether this bug could be
> verified as following steps, thanks:
> version:
> Host:
> 3.10.0-776.el7.ppc64le
> qemu-kvm-rhev-2.10.0-5.el7.ppc64le
> SLOF-20170724-2.git89f519f.el7.noarch
> 
> Guest:
> 3.10.0-776.el7.ppc64
> 
> steps:
> 1.Boot a guest on src host with drive:
>     -drive
> id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,
> file=/home/RHEL74_BE.qcow2 \
>     -device
> scsi-hd,id=image1,drive=drive_image1,bus=virtio_scsi_pci0.0,lun=0,
> bootindex=0 \
>     -qmp tcp:0:8881,server,nowait \
> 2.Launch listening mode on dst host, with same qemu cli as src end but
> different disk:
>     -drive
> id=drive_image1,if=none,snapshot=off,aio=threads,cache=none,format=qcow2,
> file=/home/mount_point/block.qcow2 \
>     -device
> scsi-hd,id=image1,drive=drive_image1,bus=virtio_scsi_pci0.0,lun=0,
> bootindex=0 \
> 3.On src host, do drive-mirror job via qmp:
> {"execute":"qmp_capabilities"}
> { "execute": "drive-mirror", "arguments": { "device": "drive_image1",
> "target": "/home/block.qcow2", "format": "qcow2", "mode": "absolute-paths",
> "sync": "full", "on-source-error": "stop", "on-target-error": "stop"} }
> after a while, block job becomes ready automatically:
> {"timestamp": {"seconds": 1510641851, "microseconds": 178028}, "event":
> "BLOCK_JOB_READY", "data": {"device": "drive_image1", "len": 10142023680,
> "offset": 10142023680, "speed": 0, "type": "mirror"}}

I think that's probably the wrong way to do a drive-mirror; I was expecting to see an nbd-server setup with a mirror with that.
See: https://kashyapc.fedorapeople.org/QEMU-Docs/_build/html/docs/live-block-operations.html#qmp-invocation-for-live-storage-migration-with-drive-mirror-nbd

(although you don't need any backing chain)

> 4.In guest, execute "stress":
> # stress --cpu 4 --io 8 --vm 2 --vm-bytes 128M

I don't think you need much of a stress test for this; just the io stress side should be enough.

> 5.On src host, do live migration with enabling "pause-before-switchover":
> (qemu) migrate_set_capability pause-before-switchover on
> (qemu) info migrate_capabilities 
> xbzrle: off
> rdma-pin-all: off
> auto-converge: off
> zero-blocks: off
> compress: off
> events: off
> postcopy-ram: off
> x-colo: off
> release-ram: off
> return-path: off
> pause-before-switchover: on
> (qemu) migrate -d tcp:10.16.42.48:5801
> (qemu) info migrate
> Migration status: active
> in order to make "migrate status" to "pre-switchover" quickly, I enlarge the
> value of "downtime" and "speed"
> (qemu) migrate_set_downtime 10
> (qemu) migrate_set_speed 1G
> (qemu) info migrate
> Migration status: pre-switchover
> (qemu) info status 
> VM status: paused (finish-migrate)

That downtime is a bit long; if you're not running the full stress then you should be able to set that downtime to a more sensible .5

> 6.On src host, do block-job-complete
> {"execute":"block-job-complete","arguments":{"device":"drive_image1"}}
> {"timestamp": {"seconds": 1510642464, "microseconds": 566489}, "event":
> "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len":
> 10163388416, "offset": 10163388416, "speed": 0, "type": "mirror"}}

Hmm is that right? Or do we need a block-job-cancel rather than complete for a migrate?

> 7.On src host
> (qemu) migrate_continue pre-switchover
> 
> Result:
> Block job and migration completed and vm works well.
> On src host:
> (qemu) info migrate
> globals: store-global-state=1, only_migratable=0, send-configuration=1,
> send-section-footer=1
> capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks:
> off compress: off events: off postcopy-ram: off x-colo: off release-ram: off
> return-path: off pause-before-switchover: on 
> Migration status: completed
> (qemu) info status 
> VM status: paused (postmigrate)
> 
> On dst host:
> (qemu) info status 
> VM status: running
> 
> 1)Due to after "block-job-complete", data will only write to target disk
> while live migration is in process, I don't know whether this steps are
> right, I just wonder if we don't execute "block-job-complete" before
> "migrate_continue pre-switchover", because once live migration complete and
> quit qemu of src end, block job would complete automatically.
>

We need to check this with kwolf.

> 2)For cancelling migration scenarios, which phases could cancel live
> migration?
> at least, during post-copy mode, migration can't be cancelled.

Yes, for the test above we should be checking cancel in precopy before and after it's entered the new pause state.

Dave
> 
> Thanks

(In reply to Dr. David Alan Gilbert from comment #8)
> (In reply to xianwang from comment #7)

[...]

> > 3.On src host, do drive-mirror job via qmp:
> > {"execute":"qmp_capabilities"}
> > { "execute": "drive-mirror", "arguments": { "device": "drive_image1",
> > "target": "/home/block.qcow2", "format": "qcow2", "mode": "absolute-paths",
> > "sync": "full", "on-source-error": "stop", "on-target-error": "stop"} }
> > after a while, block job becomes ready automatically:
> > {"timestamp": {"seconds": 1510641851, "microseconds": 178028}, "event":
> > "BLOCK_JOB_READY", "data": {"device": "drive_image1", "len": 10142023680,
> > "offset": 10142023680, "speed": 0, "type": "mirror"}}
> 
> I think that's probably the wrong way to do a drive-mirror; I was expecting
> to see an nbd-server setup with a mirror with that.

Yes, you're right.  NBD server setup is required indeed.  The below link
does talk about it.

> See:
> https://kashyapc.fedorapeople.org/QEMU-Docs/_build/html/docs/live-block-
> operations.html#qmp-invocation-for-live-storage-migration-with-drive-mirror-
> nbd
> 
> (although you don't need any backing chain)

[...]

> > 6.On src host, do block-job-complete
> > {"execute":"block-job-complete","arguments":{"device":"drive_image1"}}
> > {"timestamp": {"seconds": 1510642464, "microseconds": 566489}, "event":
> > "BLOCK_JOB_COMPLETED", "data": {"device": "drive_image1", "len":
> > 10163388416, "offset": 10163388416, "speed": 0, "type": "mirror"}}
> 
> Hmm is that right? Or do we need a block-job-cancel rather than complete for
> a migrate?

Yes, `block-job-cancel` is what libvirt uses in its `drive-mirror` + NBD
workflow.

While a ``drive-mirror` job is active, and it has issued a
BLOCK_JOB_READY event, signalling that synchronization of source and
destination has completed.  And you can take one of the two following
action:

  (1) By issuing `block-job-complete`, the synchronization will stop,
      and live QEMU will be pivoted to the copy.  (Event:
      BLOCK_JOB_COMPLETED) 
  
  (2) By issuing `block-job-cancel`, the synchronization will stop, but
      live QEMU will NOT be pivoted to the copy.  But what you end up
      with is a point-in-time (which is at the time of _triggering_ the
      'cancel' command) copy.  (Event: BLOCK_JOB_CANCELLED)

In the NBD workflow, after `block-job-cancel`, you also stop the NBD
server on the destination, by issuing QMP `nbd-server-stop`.

Created attachment 1352558 [details]
Raw QMP test procedure with NBD-based migration with 'pause-before-switchover'

This was tested with upstream QEMU:

{  
  "QMP":{  
    "version":{  
      "qemu":{  
        "micro":90,
        "minor":10,
        "major":2
      },
      "package":" (v2.11.0-rc0-47-g55ed8d6)"
    },
    "capabilities":[  

    ]
  }
}

Hi, Dave and kwolf,
FYI and that link, I have re-test this scenario as following:
version:
3.10.0-768.el7.ppc64le
qemu-kvm-rhev-2.10.0-6.el7.ppc64le
SLOF-20170724-2.git89f519f.el7.noarch

localmigration:

src end:
1.boot guest:
    -drive id=drive_image1,if=none,aio=threads,cache=none,format=qcow2,file=/home/test.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,addr=07,bootindex=0 \

check:
(qemu) info block
drive_image1 (#block173): /home/test.qcow2 (qcow2)
    Attached to:      /machine/peripheral/image1/virtio-backend
    Cache mode:       writeback, direct


2.dst end:
firstly, create an qcow2 as exported disk:
#qemu-img create -f qcow2 nbd.qcow2 20G
then boot guest:
    -blockdev node-name=node-TargetDisk,driver=qcow2,file.driver=file,file.node-name=file,file.filename=/home/nbd.qcow2 \
    -device virtio-blk,id=virtio0,drive=node-TargetDisk \

check:
(qemu) info block
node-TargetDisk: /home/nbd.qcow2 (qcow2)
    Attached to:      /machine/peripheral/virtio0/virtio-backend
    Cache mode:       writeback

3.set dst as nbd server:
# telnet 127.0.0.1 8882
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute":"nbd-server-start","arguments":{"addr":{"type":"inet","data":{"host":"127.0.0.1","port":"12345"}}}}
{"return": {}}
{"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk"}}
{"return": {}}

4.in src end, do drive-mirror
# telnet 127.0.0.1 8881
{"execute":"qmp_capabilities"}
{"return": {}}
{"execute": "drive-mirror","arguments":{"device":"drive_image1","mode":"existing","job-id":"job0","target":"nbd:127.0.0.1:12345:exportname=node-TargetDisk","sync":"top","format":"qcow2"}}
{"error": {"class": "GenericError", "desc": "Image is not in qcow2 format"}}

{"execute": "drive-mirror","arguments":{"device":"drive_image1","mode":"existing","job-id":"job0","target":"nbd:127.0.0.1:12345:exportname=node-TargetDisk","sync":"top"}}
{"return": {}}
{"timestamp": {"seconds": 1510922579, "microseconds": 879307}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 879787}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 882765}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 884881}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 887108}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 889323}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 890445}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 891936}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 893544}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 893579}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 894186}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 895497}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 895883}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 896305}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 897920}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 899138}, "event": "BLOCK_JOB_ERROR", "data": {"device": "job0", "operation": "write", "action": "report"}}
{"timestamp": {"seconds": 1510922579, "microseconds": 899443}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "job0", "len": 21474836480, "offset": 0, "speed": 0, "type": "mirror", "error": "Operation not permitted"}}

on src end:
(qemu) WARNING: Image format was not specified for 'nbd://127.0.0.1:12345/node-TargetDisk' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

on dst end:
(qemu) qemu-kvm: Server is read-only, return error

please help to check it, thanks

I think the problem here is:

 {"execute": "drive-mirror","arguments": {"device":"drive_image1","mode":"existing","job id":"job0","target":"nbd:127.0.0.1:12345:exportname=node-TargetDisk","sync":"top","format":"qcow2"}}

and the error:

{"error": {"class": "GenericError", "desc": "Image is not in qcow2 format"}}

the drive-mirror should be told the image format is raw even if the backing is really qcow2, because it's sent raw over the NBD; see kashyap's docs has:

{ "execute": "drive-mirror", "arguments": { "device": "virtio0", "target": "nbd:localhost:3333:exportname=virtio0", "sync": "full","format": "raw", "mode": "existing" } }

Hi, Dave and kwolf,
I)First:
For nbd server is not writable, the resolution maybe is that we need to add "write" permission for nbd server, but there is one little problem about qmp:
-HMP
(qemu) nbd_server_add -w node-TargetDisk  *******works well
-QMP
{"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk","writable":"true"}}
{"error": {"class": "GenericError", "desc": "Invalid parameter type for 'writable', expected: boolean"}}
{"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk","boolean":"true"}}
{"error": {"class": "GenericError", "desc": "Parameter 'boolean' is unexpected"}}
{"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk","writable":"boolean"}}
{"error": {"class": "GenericError", "desc": "Invalid parameter type for 'writable', expected: boolean"}}

so, I don't know what is the right qmp command

II)Second:
I have re-test four scenarios, all are local migrations, but only one scenario is pass, other three are all failed.

pass:
a)src end:
    -drive id=drive_image1,if=none,aio=threads,cache=none,format=qcow2,file=/home/test.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,addr=07,bootindex=0 \

check block and enable pause-before-switchover:
(qemu) info block
drive_image1 (#block143): /home/test.qcow2 (qcow2)
    Attached to:      /machine/peripheral/image1/virtio-backend
    Cache mode:       writeback, direct
(qemu) migrate_set_capability pause-before-switchover on
(qemu) info migrate_capabilities 
xbzrle: off
rdma-pin-all: off
auto-converge: off
zero-blocks: off
compress: off
events: off
postcopy-ram: off
x-colo: off
release-ram: off
return-path: off
pause-before-switchover: on


b)dst end:
    -blockdev node-name=node-TargetDisk,driver=qcow2,file.driver=file,file.node-name=file,file.filename=/home/nbd.qcow2 \
    -device virtio-blk-pci,id=virtio0,drive=node-TargetDisk,bus=pci.0,addr=07,bootindex=0 \

check:
(qemu) info block
node-TargetDisk: /home/nbd.qcow2 (qcow2)
    Attached to:      /machine/peripheral/virtio0/virtio-backend
    Cache mode:       writeback

setup nbd server on dst host(make dst qemu as nbd server)
(qemu) nbd_server_start 127.0.0.1:12345
(qemu) nbd_server_add -w node-TargetDisk

c)on src end, begin to do migration and then begin to do block-mirror on dst end
src end:
(qemu) migrate -d tcp:127.0.0.1:5801
(qemu) info migrate
globals: store-global-state=1, only_migratable=0, send-configuration=1, send-section-footer=1
Migration status: active

then, begin to do block-mirror on src end during live migration, after block job becomes "BLOCK_JOB_READY", cancel block job and stop nbd server
src end:
{"execute": "drive-mirror","arguments":{"device":"drive_image1","mode":"existing","job-id":"job0","target":"nbd:127.0.0.1:12345:exportname=node-TargetDisk","sync":"top"}}
{"return": {}}
{"timestamp": {"seconds": 1510937156, "microseconds": 30791}, "event": "BLOCK_JOB_READY", "data": {"device": "job0", "len": 21475229696, "offset": 21475229696, "speed": 0, "type": "mirror"}}
{"execute": "block-job-cancel","arguments":{"device":"job0"}}
{"return": {}}

dst end:
(qemu) nbd_server_stop

d)check the status of migration on src end, and switch to "pre-switchover"
on src:
(qemu) info migrate
globals: store-global-state=1, only_migratable=0, send-configuration=1, send-section-footer=1
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off compress: off events: off postcopy-ram: off x-colo: off release-ram: off return-path: off pause-before-switchover: on 
Migration status: pre-switchover
(qemu) info status 
VM status: paused (finish-migrate)
(qemu) migrate_continue pre-switchover
(qemu) info migrate
Migration status: completed
(qemu) info status 
VM status: paused (postmigrate)

on dst:
(qemu) info status 
VM status: running
vm works well and reboot works well.

other three scenarios are similar with above, the qemu cli are all same with above one, the difference between them is as following:

1)do drive-mirror job firstly, when block job becomes "ready", then cancel block job, then begin to do live migration, result is:migration complete, vm works well on dst end without error message in guest by "demesg", but reboot guest, the file system is broken as attachment guest;

2)do drive-mirror job, cancel job before it becomes "ready", then whether do live migration before or after cancelling block job, the result is same: migration complete, but guest can't be login via vnc and there is error massage in guest via "dmesg" as following:

[  318.990587] XFS (dm-0): metadata I/O error: block 0x1aa9860 ("xfs_trans_read_buf_map") error 117 numblks 32
[  318.990588] XFS (dm-0): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117.
[  318.992416] XFS (dm-0): Metadata CRC error detected at xfs_dir3_block_read_verify+0x78/0x1c0 [xfs], xfs_dir3_block block 0x1aad650
[  318.992422] XFS (dm-0): metadata I/O error: block 0x1aad650 ("xfs_trans_read_buf_map") error 74 numblks 8

guest reboot failed, vnc screenshot is same as attachment guest.

1)please check whether the test steps of the pass one is right, for it is pass, does it indicate this bug is fixed? 
2)I am not sure the failed three scenarios are illegal test scenarios or they are bugs.
Thanks very much

xianwang:
  Please copy kashyap's example - json is very fussy;

{ "execute": "nbd-server-add", "arguments": { "device": "virtio0","writable": true } }

Note there are no " around the true

(In reply to Dr. David Alan Gilbert from comment #14)
> xianwang:
>   Please copy kashyap's example - json is very fussy;
> 
> { "execute": "nbd-server-add", "arguments": { "device":
> "virtio0","writable": true } }
> 
> Note there are no " around the true

Dave,
I have tried as:
{"execute":"nbd-server-start","arguments":{"addr":{"type":"inet","data":{"host":"127.0.0.1","port":"12345"}}}}
{"return": {}}
{"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk","writable":true}}
{"return": {}}

it works, thanks.

But I have another question, for "pre-switchover", what is the right order of doing "live migration" and doing "block mirror" ? 
I think we should first do block-mirror, after block job becomes to "BLOCK_JOB_COMPLETED" whether by "`block-job-complete` or `block-job-cancel`, then begin do live migration, but this failed, on contrary, first do live migration, during live migration do block-mirror, it succeed, I am confused why

(In reply to xianwang from comment #15)

[...]

> I have tried as:
> {"execute":"nbd-server-start","arguments":{"addr":{"type":"inet","data":
> {"host":"127.0.0.1","port":"12345"}}}}
> {"return": {}}
> {"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk",
> "writable":true}}
> {"return": {}}
> 
> it works, thanks.
> 
> But I have another question, for "pre-switchover", what is the right order
> of doing "live migration" and doing "block mirror" ? 
> I think we should first do block-mirror, after block job becomes to
> "BLOCK_JOB_COMPLETED" whether by "`block-job-complete` or
> `block-job-cancel`, then begin do live migration, but this failed, on
> contrary, first do live migration, during live migration do block-mirror, it
> succeed, I am confused why

The right order using 'pause-before-switchover' is documented in the text file I attached in comment#10.

Copying here's the full sequence:

[I didn't copy / paste the first step intentionally; it just launches source & destination QEMU -- refer the file in comment#10].


(2) [SOURCE] Set the "pause-before-switchover" capability.
-----------------------------------------------------------------------
{"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"pause-before-switchover","state":true}]}}
{"return": {}}
-----------------------------------------------------------------------
{"execute":"query-migrate-capabilities"}
{"return": [{"state": false, "capability": "xbzrle"}, {"state": false, "capability": "rdma-pin-all"}, {"state": false, "capability": "auto-converge"}, {"state": false, "capability": "zero-blocks"}, {"state": false, "capability": "compress"}, {"state": false, "capability": "events"}, {"state": false, "capability": "postcopy-ram"}, {"state": false, "capability": "x-colo"}, {"state": false, "capability": "release-ram"}, {"state": false, "capability": "block"}, {"state": false, "capability": "return-path"}, {"state": true, "capability": "pause-before-switchover"}, {"state": false, "capability": "x-multifd"}]}
-----------------------------------------------------------------------


(3) [DEST] Start the NBD server
-----------------------------------------------------------------------
{ "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host": "localhost", "port": "3333" } } } }
{"return": {}}
{ "execute": "nbd-server-add", "arguments": { "device": "virtio0","writable": true } }
{"return": {}}
-----------------------------------------------------------------------


(4) [SOURCE] Execute the `drive-mirror` command
-----------------------------------------------------------------------
{ "execute": "drive-mirror", "arguments": { "device": "virtio0", "target": "nbd:localhost:3333:exportname=virtio0", "sync": "full","format": "raw", "mode": "existing" } }
{"return": {}}
{"timestamp": {"seconds": 1510677793, "microseconds": 645286}, "event": "BLOCK_JOB_READY", "data": {"device": "virtio0", "len": 41126400, "offset": 41126400, "speed": 0, "type": "mirror"}}
-----------------------------------------------------------------------


(5) [SOURCE] Issue the `migrate` command (and query the migration status):
-----------------------------------------------------------------------
{"execute":"migrate","arguments":{"blk":false,"inc":false,"uri":"tcp:localhost:6666"}}
{"return": {}}
{"timestamp": {"seconds": 1510744913, "microseconds": 630373}, "event": "STOP"}

{"execute":"query-migrate"}                                                                                                                                                              
{"return": {"expected-downtime": 300, "status": "pre-switchover", "setup-time": 2, "total-time": 110111, "ram": {"total": 537403392, "postcopy-requests": 0, "dirty-sync-count": 2, "page-size": 4096, "remaining": 4141056, "mbps": 268.56432, "transferred": 59759367, "duplicate": 115920, "dirty-pages-rate": 72829, "skipped": 0, "normal-bytes": 58601472, "normal": 14307}}}
-----------------------------------------------------------------------


(6) [SOURCE] Issue the `block-job-cancel` command
-----------------------------------------------------------------------
{"execute":"block-job-cancel","arguments":{"device":"virtio0"} }
{"return": {}}
{"timestamp": {"seconds": 1510678024, "microseconds": 526240}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "virtio0", "len": 41126400, "offset": 41126400, "speed": 0, "type": "mirror"}}
-----------------------------------------------------------------------


(7) [DEST] Stop the NBD server
-----------------------------------------------------------------------
{"execute":"nbd-server-stop"}
{"return": {}}
-----------------------------------------------------------------------


(8) [DEST] Resume the guest vCPUs
-----------------------------------------------------------------------
{"execute":"cont"}
{"return": {}}
-----------------------------------------------------------------------


(9) [SOURCE] Gracefully terminate the QEMU process (repeat the same on
    destination.)
-----------------------------------------------------------------------
{"execute":"quit"}
{"return": {}}
{"timestamp": {"seconds": 1510678728, "microseconds": 574318}, "event": "SHUTDOWN", "data": {"guest": false}}
-----------------------------------------------------------------------

(In reply to xianwang from comment #15)
> (In reply to Dr. David Alan Gilbert from comment #14)
> > xianwang:
> >   Please copy kashyap's example - json is very fussy;
> > 
> > { "execute": "nbd-server-add", "arguments": { "device":
> > "virtio0","writable": true } }
> > 
> > Note there are no " around the true
> 
> Dave,
> I have tried as:
> {"execute":"nbd-server-start","arguments":{"addr":{"type":"inet","data":
> {"host":"127.0.0.1","port":"12345"}}}}
> {"return": {}}
> {"execute":"nbd-server-add","arguments":{"device":"node-TargetDisk",
> "writable":true}}
> {"return": {}}
> 
> it works, thanks.
> 
> But I have another question, for "pre-switchover", what is the right order
> of doing "live migration" and doing "block mirror" ? 
> I think we should first do block-mirror, after block job becomes to
> "BLOCK_JOB_COMPLETED" whether by "`block-job-complete` or
> `block-job-cancel`, then begin do live migration, but this failed, on
> contrary, first do live migration, during live migration do block-mirror, it
> succeed, I am confused why

Since the guest is still running during live migration you've got to mirror the changes made to the disk by the guest during the live migration itself; so you can't finish the mirror before the migration.   You have to wait for the drive mirror to declare 'block job ready' before starting the migrate command, that way you know the block code has done the first pass of copying the disk across.

Thanks for Dave's and Kashyap's reply, FYI, I have test this scenario as following:

Version:
Host:
3.10.0-768.el7.ppc64le
qemu-kvm-rhev-2.10.0-6.el7.ppc64le
SLOF-20170724-2.git89f519f.el7.noarch

Guest:
3.10.0-768.el7.ppc64le

a)src end:
    -drive id=drive_image1,if=none,aio=threads,cache=none,format=qcow2,file=/home/test.qcow2 \
    -device virtio-blk-pci,id=image1,drive=drive_image1,bus=pci.0,addr=07,bootindex=0 \

check block and enable pause-before-switchover:
(qemu) info block
drive_image1 (#block143): /home/test.qcow2 (qcow2)
    Attached to:      /machine/peripheral/image1/virtio-backend
    Cache mode:       writeback, direct
(qemu) migrate_set_capability pause-before-switchover on
(qemu) info migrate_capabilities 
xbzrle: off
rdma-pin-all: off
auto-converge: off
zero-blocks: off
compress: off
events: off
postcopy-ram: off
x-colo: off
release-ram: off
return-path: off
pause-before-switchover: on
or
-----------------------------------------------------------------------
{"execute":"migrate-set-capabilities","arguments":{"capabilities":[{"capability":"pause-before-switchover","state":true}]}}
{"return": {}}
-----------------------------------------------------------------------
{"execute":"query-migrate-capabilities"}
{"return": [{"state": false, "capability": "xbzrle"}, {"state": false, "capability": "rdma-pin-all"}, {"state": false, "capability": "auto-converge"}, {"state": false, "capability": "zero-blocks"}, {"state": false, "capability": "compress"}, {"state": false, "capability": "events"}, {"state": false, "capability": "postcopy-ram"}, {"state": false, "capability": "x-colo"}, {"state": false, "capability": "release-ram"}, {"state": false, "capability": "block"}, {"state": false, "capability": "return-path"}, {"state": true, "capability": "pause-before-switchover"}, {"state": false, "capability": "x-multifd"}]}
-----------------------------------------------------------------------

b)dst end:
    -blockdev node-name=node-TargetDisk,driver=raw,file.driver=file,file.node-name=file,file.filename=/home/nbd.raw \
    -device virtio-blk-pci,id=virtio0,drive=node-TargetDisk,bus=pci.0,addr=07,bootindex=0 \

check:
(qemu) info block
node-TargetDisk: /home/nbd.raw (raw)
    Attached to:      /machine/peripheral/virtio0/virtio-backend
    Cache mode:       writeback

setup nbd server on dst host(make dst qemu as nbd server)
(qemu) nbd_server_start 10.16.69.77:12345
(qemu) nbd_server_add -w node-TargetDisk
or
-----------------------------------------------------------------------
{ "execute": "nbd-server-start", "arguments": { "addr": { "type": "inet","data": { "host": "10.16.69.77", "port": "12345" } } } }
{"return": {}}
{ "execute": "nbd-server-add", "arguments": { "device": "node-TargetDisk","writable": true } }
{"return": {}}
-----------------------------------------------------------------------

c)on src end, begin to do block-mirror and after block job becomes to "BLOCK_JOB_READY" then do live migration, when live migration becomes to "pre-switchover" and cancel block job and then "migrate_continue pre-switchover"

src end:
block-mirror:
-----------------------------------------------------------------------
{ "execute": "drive-mirror", "arguments": { "device": "drive_image1", "target": "nbd:10.16.69.77:12345:exportname=node-TargetDisk", "sync": "full","format": "raw", "mode": "existing" } }
{"return": {}}
{"timestamp": {"seconds": 1510677793, "microseconds": 645286}, "event": "BLOCK_JOB_READY", "data": {"device": "virtio0", "len": 41126400, "offset": 41126400, "speed": 0, "type": "mirror"}}
-----------------------------------------------------------------------

live migration:
(qemu) migrate -d tcp:10.16.69.77:5801
(qemu) info migrate
Migration status: pre-switchover
or
-----------------------------------------------------------------------
{"execute":"migrate","arguments":{"blk":false,"inc":false,"uri":"tcp:10.16.69.77:5801"}}
{"return": {}}
{"timestamp": {"seconds": 1510744913, "microseconds": 630373}, "event": "STOP"}

{"execute":"query-migrate"}                                                                                                                                                              
{"return": {"expected-downtime": 300, "status": "pre-switchover", "setup-time": 2, "total-time": 110111, "ram": {"total": 537403392, "postcopy-requests": 0, "dirty-sync-count": 2, "page-size": 4096, "remaining": 4141056, "mbps": 268.56432, "transferred": 59759367, "duplicate": 115920, "dirty-pages-rate": 72829, "skipped": 0, "normal-bytes": 58601472, "normal": 14307}}}
-----------------------------------------------------------------------

cancel block-job:
-----------------------------------------------------------------------
{"execute": "block-job-cancel","arguments":{"device":"drive_image1"}}
{"return": {}}
{"timestamp": {"seconds": 1510678024, "microseconds": 526240}, "event": "BLOCK_JOB_COMPLETED", "data": {"device": "virtio0", "len": 41126400, "offset": 41126400, "speed": 0, "type": "mirror"}}
-----------------------------------------------------------------------

d)on dst host, stop nbd server:
(qemu) nbd_server_stop
or
-----------------------------------------------------------------------
{"execute":"nbd-server-stop"}
{"return": {}}
-----------------------------------------------------------------------

e)on src host, continue pre-switchover
(qemu) migrate_continue pre-switchover
(qemu) info migrate
Migration status: completed
or 
-----------------------------------------------------------------------
{"execute":"query-migrate"}
{"return": {"status": "completed", "setup-time": 27, "downtime": 383168, "total-time": 401383, "ram": {"total": 8607039488, "postcopy-requests": 0, "dirty-sync-count": 5, "page-size": 4096, "remaining": 0, "mbps": 12.560209, "transferred": 621304158, "duplicate": 1964677, "dirty-pages-rate": 0, "skipped": 0, "normal-bytes": 602443776, "normal": 147081}}}
-----------------------------------------------------------------------

f)on src host, check the vm status
(qemu) info status 
VM status: running
or
-----------------------------------------------------------------------
{"execute":"query-status"}
{"return": {"status": "running", "singlestep": false, "running": true}}
-----------------------------------------------------------------------

So, this bug is fixed.

(In reply to xianwang from comment #6)
> Hi, Dave,
> I have found some information about your patch, that in this link:
> http://patchwork.ozlabs.org/cover/827760/
> there is two work flow as said in this link,
> The precopy flow is:
> active->pre-switchover->device->completed
> 
> The postcopy flow is:
> active->pre-switchover->postcopy-active->completed
> 
> But I didn't know the detail test steps for this bug, could you give me the
> detail test steps and expected results for these two scenarios? thanks

Although I have added a case for scenario "precopy flow" to migration test plan, I think I should also add the second scenario "postcopy flow", I have tried two different test steps but I didn't catch the "postcopy-active" status of migration, the test flows are as following:

a)                
block mirror-->"BLOCK_JOB_READY"-->live migration-->"Migration status: pre-switchover"-->block-job-cancel-->nbd_server_stop-->(qemu)migrate_continue pre-switchover-->(qemu) migrate_start_postcopy-->"Migration status: completed"

b)block mirror-->"BLOCK_JOB_READY"-->live migration-->(qemu) migrate_start_postcopy-->"Migration status: pre-switchover"-->block-job-cancel-->nbd_server_stop-->(qemu)migrate_continue pre-switchover-->"Migration status: completed"

questions:
1)In these two test scenarios about postcopy, I can't catch "postcopy-active" status of migration, is there something wrong for these steps? or what is the right way for postcopy flow? 

2)it seems that the state of "Migration status: pre-switchover" is a state that guest and migration process are both paused; only execute "migrate_continue pre-switchover", migration could continue and will complete directly, yes? so, any operation after "migrate_continue pre-switchover" would not work including "(qemu) migrate_start_postcopy", yes? btw, what is the condition of switching to "pre-switchover" for migration? Is that condition same with the condition of going into "downtime"?

Dave and Kashyap, could you help to reply the above questions? thanks

(In reply to Kashyap Chamarthy from comment #16)

[...]

> The right order using 'pause-before-switchover' is documented in the text
> file I attached in comment#10.
> 
> Copying here's the full sequence:
> 
> [I didn't copy / paste the first step intentionally; it just launches source
> & destination QEMU -- refer the file in comment#10].

Dave pointed out on IRC that I was missing the "-S" (Do not start CPU at start-up) in my notes in comment#10 for the destination QEMU command-line. 

So don't forget to note that when reusing those notes that you ought to start your destination QEMU with "-S"

[...]

(In reply to Kashyap Chamarthy from comment #20)
> (In reply to Kashyap Chamarthy from comment #16)
> 
> [...]
> 
> > The right order using 'pause-before-switchover' is documented in the text
> > file I attached in comment#10.
> > 
> > Copying here's the full sequence:
> > 
> > [I didn't copy / paste the first step intentionally; it just launches source
> > & destination QEMU -- refer the file in comment#10].
> 
> Dave pointed out on IRC that I was missing the "-S" (Do not start CPU at
> start-up) in my notes in comment#10 for the destination QEMU command-line. 
> 
> So don't forget to note that when reusing those notes that you ought to
> start your destination QEMU with "-S"

 So Dave explains to me on IRC that when using "-incoming" on destination, using "-S" is not *mandatory*.  (But if you _do_ start your destination QEMU with "-S", then you need to issue the QMP command "cont" to start the vCPUs again.)

To that end, I my updated instructions are here:

https://kashyapc.fedorapeople.org/virt/qemu/Live-Migration-drive-mirror+NBD-pause-before-switchover.txt

(In reply to xianwang from comment #19)
> (In reply to xianwang from comment #6)
> > Hi, Dave,
> > I have found some information about your patch, that in this link:
> > http://patchwork.ozlabs.org/cover/827760/
> > there is two work flow as said in this link,
> > The precopy flow is:
> > active->pre-switchover->device->completed
> > 
> > The postcopy flow is:
> > active->pre-switchover->postcopy-active->completed
> > 
> > But I didn't know the detail test steps for this bug, could you give me the
> > detail test steps and expected results for these two scenarios? thanks
> 
> Although I have added a case for scenario "precopy flow" to migration test
> plan, I think I should also add the second scenario "postcopy flow", I have
> tried two different test steps but I didn't catch the "postcopy-active"
> status of migration, the test flows are as following:

Postcopy with block migration is something that's not yet ready (it's waiting on a series by mreitz that does it a different way from block_mirror); so the test isn't really valid for postcopy yet.

> a)                
> block mirror-->"BLOCK_JOB_READY"-->live migration-->"Migration status:
> pre-switchover"-->block-job-cancel-->nbd_server_stop--
> >(qemu)migrate_continue pre-switchover-->(qemu)
> migrate_start_postcopy-->"Migration status: completed"
> 
> b)block mirror-->"BLOCK_JOB_READY"-->live migration-->(qemu)
> migrate_start_postcopy-->"Migration status:
> pre-switchover"-->block-job-cancel-->nbd_server_stop--
> >(qemu)migrate_continue pre-switchover-->"Migration status: completed"
> 
> questions:
> 1)In these two test scenarios about postcopy, I can't catch
> "postcopy-active" status of migration, is there something wrong for these
> steps? or what is the right way for postcopy flow? 

To catch a postcopy case you need to have a guest that's very busy accessing memory and/or a slow network bandwidth.

> 2)it seems that the state of "Migration status: pre-switchover" is a state
> that guest and migration process are both paused; only execute
> "migrate_continue pre-switchover", migration could continue and will
> complete directly, yes? so, any operation after "migrate_continue
> pre-switchover" would not work including "(qemu) migrate_start_postcopy",
> yes?

If performing postcopy, you'll have done a migrate_start_postcopy and then hit the pre-switchover state after it.
(Or migration finishes without needing postcopy, in which case you hit pre-switchover before giving migrate_start_postcopy).

> btw, what is the condition of switching to "pre-switchover" for
> migration? Is that condition same with the condition of going into
> "downtime"?

Yes, it's about the same time; but it gives the management layer a chance to do some cleanup just after going into downtime.

> 
> Dave and Kashyap, could you help to reply the above questions? thanks

(In reply to Dr. David Alan Gilbert from comment #22)
> (In reply to xianwang from comment #19)
> > (In reply to xianwang from comment #6)
> > > Hi, Dave,
> > > I have found some information about your patch, that in this link:
> > > http://patchwork.ozlabs.org/cover/827760/
> > > there is two work flow as said in this link,
> > > The precopy flow is:
> > > active->pre-switchover->device->completed
> > > 
> > > The postcopy flow is:
> > > active->pre-switchover->postcopy-active->completed
> > > 
> > > But I didn't know the detail test steps for this bug, could you give me the
> > > detail test steps and expected results for these two scenarios? thanks
> > 
> > Although I have added a case for scenario "precopy flow" to migration test
> > plan, I think I should also add the second scenario "postcopy flow", I have
> > tried two different test steps but I didn't catch the "postcopy-active"
> > status of migration, the test flows are as following:
> 
> Postcopy with block migration is something that's not yet ready (it's
> waiting on a series by mreitz that does it a different way from
> block_mirror); so the test isn't really valid for postcopy yet.
> 
FYI, I didn't need to add the case of "postcopy flow"? 

> > a)                
> > block mirror-->"BLOCK_JOB_READY"-->live migration-->"Migration status:
> > pre-switchover"-->block-job-cancel-->nbd_server_stop--
> > >(qemu)migrate_continue pre-switchover-->(qemu)
> > migrate_start_postcopy-->"Migration status: completed"
> > 
> > b)block mirror-->"BLOCK_JOB_READY"-->live migration-->(qemu)
> > migrate_start_postcopy-->"Migration status:
> > pre-switchover"-->block-job-cancel-->nbd_server_stop--
> > >(qemu)migrate_continue pre-switchover-->"Migration status: completed"
> > 
> > questions:
> > 1)In these two test scenarios about postcopy, I can't catch
> > "postcopy-active" status of migration, is there something wrong for these
> > steps? or what is the right way for postcopy flow? 
> 
> To catch a postcopy case you need to have a guest that's very busy accessing
> memory and/or a slow network bandwidth.
> 
> > 2)it seems that the state of "Migration status: pre-switchover" is a state
> > that guest and migration process are both paused; only execute
> > "migrate_continue pre-switchover", migration could continue and will
> > complete directly, yes? so, any operation after "migrate_continue
> > pre-switchover" would not work including "(qemu) migrate_start_postcopy",
> > yes?
> 
> If performing postcopy, you'll have done a migrate_start_postcopy and then
> hit the pre-switchover state after it.
> (Or migration finishes without needing postcopy, in which case you hit
> pre-switchover before giving migrate_start_postcopy).
> 
As you said, I have "stress --io 4" in guest, the scenario "a)" of my test is same with that you said the first one, the scenario "b)" of my test is as you said the other scenario. I think the first scenario is more reasonable;
so how do you think it's better to add the scenario "a)" to test plan or don't need to add "postcopy flow" this case?

 
> > btw, what is the condition of switching to "pre-switchover" for
> > migration? Is that condition same with the condition of going into
> > "downtime"?
> 
> Yes, it's about the same time; but it gives the management layer a chance to
> do some cleanup just after going into downtime.
> 
> > 
> > Dave and Kashyap, could you help to reply the above questions? thanks

(In reply to Dr. David Alan Gilbert from comment #22)
> (In reply to xianwang from comment #19)
> > (In reply to xianwang from comment #6)
> > > Hi, Dave,
> > > I have found some information about your patch, that in this link:
> > > http://patchwork.ozlabs.org/cover/827760/
> > > there is two work flow as said in this link,
> > > The precopy flow is:
> > > active->pre-switchover->device->completed
> > > 
> > > The postcopy flow is:
> > > active->pre-switchover->postcopy-active->completed
> > > 
> > > But I didn't know the detail test steps for this bug, could you give me the
> > > detail test steps and expected results for these two scenarios? thanks
> > 
> > Although I have added a case for scenario "precopy flow" to migration test
> > plan, I think I should also add the second scenario "postcopy flow", I have
> > tried two different test steps but I didn't catch the "postcopy-active"
> > status of migration, the test flows are as following:
> 
> Postcopy with block migration is something that's not yet ready (it's
> waiting on a series by mreitz that does it a different way from
> block_mirror); so the test isn't really valid for postcopy yet.
> 
FYI, I didn't need to add the case of "postcopy flow"? 

> > a)                
> > block mirror-->"BLOCK_JOB_READY"-->live migration-->"Migration status:
> > pre-switchover"-->block-job-cancel-->nbd_server_stop--
> > >(qemu)migrate_continue pre-switchover-->(qemu)
> > migrate_start_postcopy-->"Migration status: completed"
> > 
> > b)block mirror-->"BLOCK_JOB_READY"-->live migration-->(qemu)
> > migrate_start_postcopy-->"Migration status:
> > pre-switchover"-->block-job-cancel-->nbd_server_stop--
> > >(qemu)migrate_continue pre-switchover-->"Migration status: completed"
> > 
> > questions:
> > 1)In these two test scenarios about postcopy, I can't catch
> > "postcopy-active" status of migration, is there something wrong for these
> > steps? or what is the right way for postcopy flow? 
> 
> To catch a postcopy case you need to have a guest that's very busy accessing
> memory and/or a slow network bandwidth.
> 
> > 2)it seems that the state of "Migration status: pre-switchover" is a state
> > that guest and migration process are both paused; only execute
> > "migrate_continue pre-switchover", migration could continue and will
> > complete directly, yes? so, any operation after "migrate_continue
> > pre-switchover" would not work including "(qemu) migrate_start_postcopy",
> > yes?
> 
> If performing postcopy, you'll have done a migrate_start_postcopy and then
> hit the pre-switchover state after it.
> (Or migration finishes without needing postcopy, in which case you hit
> pre-switchover before giving migrate_start_postcopy).
> 
As you said, I have "stress --io 4" in guest, the scenario "a)" of my test is same with that you said the first one, the scenario "b)" of my test is as you said the other scenario. I think the first scenario is more reasonable;
so how do you think it's better to add the scenario "a)" to test plan or don't need to add "postcopy flow" this case?

 
> > btw, what is the condition of switching to "pre-switchover" for
> > migration? Is that condition same with the condition of going into
> > "downtime"?
> 
> Yes, it's about the same time; but it gives the management layer a chance to
> do some cleanup just after going into downtime.
> 
> > 
> > Dave and Kashyap, could you help to reply the above questions? thanks

For the postcopy scenario is need to confirm, I modify this bug to on_qa, if this issue resolved, I will verify this bug, thanks for understanding

Yes, you don't need to do postcopy; we'll sort out postcopy+storage behaviour when Max's patches are done.

(In reply to Dr. David Alan Gilbert from comment #25)
> Yes, you don't need to do postcopy; we'll sort out postcopy+storage
> behaviour when Max's patches are done.

OK, got it, thanks for Dave's confirmation

According to comment25, this bug is fixed.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:1104