Created attachment 1343300 [details] node0 nova compute log User executes: $ openstack server migrate --block-migration --live compute-1.localdomain inst2 In logs: DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/be8e35a8-53ca-4062-92fb-33d635fbc891) already exists, it is expected not to exist. Logs attached.
Created attachment 1343301 [details] node1 nova-compute.log
Avi, Could you please supply a complete set of reproducer steps for this issue, starting with the creation of a new instance? Could you also please describe the storage configuration on the 2 compute hosts. Could you please supply sosreports for all nodes covering the entire period from the creation of a new instance to the observed failure. Thanks, Matt
To be clear, the bug here is that we ended up in a state where the destination directory exists on a host where it shouldn't. So the actual failure we're seeing here, DestinationDiskExists, is a symptom of that bug which happens long after the bug occurred. I looked through the logs above and I saw various errors, but all were in pre-live-migrate, so nothing should have been created on the destination at that point. Use of shared storage would also potentially be relevant.
we created a simple setup including one controller + 2 compute nodes Cinder backend is XtremIO storage configure to work in multipath mode 1) created an instance 2) attach volume to instance from XtremIO backend 3) run migration command from one node to anther to verify that the attach volume can be migrate too
I can't reproduce this. An instance with no attached volume can live migrate between 2 computes multiple times. Same for a boot-from-volume instance using XtremIO storage. Note that live migration of a boot-from-image instance with attached storage was not supported in OSP8.
I run the scenario again getting the following error DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/5cda9ca6-46f4-4a24-b033-30b49a2f2d75) already exists, it is expected not to exist. The steps needed to run the scenario 1) Create volume from Image # openstack volume create --image cirros-0.3.4-x86_64-disk.img image1 2) Create instance from volume # openstack server create --flavor m1.tiny --volume image1 instance1 3) Create volume # openstack volume create --size 1 vol1 4) Attach volume to instance # openstack server add volume inst1 vol1 5) run instance migration to other node # server migrate --live compute-1.localdomain --block-migration inst1
(In reply to Avi Avraham from comment #8) > I run the scenario again getting the following error > DestinationDiskExists: The supplied disk path > (/var/lib/nova/instances/5cda9ca6-46f4-4a24-b033-30b49a2f2d75) already > exists, it is expected not to exist. > > The steps needed to run the scenario > 1) Create volume from Image > # openstack volume create --image cirros-0.3.4-x86_64-disk.img image1 > 2) Create instance from volume > # openstack server create --flavor m1.tiny --volume image1 instance1 > 3) Create volume > # openstack volume create --size 1 vol1 > 4) Attach volume to instance > # openstack server add volume inst1 vol1 > 5) run instance migration to other node > # server migrate --live compute-1.localdomain --block-migration inst1 This is also different to the bug report in https://bugzilla.redhat.com/show_bug.cgi?id=1386469#c26 and I can't reproduce this either.
I'm going to close this bug as we've never managed to replicate it. Continuing discussion on the original bug, which is what we've actually been working on.