Created attachment 831570 [details]
logs

Description of problem:
Disk that was live migrated between storage domains, cannot be hotplugged/hotunplugged to/from VM. Note that hotplug/unplug works fine in case the disk is part of a snapshot, so the hotplug/unplug fails because of another phase of the LSM.

Version-Release number of selected component (if applicable):
is25
vdsm-4.13.0-0.10.beta1.el6ev.x86_64
rhevm-3.3.0-0.37.beta1.el6ev.noarch


How reproducible:
100%

Steps to Reproduce:
Reproduced on a block storage with more than 1 SDs 
1. create a VM and install OS 
2. add a second disk to the VM and hotplug it to the VM
3. Move the new disk to another SD (live migration)
4. hotunplug the disk from the VM

Actual results:
The operation fails on vdsm:

Thread-48016::DEBUG::2013-12-02 15:05:43,673::vm::3487::vm.Vm::(hotunplugDisk) vmId=`28ae983b-1815-4a27-b751-4ddd4af5b90a`::Hotunplug disk xml: <disk device="disk" snapshot="no" type="block">
        <address bus="0x00" domain="0x0000" function="0x0" slot="0x09" type="pci"/>
        <source dev="/rhev/data-center/mnt/blockSD/3bbf19c4-3a0d-4a91-883a-9824245659ee/images/b1a4778d-354e-4720-9ec3-928002bd6483/07accae5-a751-42c9-bbc2-a3fcd7286ebb"/>
        <target bus="virtio" dev="vdd"/>
        <serial>b1a4778d-354e-4720-9ec3-928002bd6483</serial>
        <driver cache="none" error_policy="stop" io="native" name="qemu" type="qcow2"/>
</disk>

Thread-48016::ERROR::2013-12-02 15:05:43,673::BindingXMLRPC::1003::vds::(wrapper) unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/BindingXMLRPC.py", line 989, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/BindingXMLRPC.py", line 272, in vmHotunplugDisk
    return vm.hotunplugDisk(params)
  File "/usr/share/vdsm/API.py", line 454, in hotunplugDisk
    return curVm.hotunplugDisk(params)
  File "/usr/share/vdsm/vm.py", line 3490, in hotunplugDisk
    self.sdIds.remove(drive.domainID)
ValueError: list.remove(x): x not in list


Note that disk hotunplug operation will fail also (need to have a disk that was migrated while it was deactivated and the VM was running)

Expected results:
Hotplug/unplug should work 

Additional info: logs