Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1482474

Summary: [Memory hot unplug] LookupError: Device instance for device identified by alias dimm6 and type memory not found
Product: [oVirt] ovirt-engine Reporter: Israel Pinto <ipinto>
Component: BLL.VirtAssignee: jniederm
Status: CLOSED WORKSFORME QA Contact: Israel Pinto <ipinto>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.2.0CC: bugs, ipinto, michal.skrivanek
Target Milestone: ovirt-4.2.0Flags: michal.skrivanek: ovirt-4.2?
rule-engine: planning_ack+
ipinto: devel_ack?
ipinto: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-09-20 13:59:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1228543    
Attachments:
Description Flags
logs
none
VM devices - unplug memory none

Description Israel Pinto 2017-08-17 11:20:25 UTC
Created attachment 1314642 [details]
logs

Description of problem:
Failed to hot unplug memory device. with error:
LookupError: Device instance for device identified by alias dimm6 and type memory not found

Version-Release number of selected component (if applicable):
Software version:4.2.0-0.0.master.20170813134654.gitaee967b.el7.centos


Steps to Reproduce:
1. Create VM with OS with 4GB, Run VM
2. Hotplug about 6GB of memory (With different sizes) 
have at least 10 memory devices 
3. Run on VM 2 load process with 4.5 GB each
4. Start Hot unplug memory.     

Actual results:
After unplug of 5 GB tone of the process on VM stopped,
Continue to remove memory and in the last one got the error.

Happened once.

Additional info:
VDSM log:
2017-08-17 10:44:54,902+0300 INFO  (jsonrpc/6) [vdsm.api] START hotunplugMemory(params={'vmId': 'b9cad82e-2708-4b14-9392-7ea66959d38f', 'memory': {'node': 0, 'specParams': {'node': '0', 'size': '131072'}, 'alias': 'dimm6', 'deviceId': '65d028d7-d507-4002-881f-8636ed9bf6c4', 'address': {'slot': '6', 'base': '0x230000000', 'type': 'dimm'}, 'device': 'memory', 'type': 'memory', 'size': 131072}}) from=::ffff:10.35.161.243,50070, flow_id=737c87a4-0f99-481a-b5a8-766610376961 (api:46)
2017-08-17 10:44:54,903+0300 INFO  (jsonrpc/6) [vdsm.api] FINISH hotunplugMemory error=Device instance for device identified by alias dimm6 and type memory not found from=::ffff:10.35.161.243,50070, flow_id=737c87a4-0f99-481a-b5a8-766610376961 (api:50)
2017-08-17 10:44:54,904+0300 ERROR (jsonrpc/6) [api] FINISH hotunplugMemory error=Device instance for device identified by alias dimm6 and type memory not found (api:127)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 117, in method
    ret = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 404, in hotunplugMemory
    return self.vm.hotunplugMemory(params)
  File "<string>", line 2, in hotunplugMemory
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
    ret = func(*args, **kwargs)
  File "<string>", line 2, in hotunplugMemory
  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 94, in method
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 3069, in hotunplugMemory
    self._devices, hwclass.MEMORY, params['memory']['alias'])
  File "/usr/lib/python2.7/site-packages/vdsm/virt/vmdevices/common.py", line 104, in lookup_device_by_alias
    'and type %s not found' % (alias, dev_type,))
LookupError: Device instance for device identified by alias dimm6 and type memory not found
2017-08-17 10:44:54,914+0300 INFO  (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call VM.hotunplugMemory failed (error 100) in 0.01 seconds (__init__:604)


Engine log:
2017-08-17 10:44:54,895+03 INFO  [org.ovirt.engine.core.vdsbroker.HotUnplugMemoryVDSCommand] (default task-10) [737c87a4-0f99-481a-b5a8-766610376961] START, HotUnplugMemoryVDSCommand(HostName = host_mixed_1, Params:{hostId='a582cc45-5d8c-40ab-8600-5e12476d1c52'}), log id: 1e0eedbb
2017-08-17 10:44:54,918+03 ERROR [org.ovirt.engine.core.vdsbroker.HotUnplugMemoryVDSCommand] (default task-10) [737c87a4-0f99-481a-b5a8-766610376961] Failed in 'HotUnplugMemoryVDS' method
2017-08-17 10:44:54,926+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-10) [737c87a4-0f99-481a-b5a8-766610376961] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM host_mixed_1 command HotUnplugMemoryVDS failed: General Exception: ('Device instance for device identified by alias dimm6 and type memory not found',)
2017-08-17 10:44:54,926+03 ERROR [org.ovirt.engine.core.vdsbroker.HotUnplugMemoryVDSCommand] (default task-10) [737c87a4-0f99-481a-b5a8-766610376961] Command 'HotUnplugMemoryVDSCommand(HostName = host_mixed_1, Params:{hostId='a582cc45-5d8c-40ab-8600-5e12476d1c52'})' execution failed: VDSGenericException: VDSErrorException: Failed to HotUnplugMemoryVDS, error = General Exception: ('Device instance for device identified by alias dimm6 and type memory not found',), code = 100
2017-08-17 10:44:54,927+03 INFO  [org.ovirt.engine.core.vdsbroker.HotUnplugMemoryVDSCommand] (default task-10) [737c87a4-0f99-481a-b5a8-766610376961] FINISH, HotUnplugMemoryVDSCommand, log id: 1e0eedbb
2017-08-17 10:44:54,959+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-10) [737c87a4-0f99-481a-b5a8-766610376961] EVENT_ID: MEMORY_HOT_UNPLUG_FAILED(2,047), Failed to hot unplug memory device (65d028d7-d507-4002-881f-8636ed9bf6c4) of size 131072 out of VM 'test_memory_hotplug': General Exception: ('Device instance for device identified by alias dimm6 and type memory not found',)

Comment 1 jniederm 2017-08-29 18:50:32 UTC
Hi Israel,

the error was caused by asking for hot unplug of particular dimm two times withing 10 seconds. Both engine and vdsm logs show:
* request for unplug of dimm6 at 2017-08-17 10:44:44
* completion of this request at 2017-08-17 10:44:48
* another request for unplug of dimm6 at 2017-08-17 10:44:54, just 10 seconds after the previous one

However UI is not supposed to allow that and I struggle replicate that. The Hot Unplug button is supposed to remain disabled for 15 seconds after being pressed.

Can you replicate that?

Comment 2 Israel Pinto 2017-08-30 07:05:52 UTC
I recheck it 
It not possible to do unplug twice to the same device, see attached screenshot.
The device is block after you unplug it.

Comment 3 Israel Pinto 2017-08-30 07:07:14 UTC
Created attachment 1319885 [details]
VM devices - unplug memory

Comment 4 jniederm 2017-08-30 12:46:04 UTC
Hi, can you reproduce this bug according to Steps to reproduce? I haven't managed to reproduce this problem.

Comment 5 Israel Pinto 2017-09-18 07:23:39 UTC
Hi,
I recheck it with version [1] and it not possible to unplug the same device twice, while unplug the memory device the device is blocked from the GUI.
So it not reproduced.     
see attached screenshot.

[1] Software version:4.2.0-0.0.master.20170911195522.git5ddba50.el7.centos