Bug 1475272

Summary: MAC addresses are not freed when a storage domain is destroyed from a data-center
Product: [oVirt] ovirt-engine Reporter: Eyal Shenitzky <eshenitz>
Component: BLL.StorageAssignee: Maor <mlipchuk>
Status: CLOSED CURRENTRELEASE QA Contact: Natalie Gavrielov <ngavrilo>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2.0CC: amureini, bugs, mburman
Target Milestone: ovirt-4.1.5Flags: rule-engine: ovirt-4.1+
Target Release: 4.1.5.2   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-23 08:03:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1405761    
Bug Blocks:    
Attachments:
Description Flags
engine and vdsm logs none

Description Eyal Shenitzky 2017-07-26 11:00:25 UTC 
Created attachment 1304690 [details]
engine and vdsm logs

Description of problem:
When destroying storage domain with a VM which has a disk on the same domain, the VM MAC address failed to free and return to the storage pool.

Eventually, the MAC pool got empty and it was impossible to add VMs with NIC while there was less VMs then the address pool number in the engine. 

The behavior is similar to https://bugzilla.redhat.com/show_bug.cgi?id=1405761
But with destroy command instead of detach command.

2017-07-25 14:36:53,930+03 WARN  [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (default task-25) [] Validation of action 'AddVmFromTemplate' failed for user admin@in
ternal-authz. Reasons: VAR__ACTION__ADD,VAR__TYPE__VM,MAC_POOL_NOT_ENOUGH_MAC_ADDRESSES
2017-07-25 14:36:53,931+03 INFO  [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (default task-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[vm_TestCase118
61_REST_ISCS_2514365131=VM_NAME]', sharedLocks='[71b5212b-2959-449d-920a-440ae6b4a801=TEMPLATE, dc58c4bd-4f45-49af-bf4b-259c34b34f4e=DISK]'}'
2017-07-25 14:36:53,942+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-25) [] Operation Failed: [Cannot add VM. Not enough MAC addre
sses left in MAC Address Pool.]

Version-Release number of selected component (if applicable):
Engine - 4.2.0-0.0.master.20170718165351.gita56ecd5.el7.centos
VDSM - 4.20.1-207.git518eecc.el7.centos.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Create a storage domain
2.Create a VM with disk and NIC on the domain from step 1
3.Deactivate and destroy the domain from step 1

* repeat steps 1-3 for several times until the MAC poll addresses will become empty

Actual results:
After destroying the domain with the VMs the MAC address pool got empty

Expected results:
MAC addresses should get back to the engine addresses pool

Additional info:
VDSM and Engine logs attached
Comment 1 Allon Mureinik 2017-07-26 11:53:35 UTC 
This may have implications for DR scenarios.
I'm guessing this is a reasonably low hanging fruit, and should follow the same solution as bug 1405761.

If it isn't, let's rethink the targetting.
Comment 2 Natalie Gavrielov 2017-08-16 14:44:21 UTC 
Verified, using builds: 
ovirt-engine-4.1.5.2-0.1.el7.noarch
vdsm-4.19.25-1.el7ev.x86_64

Scenario performed:
1. Create SD.
2. Create VM, with disk (on SD from step 1) and nic (save it's mac address - will be 
   used in step 4).
3. Deactivate and destroy the domain.
4. Create a VM - and manually configure it's mac address to be the mac address of the 
   VM created in step 2.

Result: VM was created (since this mac address is no longer taken),