Bug 1475272 - MAC addresses are not freed when a storage domain is destroyed from a data-center
Summary: MAC addresses are not freed when a storage domain is destroyed from a data-ce...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage
Version: 4.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ovirt-4.1.5
: 4.1.5.2
Assignee: Maor
QA Contact: Natalie Gavrielov
URL:
Whiteboard:
Depends On: 1405761
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-07-26 11:00 UTC by Eyal Shenitzky
Modified: 2017-08-23 08:03 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2017-08-23 08:03:48 UTC
oVirt Team: Storage
Embargoed:
rule-engine: ovirt-4.1+


Attachments (Terms of Use)
engine and vdsm logs (825.33 KB, application/x-gzip)
2017-07-26 11:00 UTC, Eyal Shenitzky
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 79854 0 master MERGED core: Release mac addresses on force remove SD. 2017-08-02 21:01:34 UTC
oVirt gerrit 80151 0 ovirt-engine-4.1 MERGED core: Release mac addresses on force remove SD. 2017-08-09 07:31:47 UTC

Description Eyal Shenitzky 2017-07-26 11:00:25 UTC
Created attachment 1304690 [details]
engine and vdsm logs

Description of problem:
When destroying storage domain with a VM which has a disk on the same domain, the VM MAC address failed to free and return to the storage pool.

Eventually, the MAC pool got empty and it was impossible to add VMs with NIC while there was less VMs then the address pool number in the engine. 

The behavior is similar to https://bugzilla.redhat.com/show_bug.cgi?id=1405761
But with destroy command instead of detach command.

2017-07-25 14:36:53,930+03 WARN  [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (default task-25) [] Validation of action 'AddVmFromTemplate' failed for user admin@in
ternal-authz. Reasons: VAR__ACTION__ADD,VAR__TYPE__VM,MAC_POOL_NOT_ENOUGH_MAC_ADDRESSES
2017-07-25 14:36:53,931+03 INFO  [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (default task-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[vm_TestCase118
61_REST_ISCS_2514365131=VM_NAME]', sharedLocks='[71b5212b-2959-449d-920a-440ae6b4a801=TEMPLATE, dc58c4bd-4f45-49af-bf4b-259c34b34f4e=DISK]'}'
2017-07-25 14:36:53,942+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-25) [] Operation Failed: [Cannot add VM. Not enough MAC addre
sses left in MAC Address Pool.]

Version-Release number of selected component (if applicable):
Engine - 4.2.0-0.0.master.20170718165351.gita56ecd5.el7.centos
VDSM - 4.20.1-207.git518eecc.el7.centos.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Create a storage domain
2.Create a VM with disk and NIC on the domain from step 1
3.Deactivate and destroy the domain from step 1

* repeat steps 1-3 for several times until the MAC poll addresses will become empty

Actual results:
After destroying the domain with the VMs the MAC address pool got empty

Expected results:
MAC addresses should get back to the engine addresses pool

Additional info:
VDSM and Engine logs attached

Comment 1 Allon Mureinik 2017-07-26 11:53:35 UTC
This may have implications for DR scenarios.
I'm guessing this is a reasonably low hanging fruit, and should follow the same solution as bug 1405761.

If it isn't, let's rethink the targetting.

Comment 2 Natalie Gavrielov 2017-08-16 14:44:21 UTC
Verified, using builds: 
ovirt-engine-4.1.5.2-0.1.el7.noarch
vdsm-4.19.25-1.el7ev.x86_64

Scenario performed:
1. Create SD.
2. Create VM, with disk (on SD from step 1) and nic (save it's mac address - will be 
   used in step 4).
3. Deactivate and destroy the domain.
4. Create a VM - and manually configure it's mac address to be the mac address of the 
   VM created in step 2.

Result: VM was created (since this mac address is no longer taken),


Note You need to log in before you can comment on or make changes to this bug.