Bug 1475272 - MAC addresses are not freed when a storage domain is destroyed from a data-center
MAC addresses are not freed when a storage domain is destroyed from a data-ce...
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage (Show other bugs)
Unspecified Unspecified
unspecified Severity high (vote)
: ovirt-4.1.5
Assigned To: Maor
Natalie Gavrielov
Depends On: 1405761
  Show dependency treegraph
Reported: 2017-07-26 07:00 EDT by Eyal Shenitzky
Modified: 2017-08-23 04:03 EDT (History)
3 users (show)

See Also:
Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2017-08-23 04:03:48 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑4.1+

Attachments (Terms of Use)
engine and vdsm logs (825.33 KB, application/x-gzip)
2017-07-26 07:00 EDT, Eyal Shenitzky
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 79854 master MERGED core: Release mac addresses on force remove SD. 2017-08-02 17:01 EDT
oVirt gerrit 80151 ovirt-engine-4.1 MERGED core: Release mac addresses on force remove SD. 2017-08-09 03:31 EDT

  None (edit)
Description Eyal Shenitzky 2017-07-26 07:00:25 EDT
Created attachment 1304690 [details]
engine and vdsm logs

Description of problem:
When destroying storage domain with a VM which has a disk on the same domain, the VM MAC address failed to free and return to the storage pool.

Eventually, the MAC pool got empty and it was impossible to add VMs with NIC while there was less VMs then the address pool number in the engine. 

The behavior is similar to https://bugzilla.redhat.com/show_bug.cgi?id=1405761
But with destroy command instead of detach command.

2017-07-25 14:36:53,930+03 WARN  [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (default task-25) [] Validation of action 'AddVmFromTemplate' failed for user admin@in
2017-07-25 14:36:53,931+03 INFO  [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (default task-25) [] Lock freed to object 'EngineLock:{exclusiveLocks='[vm_TestCase118
61_REST_ISCS_2514365131=VM_NAME]', sharedLocks='[71b5212b-2959-449d-920a-440ae6b4a801=TEMPLATE, dc58c4bd-4f45-49af-bf4b-259c34b34f4e=DISK]'}'
2017-07-25 14:36:53,942+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-25) [] Operation Failed: [Cannot add VM. Not enough MAC addre
sses left in MAC Address Pool.]

Version-Release number of selected component (if applicable):
Engine - 4.2.0-0.0.master.20170718165351.gita56ecd5.el7.centos
VDSM - 4.20.1-207.git518eecc.el7.centos.x86_64

How reproducible:

Steps to Reproduce:
1.Create a storage domain
2.Create a VM with disk and NIC on the domain from step 1
3.Deactivate and destroy the domain from step 1

* repeat steps 1-3 for several times until the MAC poll addresses will become empty

Actual results:
After destroying the domain with the VMs the MAC address pool got empty

Expected results:
MAC addresses should get back to the engine addresses pool

Additional info:
VDSM and Engine logs attached
Comment 1 Allon Mureinik 2017-07-26 07:53:35 EDT
This may have implications for DR scenarios.
I'm guessing this is a reasonably low hanging fruit, and should follow the same solution as bug 1405761.

If it isn't, let's rethink the targetting.
Comment 2 Natalie Gavrielov 2017-08-16 10:44:21 EDT
Verified, using builds: 

Scenario performed:
1. Create SD.
2. Create VM, with disk (on SD from step 1) and nic (save it's mac address - will be 
   used in step 4).
3. Deactivate and destroy the domain.
4. Create a VM - and manually configure it's mac address to be the mac address of the 
   VM created in step 2.

Result: VM was created (since this mac address is no longer taken),

Note You need to log in before you can comment on or make changes to this bug.