This bug fixes several issues with insufficient synchronization when accessing MAC Pools, which may cause the Manager to allocate the same MAC address to two different virtual machines.
DescriptionGermano Veit Michel
2016-11-16 00:24:33 UTC
Description of problem:
When additional VmPools are created, many of the VMs from the new VmPool end up with the same MAC address of existing VMs from other Pools which were created many days ago.
When these VMs with duplicate MACs are started, they lose the tap device as the MAC is already in use. See:
2016-11-09 03:00:10,787 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-30) [699b0a6d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 has MAC address 00:1a:4a:16:01:6c which is in use, therefore it is being unplugged from VM AAA_03.
So the VM ends up without Networking, rendering it completely useless.
Here is an example of two VMs from different Pools and created in completely different points in time:
2016-10-24 14:31:35.292+08 | VM AAA_03 creation has been completed.
2016-10-24 14:32:19.741+08 | VM Pool AAA (containing 20 VMs) was created by <removed>
2016-10-31 16:16:35.77+08 | VM BBB_01 creation has been completed.
2016-10-31 16:16:37.132+08 | VM Pool BBB (containing 10 VMs) was created by <removed>
Result:
vm_name | mac_addr
------------------+-------------------
AAA_03 | 00:1a:4a:16:01:6c
BBB_013 | 00:1a:4a:16:01:6c
Deleting all Pools and creating them again seems to yield the same result.
Version-Release number of selected component (if applicable):
ovirt-engine-4.0.4.4-0.1.el7ev.noarch
How reproducible:
* 0% after many tries
* Happens constantly on customer site
Actual results:
Duplicate MACs
Expected results:
No Duplicate MACs
Additional info:
* Allow Duplicate Macs option is disabled
* Using default MAC pool, no extra configurations
* Templates which these VMs are based on were imported and have MACs out of the MacPool range
(Does not seem related as I tried this as well)
* Pools are created by different users (aaa)
(Does not seem related as I tried this as well)
* It's always 2 VMs with the same MAC, never more than 2.
Verified with rhevm-4.1.0-0.3.beta2.el7.
I was not able to re produce the bug, added the test case to our automation, so if there's still a problem, hopefully we'll hit it some day.
Description of problem: When additional VmPools are created, many of the VMs from the new VmPool end up with the same MAC address of existing VMs from other Pools which were created many days ago. When these VMs with duplicate MACs are started, they lose the tap device as the MAC is already in use. See: 2016-11-09 03:00:10,787 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-6-thread-30) [699b0a6d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Network Interface nic1 has MAC address 00:1a:4a:16:01:6c which is in use, therefore it is being unplugged from VM AAA_03. So the VM ends up without Networking, rendering it completely useless. Here is an example of two VMs from different Pools and created in completely different points in time: 2016-10-24 14:31:35.292+08 | VM AAA_03 creation has been completed. 2016-10-24 14:32:19.741+08 | VM Pool AAA (containing 20 VMs) was created by <removed> 2016-10-31 16:16:35.77+08 | VM BBB_01 creation has been completed. 2016-10-31 16:16:37.132+08 | VM Pool BBB (containing 10 VMs) was created by <removed> Result: vm_name | mac_addr ------------------+------------------- AAA_03 | 00:1a:4a:16:01:6c BBB_013 | 00:1a:4a:16:01:6c Deleting all Pools and creating them again seems to yield the same result. Version-Release number of selected component (if applicable): ovirt-engine-4.0.4.4-0.1.el7ev.noarch How reproducible: * 0% after many tries * Happens constantly on customer site Actual results: Duplicate MACs Expected results: No Duplicate MACs Additional info: * Allow Duplicate Macs option is disabled * Using default MAC pool, no extra configurations * Templates which these VMs are based on were imported and have MACs out of the MacPool range (Does not seem related as I tried this as well) * Pools are created by different users (aaa) (Does not seem related as I tried this as well) * It's always 2 VMs with the same MAC, never more than 2.