Bug 1480238

Summary: RESTAPI- PUT request to update DC from 4.0->4.1 fails with REST response 'Cannot migrate MACs to another MAC pool, because that action would create duplicates in target MAC pool, which are not allowed. Problematic MACs are 00:1a:4a:16:25:b2'
Product: [oVirt] ovirt-engine Reporter: Avihai <aefrat>
Component: BLL.NetworkAssignee: Martin Mucha <mmucha>
Status: CLOSED CURRENTRELEASE QA Contact: Avihai <aefrat>
Severity: medium Docs Contact:
Priority: medium    
Version: 4.2.0CC: aefrat, bugs, mburman, pmatyas, ratamir
Target Milestone: ovirt-4.2.0Keywords: Automation, AutomationBlocker, Regression
Target Release: ---Flags: rule-engine: ovirt-4.2+
rule-engine: blocker+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1497614 (view as bug list) Environment:
Last Closed: 2017-12-20 11:40:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1497614    
Attachments:
Description Flags
engine & vdsm logs
none
REST print screen none

Description Avihai 2017-08-10 13:15:30 UTC
Created attachment 1311766 [details]
engine & vdsm logs

Description of problem:
Update DC from 4.0->4.1 fails ONLY with RESTAPI - Manually upgrading DC works.

REST response is:
'Cannot migrate MACs to another MAC pool, because that action would create duplicates in target MAC pool, which are not allowed. Problematic MACs are 	00:1a:4a:16:25:b2'

00:1a:4a:16:25:b2 is the MAC of the VM that is created on the new 4.0 DC/cluster

Many of my automated runs on storage_qcow2_v3 TP has the similar flow that fails when upgrading DC to 4.1 , so this is an automation blocker for me.

Version-Release number of selected component (if applicable):
Engine - > 4.2.0-0.0.master.20170809175855.git1082d9d.el7.centos
VDSM   -> 4.20.2-50.gitb718903

How reproducible:
100%

Steps to Reproduce:

Done Manually/RESTAPI:
1) Create 4.0 DC + cluster with a matching host
2) Create VM with NIC + a thin disk (1G)+ snapshot.
3) Upgrade cluster to 4.1 

Done ONLY VIA RESTAPI:
4) upgrade DC to 4.1 

Actual results:
REST request:
2017-08-10 15:34:17,016 - MainThread - datacenters - DEBUG - PUT request

url:/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c 

body:

<data_center>
    <version>
        <major>4</major>
        <minor>1</minor>
    </version>
</data_center>

Response is :
Cannot migrate MACs to another MAC pool, because that action would create duplicates in target MAC pool, which are not allowed. Problematic MACs are 	00:1a:4a:16:25:b2

This problematic MAC is the newly created VM MAC. (00:1a:4a:16:25:b2)


Expected results:
Upgrade succeeds.

Additional info:

Engine:
2017-08-10 15:34:17,132+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-5) [] Operation Failed: [Cannot migrate MACs to another MAC pool, because that action would create duplicates in target MAC pool, which are not allowed. Problematic MACs are 	00:1a:4a:16:25:b2]

REST REQUEST:
2017-08-10 15:34:17,016 - MainThread - datacenters - DEBUG - PUT request content is --  

url:/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c 


body:

<data_center>
    <version>
        <major>4</major>
        <minor>1</minor>
    </version>
</data_center>


REST RESPONSE:
2017-08-10 15:34:17,138 - MainThread - api_utils - ERROR - Failed to update element NOT as expected:
	Status: 409
	Reason: Conflict
	Detail: [Cannot migrate MACs to another MAC pool, because that action would create duplicates in target MAC pool, which are not allowed. Problematic MACs are 	00:1a:4a:16:25:b2]




-REST GET DC before Upgrade the get to the going to be upgraded DC  :

2017-08-10 15:34:17,013 - MainThread - datacenters - DEBUG - Response body for GET request is: 
<data_centers>
    <data_center href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c" id="ad59c7c8-05a6-4290-ad71-3b6103b1207c">
        <name>dc_upgrade_4_0_to_4_1</name>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/clusters" rel="clusters"/>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/networks" rel="networks"/>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/storagedomains" rel="storagedomains"/>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/permissions" rel="permissions"/>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/quotas" rel="quotas"/>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/qoss" rel="qoss"/>
        <link href="/ovirt-engine/api/datacenters/ad59c7c8-05a6-4290-ad71-3b6103b1207c/iscsibonds" rel="iscsibonds"/>
        <local>false</local>
        <quota_mode>disabled</quota_mode>
        <status>up</status>
        <storage_format>v3</storage_format>
        <supported_versions>
            <version>
                <major>4</major>
                <minor>1</minor>
            </version>
            <version>
                <major>4</major>
                <minor>0</minor>
            </version>
        </supported_versions>
        <version>
            <major>4</major>
            <minor>0</minor>
        </version>
        <mac_pool href="/ovirt-engine/api/macpools/58ca604b-017d-0374-0220-00000000014e" id="58ca604b-017d-0374-0220-00000000014e"/>

Comment 1 Dan Kenigsberg 2017-08-10 14:07:08 UTC
Could you please check if you see the same bug in recent 4.1 builds?

Comment 2 Avihai 2017-08-13 07:58:47 UTC
Hi Dan , Issue does not occur on latest 4.1 . 

Tested on ovirt-engine-4.1.5.2-0.1.el7.noarch

Comment 3 Martin Mucha 2017-08-13 09:30:20 UTC
just to give some specific info:

error got into master: Aug 9 2:39 PM
error was fixed in master: Aug 13 11:10 AM

So it was (luckily) very short period of time, thanks to the reporter(s) of this bug.

Comment 4 Avihai 2017-08-13 11:44:13 UTC
Verified at 4.2.0-0.0.master.20170811144920.gita423008.el7.centos

Comment 5 Avihai 2017-08-13 14:41:53 UTC
My bad , running automation again I still see the same issue in latest master.
It appears I did not add a VM when I verified this issue so the upgrade worked.

The Same setup (as above) is your's for further debug.

please ping me when done.

Engine log:
2017-08-13 17:18:14,409+03 INFO  [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-8) [5a71d545] Running command: CreateUserSessionCommand internal: false.
2017-08-13 17:18:14,471+03 WARN  [org.ovirt.engine.core.bll.storage.pool.UpdateStoragePoolCommand] (default task-8) [cca7ce5f-8055-469b-b877-1b27402b5da2] Validation of action 'UpdateStoragePool' failed for user
 admin@internal-authz. Reasons: VAR__TYPE__STORAGE__POOL,VAR__ACTION__UPDATE,ACTION_TYPE_FAILED_CANNOT_MIGRATE_MACS_DUE_TO_DUPLICATES,$ACTION_TYPE_FAILED_CANNOT_MIGRATE_MACS_DUE_TO_DUPLICATES_LIST         00:1a:
4a:16:25:ad,$ACTION_TYPE_FAILED_CANNOT_MIGRATE_MACS_DUE_TO_DUPLICATES_LIST_COUNTER 1
2017-08-13 17:18:14,481+03 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-8) [] Operation Failed: [Cannot migrate MACs to another MAC pool, because that action would create d
uplicates in target MAC pool, which are not allowed. Problematic MACs are    00:1a:4a:16:25:ad]

Comment 6 Avihai 2017-08-13 14:44:10 UTC
DC name = dc1
ID = 80ef202f-2454-483d-a1f0-beedb57e21c7

See print screen of the issue with postman app .

Comment 7 Avihai 2017-08-13 14:45:13 UTC
Created attachment 1312718 [details]
REST print screen

Comment 8 Avihai 2017-08-13 16:30:48 UTC
build was wrong , waiting for fixed build.

Comment 9 Avihai 2017-08-14 14:50:01 UTC
verified on 4.2.0-0.0.master.20170813134654.gitaee967b.el7.centos

Comment 10 Martin Mucha 2017-08-15 16:04:24 UTC
*** Bug 1481200 has been marked as a duplicate of this bug. ***

Comment 11 Sandro Bonazzola 2017-12-20 11:40:29 UTC
This bugzilla is included in oVirt 4.2.0 release, published on Dec 20th 2017.

Since the problem described in this bug report should be
resolved in oVirt 4.2.0 release, published on Dec 20th 2017, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.