Bug 1462198
Summary: | Cluster assigning MACs from a different's cluster MAC pool | ||||||
---|---|---|---|---|---|---|---|
Product: | [oVirt] ovirt-engine | Reporter: | nicolas | ||||
Component: | Backend.Core | Assignee: | Martin Mucha <mmucha> | ||||
Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Meni Yakove <myakove> | ||||
Severity: | low | Docs Contact: | |||||
Priority: | medium | ||||||
Version: | 4.1.1.8 | CC: | bugs, danken, ylavi | ||||
Target Milestone: | ovirt-4.2.0 | Flags: | rule-engine:
ovirt-4.2+
|
||||
Target Release: | --- | ||||||
Hardware: | Unspecified | ||||||
OS: | Unspecified | ||||||
Whiteboard: | |||||||
Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
Doc Text: | Story Points: | --- | |||||
Clone Of: | Environment: | ||||||
Last Closed: | 2017-11-09 13:41:28 UTC | Type: | Bug | ||||
Regression: | --- | Mount Type: | --- | ||||
Documentation: | --- | CRM: | |||||
Verified Versions: | Category: | --- | |||||
oVirt Team: | Network | RHEL 7.3 requirements from Atomic Host: | |||||
Cloudforms Team: | --- | Target Upstream Version: | |||||
Embargoed: | |||||||
Attachments: |
|
Description
nicolas
2017-06-16 12:13:04 UTC
this is extremely unlikely to happen, that would have to mean, that some code related to cluster A holds ID of cluster B and use that ID to get related MAC pool instead of one it should use... 1.can you give us example of MAC which were badly allocated from different ranges? 2. do you know what operation led to this state? 'What' asked for mac allocation? Created attachment 1291046 [details]
MAC addresses result
1) I have plenty of them, I run the code below in Python to extract MAC addresses on the VDI cluster (which contains the Adicional-DOCINT2 pool) and I'm attaching the result. Note that ONLY MAC addresses in the 00:1a:4a:97:ee:XX format should be valid according to the configuration.
for vm in vms_serv.list(search='Datacenter=VDI'):
nics = conn.follow_link(vm.nics)
for nic in nics:
mac = nic.mac.address
print "VM: %s: %s" % (vm.name, mac)
2) All machines are new machines based on a Windows template. The funny thing is that the base machine (from which the template was created) seems to have a correct MAC address. Every instantiated machine has 2 interfaces. However, the template only has one, from what I assume that students added manually one interface. Another thing you can see in the attached file is that most of the machines have an incorrect range's MAC addresses for both for the "template" interface and the one that they added manually. I tried to reproduce the issue creating a new machine based on this template, but I couldn't. I tried both from the adminportal and the userportal, and this time it seems to be assigning valid MAC addresses, however, as you can see, there are lots of invalid addresses. As far as I know, no engine update has been applied from the time these machines were created up until now.
If you need to debug something on our platform we can provide you some credentials so you can try by yourself if you wish.
foreword: I'm sorry, I'm not that familiar with python & python api, maybe others will help and correct me if needed. Ok, I took you attachment and did: at MAC-addresses.txt | sed "s/.* //" | sed "s/:..$//" | sort | uniq -c 12 00:1a:4a:4d:cc 11 00:1a:4a:4d:dd 7 00:1a:4a:97:5f 59 00:1a:4a:97:6e 208 00:1a:4a:97:6f 21 00:1a:4a:97:ee which shows, that range 00:1a:4a:97:6e:00 - 00:1a:4a:97:6f:ff which belongs to Default MacPool, is much more often used. This is little bit odd. Because after patch I00f4ebf8371c1a0e531baf7ef764c99d0be63ab2 was merged (it is contained in your version), all ranges should be visited equally often when serving mac requests. Lets assume, that all ranges are part of one mac pool(to simulate, that 'stealing' macs is possible). That being said, range 00:1a:4a:97:6e:00 - 00:1a:4a:97:6f:ff should serve ± same amount of macs as other ranges. But here that does not hold; this range holds 267 while second most frequest range holds only 21. From this I'd conclude, that machines from report: a) were created when mac pool settings were different. b) mac addresses were assigned manually, and not assigned by engine. ——— I'll try to debug creating new vms from template into different cluster, as I'm not sure what will happen. === (note for me) found bugs in: I00f4ebf8371c1a0e531baf7ef764c99d0be63ab2 while checking for free macs in pool, last used range advances, leading to situation, that first all odd/even ranges are empties, and only then we proceed on remaining ones. Check for full macpool must not be operation considered as one 'using' mac range. 1. I created new VM in Default cluster with one nic, letting engine decide which mac to use. I verified, that selected mac ( 00:1a:4a:16:01:03)belongs to default cluster mac pool(00:1a:4a:16:01:00-00:1a:4a:16:01:01, 00:1a:4a:16:01:02-00:1a:4a:16:01:03) 2. I created another mac pool with range: 00:1a:4a:16:02:01-00:1a:4a:16:02:ff 3. made template out of VM(from step1), using VirtualMachines->Make Template. UI does not show mac when examining created template. 4. I created new cluster, and assigned to it newly created mac pool(from step2). 5. I created new VM using this template(from step 3), via Templates->newVm. I selected newly created cluster(from step4), named it, and asked for extra vmnic, as you students theoretially did. results: several attempts all creates only VMs residing in cluster created in step 4, using mac belonging to relevant mac pool(created in step 2), and not from one belonging to default cluster. So, creating vm from template does not reproduce your issue, WHILE TESTING ON MASTER. I'll retest on your version specifically, but there's very slim chance of change, there is almost no code touching of mac pools, and one present is almost certainly unrelated. For the record, on comment #3, manually assigning MACs is discarded in our case, I talked to the teacher and he told me how they created their machines (basically cloning) and no nic MACs were touched manually. However, I cannot discard case a (different pools at machine creation time). It's quite possible that some of us created an additional MAC pool on the second cluster (VDI). same behavior in 4.1.1.8. so to recap, I cannot reproduce claim, that creating VM from template (into different cluster) creates invalid macs. So if you tell, that no macs were added manually, it most probably must have been comment #3/ conclusion b. If you change mac pool setting related to cluster, then all already assigned macs are not reallocated (because VM can be running and can cause issues). I'd ask you to fix all invalid macs (or just be able to identify them somehow in future) and monitor if new invalid mac is assigned in future, and ideally how it was created, via which action. So far I tested everything I was able to come up with without any luck finding error. For one problem I found (should not affect your case) I provided a fix already. Ok, I'll monitor it again. Probably in a few months we well be able to check if this happens again, because the course will start again, with new machines, new students and so. Thanks. ok, if you have new info, please let me know. But please remember, if you just tell me: "it happened again", it won't help us(me) much. I need some certainty — we have this env, we did this, and this was the result. I did all I could come up with, but did not find any issue; if it happen again, I need narrower area so search. Thanks for you understanding. Ok, I'll lower the severity and postpone it to 4.2.0 |