Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1321637

Summary: libvirt fails to create a macvtap deivce if an attempted name was already created by some process other than libvirt
Product: Red Hat Enterprise Linux 6 Reporter: Laine Stump <laine>
Component: libvirtAssignee: Laine Stump <laine>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.7CC: dyuan, jsuchane, laine, mzhan, rbalakri, salmy, sherold, shyu, virt-bugs, xuzhang, yalzhang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-0.10.2-60.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1321546 Environment:
Last Closed: 2016-05-10 19:26:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Laine Stump 2016-03-28 17:37:08 UTC 
The following bug was introduced into the RHEL6 version of libvirt by the final patch in Bug 1276478. This means that there isn't yet a regression in a released build of libvirt, but there will be if this bug isn't resolved before release.

(I removed a proposed (incorrect) patch and related discussion from the transcript below, as it's irrelevant to getting the problem fixed)

+++ This bug was initially created as a clone of Bug #1321546 +++

Description of problem:

libvirt fail to generate correct macvtap name if there is already one 

Version-Release number of selected component (if applicable):

libvirt-1.3.2-1.el7.x86_64

How reproducible:

100%

Steps to Reproduce:

1. Create a macvtap device by ip command named macvtap1

# ip l add li enp2s0  macvtap1  type  macvtap
# ip -d l show macvtap1
6: macvtap1@enp2s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 500
    link/ether d2:d9:bb:d7:f6:5b brd ff:ff:ff:ff:ff:ff promiscuity 0 
    macvtap  mode vepa addrgenmode eui64 

2. Start a defined guest with interface type is direct

# virsh dumpxml r7 |grep "interface type" -A4
    <interface type='direct'>
      <mac address='52:54:00:0c:84:4d'/>
      <source dev='enp2s0' mode='vepa'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x02' slot='0x01' function='0x0'/>

# time virsh start r7 
error: Failed to start domain r7
error: internal error: Too many unreserved macvtap devices in use


real	0m50.638s
user	0m0.013s
sys	0m0.008s


3. Check libvirtd log

# grep virNetDevMacVLanCreateWithVPortProfile /tmp/libvirtd.log |wc -l
8192


2016-03-28 08:35:30.890+0000: 28927: info : virNetDevMacVLanReserveID:166 : reserving device macvtap1
2016-03-28 08:35:30.902+0000: 28927: debug : virFileClose:103 : Closed fd 25
2016-03-28 08:35:30.902+0000: 28927: info : virNetDevMacVLanReleaseID:203 : releasing device macvtap1
2016-03-28 08:35:30.902+0000: 28927: info : virNetDevMacVLanCreateWithVPortProfile:1089 : Device macvtap1 wasn't reserved but already existed, skipping
2016-03-28 08:35:30.902+0000: 28927: info : virNetDevMacVLanReserveID:166 : reserving device macvtap1


Actual results:


Expected results:

guest should started successfully with correct macvtap device, like macvtap2 if there is already macvtap1(which create not by libvirt) existing on host. 

Additional info:

this is a regression of commit 370608b, which introduce bitmap of in-use macvtap devices.



--- Additional comment from Laine Stump on 2016-03-28 13:13:54 EDT ---


The proper fix is to call virBitmapNextClearBit() (inside virNetDevMacVLanReserveID()) with the most recently failed id (or -1) rather than with 0. This is what I've done in the following patch, now pushed upstream:


commit 5b5f12cffa86a1b4527bde620c2a86c5e96ad7c5
Author: Laine Stump <laine>
Date:   Mon Mar 28 10:14:04 2016 -0400

    util: avoid getting stuck on macvtapN name created outside libvirt
Comment 1 Laine Stump 2016-03-28 19:51:09 UTC 
The upstream patch has been backported and posted on rhvirt-patches for inclusion in an upcoming build:

http://post-office.corp.redhat.com/archives/rhvirt-patches/2016-March/msg00439.html
Comment 2 Laine Stump 2016-03-28 19:51:53 UTC 
(Note that this bug was never in an officially release package, so no Doc Text is needed.)
Comment 3 Mike McCune 2016-03-28 22:54:17 UTC 
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune with any questions
Comment 4 Shanzhi Yu 2016-03-29 06:18:26 UTC 
Reproduce this with libvirt-0.10.2-59.el6.x86_64.
Steps to reproduce:

1. add macvtap device named macvtap1
# ip link add link eth0 name macvtap1 type macvtap

# ip -d l show macvtap1
10: macvtap1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 8a:58:b9:cc:28:a9 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    macvtap  mode vepa 

2. start a guest with direct type interface

# virsh dumpxml r7 |grep "interface type" -A4
    <interface type='direct'>
      <mac address='52:54:00:0c:84:4d'/>
      <source dev='eth0' mode='vepa'/>
      <model type='rtl8139'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>

# time virsh start r7 
error: Failed to start domain r7
error: internal error Too many unreserved macvtap devices in use


real	0m0.806s
user	0m0.018s
sys	0m0.019s
Comment 7 yalzhang@redhat.com 2016-03-31 05:59:55 UTC 
Verified the bug on libvirt-0.10.2-60.el6.x86_64 as below, the results is as expected.

1. create a macvtap device named macvtap1
# ip link add link eth0 name macvtap1 type macvtap

# ip -d l show macvtap1
37: macvtap1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether da:51:3a:91:7d:4b brd ff:ff:ff:ff:ff:ff promiscuity 0 
    macvtap  mode vepa 

2. start a guest with direct interface
# virsh dumpxml rh6 | grep /interface -B4  
 <interface type='direct'>
      <mac address='52:54:00:34:e5:52'/>
      <source dev='eth0' mode='vepa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>

# time virsh start rh6
Domain rh6 started


real	0m0.578s
user	0m0.006s
sys	0m0.009s

# ip link show | grep macvtap
# ip link show | grep macvtap
37: macvtap1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
39: macvtap0@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500

One more scenario:
1. create a macvtap device named macvtap0
# ip link add link eth0 name macvtap0 type macvtap
# ip link add link eth2 name macvtap3 type macvtap
# ip -d link show macvtap0
83: macvtap0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether 26:32:c2:53:6f:87 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    macvtap  mode vepa 
# ip -d link show macvtap3
84: macvtap3@eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
    link/ether e6:13:b1:d9:4d:85 brd ff:ff:ff:ff:ff:ff promiscuity 0 
    macvtap  mode vepa 

2. start a guest with 4 direct interfaces, the source dev is eth0 and eth2.
# virsh dumpxml rh6 | grep /interface -B5
    </controller>
    <interface type='direct'>
      <mac address='52:54:00:34:e5:52'/>
      <source dev='eth0' mode='vepa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <interface type='direct'>
      <mac address='52:54:00:c9:29:2b'/>
      <source dev='eth0' mode='vepa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </interface>
    <interface type='direct'>
      <mac address='52:54:00:6e:f4:07'/>
      <source dev='eth2' mode='vepa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0a' function='0x0'/>
    </interface>
    <interface type='direct'>
      <mac address='52:54:00:2d:21:34'/>
      <source dev='eth2' mode='vepa'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </interface>

# time virsh start rh6
Domain rh6 started
real	0m0.920s
user	0m0.007s
sys	0m0.008s

# ip link show | grep macvtap
83: macvtap0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
84: macvtap3@eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
85: macvtap1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500
86: macvtap2@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500
87: macvtap4@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500
88: macvtap5@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500

3. log in the guest to check the network, both 4 interfaces can get ip address. And the guest can ping outside.
Comment 8 yalzhang@redhat.com 2016-03-31 13:24:35 UTC 
Add more scenarios, all the results is as expected on libvirt-0.10.2-60.el6.x86_64.
From the result, we can see that when libvirt generate a macvtap device, it will check the name from macvtap0, if macvtap0 exists, it will check macvtap1 and so on. 

1. Create macvtap1, macvtap3, macvtap5, then start a guest with direct interface, it will generate macvtap0. Start a 2nd guest, it will be macvtap2, the 3rd, macvtap4, the 4th, macvtap6, and so on.

2. Create 100+ macvtap devices, then start a guest with direct interface, check the time, it is OK.
# time virsh start r6new
Domain r6new started


real	0m0.595s
user	0m0.004s
sys	0m0.013s
# ip link show | grep macvtap
21: macvtap1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
22: macvtap3@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500
23: macvtap5@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
31: macvtap0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
....
134: macvtap104@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500 <=== libvirt create the macvtap104 when start the guest.

3. Use macvtap network
# virsh net-dumpxml direct-macvtap
<network>
  <name>direct-macvtap</name>
  <uuid>b40c3968-d0c7-ad7c-c411-a8b63ff44620</uuid>
  <forward dev='eth0' mode='bridge'>
    <interface dev='eth0'/>
    <interface dev='eth2'/>
  </forward>
</network>
# ip link add link eth0 name macvtap0 type macvtap
# ip link add link eth0 name macvtap3 type macvtap
# ip link show | grep macvtap
135: macvtap0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
136: macvtap3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
# virsh edit r6
...
   <interface type='network'>
      <source network='direct-macvtap'/>
    </interface>
...
# time virsh start r6
Domain r6 started


real	0m0.594s
user	0m0.010s
sys	0m0.006s

# ip link show | grep macvtap
135: macvtap0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
136: macvtap3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
141: macvtap1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500
# virsh attach-interface r6 network direct-macvtap
Interface attached successfully
# ip link show | grep macvtap
135: macvtap0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
136: macvtap3@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 500
141: macvtap1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500
142: macvtap2@eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UNKNOWN qlen 500

As the test above, change the states to verified.
Comment 10 errata-xmlrpc 2016-05-10 19:26:37 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0738.html