Bug 1288412
| Summary: | MAC address of VLAN over TEAM interface keeps on changing frequently. | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Shubhrata Priyadarshinee <shubhrata_priyadarsh> | ||||||
| Component: | NetworkManager | Assignee: | Rashid Khan <rkhan> | ||||||
| Status: | CLOSED NOTABUG | QA Contact: | Desktop QE <desktop-qa-list> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | medium | ||||||||
| Version: | 7.2 | CC: | bgalvani, jklimes, linux-bugs, lrintel, rkhan, shubhrata_priyadarsh, sreekanth_reddy, sujith_pandel, tgummels, thaller | ||||||
| Target Milestone: | rc | ||||||||
| Target Release: | 7.3 | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2015-12-07 14:01:55 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Bug Depends On: | |||||||||
| Bug Blocks: | 1304407 | ||||||||
| Attachments: |
|
||||||||
Observations: * Since the switch-ports are vlan-tagged, p1p2 and p2p2 interfaces won't get IP address initially. After teaming them up, the team-master also wont get the IP address. VLAN interface created over the team-master does get the IP address. * NetworkManager changes the state of these 3 interfaces (p1p2, p2p2 and team-master) to failed/disconnected based on the failure to get DHCP response. Later, NM start auto-activating them (state = up) again to repeat the whole cycle again. This seems to be causing the vlan interface also to change the state from up->down continuously. I am not sure whether this is the reason why VLAN interface is changing its MAC address frequently. Can you tell us the reason behind this behavior? (In reply to Sujith from comment #1) > Observations: > * Since the switch-ports are vlan-tagged, p1p2 and p2p2 interfaces won't get Correction in Comment#1:Point#1: It should've been p2p1 and p2p2. Adding to the above observations: * After marking p2p1, p2p2, team-master, team-slaves and vlan interfaces with 'NM_CONTROLLED=no', the issue is not observed i.e vlan mac & ip remains constant. * Similar behavior (as the issue) is seen when a vlan interface is created over a bond-master having these two interfaces (p2p1 and p2p2) as bond-slaves. However the mac address remains constant and hence the vlan interface gets back the original IP address. Log shows that all these interfaces are going up->down and the same cycle repeats. Do we need a separate bug-report to track this behavior? * In both cases (1.vlan on team-master; 2.vlan on bond-master) ping to an external machine is not disrupted. Hi, when a team connection is added with command "nmcli con add type team ifname loms", it gets created with ipv4.method=auto (DHCP) and ipv6.method=auto (RA and possibly DHCPv6). This means that NetworkManager will actively try to get addresses for the team interface and if DHCP/RA fail the connection will be torn down and retried later, as is happening in the log from sosreport: <warn> (loms): DHCPv4 request timed out. <warn> (loms): Activation: failed for connection 'team-loms' and this seems to be the cause of the cycles you are seeing. As explained in [1], if the purpose of the team connection is to provide only L2 connectivity for the VLAN, IP addressing should be disabled on it, i.e.: # nmcli con add type team ifname loms # nmcli con mod team-loms ipv4.method disable ipv6.method ignore Please try if this solves the issue. [1] https://access.redhat.com/solutions/1608803 Hi, After Adding this above command, issue is not observed. Is this a expected behavior from NetworkManager or is a bug? Document you have attached is support for Red Hat Enterprise Linux 7.1 or earlier. Need to add support for 7.2 also? (In reply to Shubhrata Priyadarshinee from comment #5) > Hi, > After Adding this above command, issue is not observed. > > Is this a expected behavior from NetworkManager or is a bug? It is the expected behavior, if no IP configuration is required on the team connection NM must be configured accordingly. > Document you have attached is support for Red Hat Enterprise Linux 7.1 or > earlier. Need to add support for 7.2 also? Yes, I'll ask to update the documentation. As this seems a configuration issue I'm closing the bug, please reopen if necessary. Hi Beniamino, (In reply to Sujith from comment #1) > I am not sure whether this is the reason why VLAN interface is changing its > MAC address frequently. Can you tell us the reason behind this behavior? Why does the MAC address of VLAN change whenever the VLAN interface goes into the cycle of interface state up->down->up.? Regards, Sujith (In reply to Sujith from comment #7) > Hi Beniamino, > (In reply to Sujith from comment #1) > > I am not sure whether this is the reason why VLAN interface is changing its > > MAC address frequently. Can you tell us the reason behind this behavior? > > Why does the MAC address of VLAN change whenever the VLAN interface goes > into the cycle of interface state up->down->up.? Hi Sujith, the correct behavior you should see after configuring connections as described in comment 4 is that the vlan always comes up with the same MAC, which matches the one of the underlying device (bond/team). The MAC of a bond or team in some cases in initially generated randomly by kernel and then updated when new devices are enslaved. In the past a bug existed (see bug 1264322) which caused vlans to come up with the initial random MAC address assigned to bond/team. In order to fix this, NM now tries to keep the MAC of a vlan aligned with the one of the underlying device. This is the reason of the behavior you are seeing. (In reply to Beniamino Galvani from comment #8) > the correct behavior you should see after configuring connections as > described in comment 4 is that the vlan always comes up with the same > MAC, which matches the one of the underlying device (bond/team). > > The MAC of a bond or team in some cases in initially generated > randomly by kernel and then updated when new devices are enslaved. In > the past a bug existed (see bug 1264322) which caused vlans to come > up with the initial random MAC address assigned to bond/team. In order > to fix this, This is the reason of the behavior you > are seeing. Hi Beniamino, I do not have access to BZ#1264322. Can you grant access to me? Disabling the IP works properly. I am just trying to understand what was going wrong leading to this issue. In the case of VLAN over bond-master, the MAC address of VLAN was not changing. It remained constant throughout. Just the interface was shifting between up & down state. > NM now tries to keep the MAC of a vlan aligned with the one of the underlying > device. I don't see this happening here. In the case of VLAN over team-master, the MAC was not inter-changing between the two, it was changing randomly. Hence DHCP-server was out of free-IPs after a time period. Thanks and Regards, Sujith (In reply to Sujith from comment #9) > Hi Beniamino, > I do not have access to BZ#1264322. Can you grant access to me? Sorry, I didn't notice that the bug was not publicly accessible. Unfortunately most of comments are also private so I guess I can't make it public. > Disabling the IP works properly. > I am just trying to understand what was going wrong leading to this issue. > > In the case of VLAN over bond-master, the MAC address of VLAN was not > changing. It remained constant throughout. Just the interface was shifting > between up & down state. Ok, this is expected. > > NM now tries to keep the MAC of a vlan aligned with the one of the underlying > > device. > > I don't see this happening here. In the case of VLAN over team-master, the > MAC was not inter-changing between the two, it was changing randomly. > Hence DHCP-server was out of free-IPs after a time period. Did you see both VLAN and team-master changing the MAC? Or only the VLAN was getting a random one while the one on team-master was stable? I can't reproduce this, do you happen to have NM logs with debug enabled? (To enable debug do 'nmcli general logging level debug' or, to make the change persistent, add "level=DEBUG" to the "[logging]" section of /etc/NetworkManager/NetworkManager.conf and restart NM). Thanks. (In reply to Beniamino Galvani from comment #10) Hi Beniamino, > Did you see both VLAN and team-master changing the MAC? Or only the > VLAN was getting a random one while the one on team-master was stable? It has been a while and I don't remember noting that point on team-master behavior. > I can't reproduce this, do you happen to have NM logs with debug > enabled? (To enable debug do 'nmcli general logging level debug' or, > to make the change persistent, add "level=DEBUG" to the "[logging]" > section of /etc/NetworkManager/NetworkManager.conf and restart NM). Hmm..We had the setup removed after the bug was closed. Give me a few days and I'll try setting it up again to collect logs. Regards, Sujith Created attachment 1116637 [details] NM debug logs (In reply to Beniamino Galvani from comment #10) > I can't reproduce this, do you happen to have NM logs with debug > enabled? (To enable debug do 'nmcli general logging level debug' or, > to make the change persistent, add "level=DEBUG" to the "[logging]" > section of /etc/NetworkManager/NetworkManager.conf and restart NM). Hi Beniamino, I attached the debug messages, where you can find how mac and inet4 address of VLAN over team master interface keeps on changing between the intervals. Thanks Shubhrata |
Created attachment 1102153 [details] sosreport Description of problem: Interfaces are teamed and then vlan is added, the MAC address of vlan keeps on changing and new ip is obtained for vlan interface repeatedly. Version-Release number of selected component (if applicable): NetworkManager-1.0.6-27 kernel:- 3.10.0-327 How reproducible: Always Steps to Reproduce: 1.Setup server with 2 interfaces, both connected to swich ports having vlan tagging enabled. 2. install RHEL-7.2 GA. 3. Create team interface using above two interfaces. Note that Team interface should not get any IP address. #nmcli con add type team ifname loms #nmcli con add type team-slave con-name lom1 ifname p2p1 master loms #nmcli con add type team-slave con-name lom2 ifname p2p2 master loms #nmcli con up lom1 #nmcli con up lom2 #nmcli con up team-loms 4. Create vlan interface over the team interface created in step 3. #nmcli con add type vlan ifname vlan1140 dev loms id 1140 5.Once the vlan connection came up, it starts leasing new ip addresses every few seconds. Actual results: MAC address and IP address of vlan interface keeps on changing frequently. Expected results: MAC address of vlan interface should remain constant. Additional info: - Issue observed in Fedora 23 also. - Issue specific to NetworkManager.