Observations:
* Since the switch-ports are vlan-tagged, p1p2 and p2p2 interfaces won't get IP address initially. After teaming them up, the team-master also wont get the IP address. VLAN interface created over the team-master does get the IP address.

* NetworkManager changes the state of these 3 interfaces (p1p2, p2p2 and team-master) to failed/disconnected based on the failure to get DHCP response. Later, NM start auto-activating them (state = up) again to repeat the whole cycle again.
This seems to be causing the vlan interface also to change the state from up->down continuously.

I am not sure whether this is the reason why VLAN interface is changing its MAC address frequently. Can you tell us the reason behind this behavior?

(In reply to Sujith from comment #1)
> Observations:
> * Since the switch-ports are vlan-tagged, p1p2 and p2p2 interfaces won't get

Correction in Comment#1:Point#1:
It should've been p2p1 and p2p2.


Adding to the above observations:
* After marking p2p1, p2p2, team-master, team-slaves and vlan interfaces with 'NM_CONTROLLED=no', the issue is not observed i.e vlan mac & ip remains constant.

* Similar behavior (as the issue) is seen when a vlan interface is created over a bond-master having these two interfaces (p2p1 and p2p2) as bond-slaves. However the mac address remains constant and hence the vlan interface gets back the original IP address. 
Log shows that all these interfaces are going up->down and the same cycle repeats.

Do we need a separate bug-report to track this behavior?

* In both cases (1.vlan on team-master; 2.vlan on bond-master) ping to an external machine is not disrupted.

Hi,

when a team connection is added with command "nmcli con add type team
ifname loms", it gets created with ipv4.method=auto (DHCP) and
ipv6.method=auto (RA and possibly DHCPv6). This means that
NetworkManager will actively try to get addresses for the team
interface and if DHCP/RA fail the connection will be torn down and
retried later, as is happening in the log from sosreport:

 <warn>  (loms): DHCPv4 request timed out.
 <warn>  (loms): Activation: failed for connection 'team-loms'

and this seems to be the cause of the cycles you are seeing. As
explained in [1], if the purpose of the team connection is to provide
only L2 connectivity for the VLAN, IP addressing should be disabled on
it, i.e.:

 # nmcli con add type team ifname loms
 # nmcli con mod team-loms ipv4.method disable ipv6.method ignore

Please try if this solves the issue.

[1] https://access.redhat.com/solutions/1608803

Hi,
After Adding this above command, issue is not observed. 

Is this a expected behavior from NetworkManager or is a bug?

Document you have attached is support for Red Hat Enterprise Linux 7.1 or earlier. Need to add support for 7.2 also?

(In reply to Shubhrata Priyadarshinee from comment #5)
> Hi,
> After Adding this above command, issue is not observed. 
> 
> Is this a expected behavior from NetworkManager or is a bug?

It is the expected behavior, if no IP configuration is required on the team connection NM must be configured accordingly.

> Document you have attached is support for Red Hat Enterprise Linux 7.1 or
> earlier. Need to add support for 7.2 also?

Yes, I'll ask to update the documentation.

As this seems a configuration issue I'm closing the bug, please reopen if necessary.

Hi Beniamino,
(In reply to Sujith from comment #1)
> I am not sure whether this is the reason why VLAN interface is changing its
> MAC address frequently. Can you tell us the reason behind this behavior?

Why does the MAC address of VLAN change whenever the VLAN interface goes into the cycle of interface state up->down->up.?

Regards,
Sujith

(In reply to Sujith from comment #7)
> Hi Beniamino,
> (In reply to Sujith from comment #1)
> > I am not sure whether this is the reason why VLAN interface is changing its
> > MAC address frequently. Can you tell us the reason behind this behavior?
> 
> Why does the MAC address of VLAN change whenever the VLAN interface goes
> into the cycle of interface state up->down->up.?

Hi Sujith,

the correct behavior you should see after configuring connections as
described in comment 4 is that the vlan always comes up with the same
MAC, which matches the one of the underlying device (bond/team).

The MAC of a bond or team in some cases in initially generated
randomly by kernel and then updated when new devices are enslaved.  In
the past a bug existed (see bug 1264322) which caused vlans to come
up with the initial random MAC address assigned to bond/team.  In order
to fix this, NM now tries to keep the MAC of a vlan aligned with the
one of the underlying device.  This is the reason of the behavior you
are seeing.

(In reply to Beniamino Galvani from comment #8)
> the correct behavior you should see after configuring connections as
> described in comment 4 is that the vlan always comes up with the same
> MAC, which matches the one of the underlying device (bond/team).
> 
> The MAC of a bond or team in some cases in initially generated
> randomly by kernel and then updated when new devices are enslaved.  In
> the past a bug existed (see bug 1264322) which caused vlans to come
> up with the initial random MAC address assigned to bond/team.  In order
> to fix this,   This is the reason of the behavior you
> are seeing.

Hi Beniamino,
I do not have access to BZ#1264322. Can you grant access to me?
Disabling the IP works properly.
I am just trying to understand what was going wrong leading to this issue.

In the case of VLAN over bond-master, the MAC address of VLAN was not changing. It remained constant throughout. Just the interface was shifting between up & down state.

> NM now tries to keep the MAC of a vlan aligned with the one of the underlying 
> device.

I don't see this happening here. In the case of VLAN over team-master, the MAC was not inter-changing between the two, it was changing randomly. 
Hence DHCP-server was out of free-IPs after a time period.

Thanks and Regards,
Sujith

(In reply to Sujith from comment #9)
> Hi Beniamino,
> I do not have access to BZ#1264322. Can you grant access to me?

Sorry, I didn't notice that the bug was not publicly accessible.
Unfortunately most of comments are also private so I guess I can't
make it public.

> Disabling the IP works properly.
> I am just trying to understand what was going wrong leading to this issue.
>
> In the case of VLAN over bond-master, the MAC address of VLAN was not
> changing. It remained constant throughout. Just the interface was shifting
> between up & down state.

Ok, this is expected.

> > NM now tries to keep the MAC of a vlan aligned with the one of the underlying
> > device.
>
> I don't see this happening here. In the case of VLAN over team-master, the
> MAC was not inter-changing between the two, it was changing randomly.
> Hence DHCP-server was out of free-IPs after a time period.

Did you see both VLAN and team-master changing the MAC? Or only the
VLAN was getting a random one while the one on team-master was stable?

I can't reproduce this, do you happen to have NM logs with debug
enabled? (To enable debug do 'nmcli general logging level debug' or,
to make the change persistent, add "level=DEBUG" to the "[logging]"
section of /etc/NetworkManager/NetworkManager.conf and restart NM).

Thanks.

(In reply to Beniamino Galvani from comment #10)

Hi Beniamino,

> Did you see both VLAN and team-master changing the MAC? Or only the
> VLAN was getting a random one while the one on team-master was stable?

It has been a while and I don't remember noting that point on team-master behavior.

> I can't reproduce this, do you happen to have NM logs with debug
> enabled? (To enable debug do 'nmcli general logging level debug' or,
> to make the change persistent, add "level=DEBUG" to the "[logging]"
> section of /etc/NetworkManager/NetworkManager.conf and restart NM).

Hmm..We had the setup removed after the bug was closed.
Give me a few days and I'll try setting it up again to collect logs.

Regards,
Sujith

Created attachment 1116637 [details]
NM debug logs

(In reply to Beniamino Galvani from comment #10)
> I can't reproduce this, do you happen to have NM logs with debug
> enabled? (To enable debug do 'nmcli general logging level debug' or,
> to make the change persistent, add "level=DEBUG" to the "[logging]"
> section of /etc/NetworkManager/NetworkManager.conf and restart NM).

Hi Beniamino,
I attached the debug messages, where you can find how mac and inet4 address of VLAN over team master interface keeps on changing between the intervals.


Thanks
Shubhrata