Bug 1932247
| Summary: | Veths attached to nmstate controlled bridge become managed | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Petr Horáček <phoracek> | ||||
| Component: | nmstate | Assignee: | Fernando F. Mancera <ferferna> | ||||
| Status: | CLOSED ERRATA | QA Contact: | Mingyu Shi <mshi> | ||||
| Severity: | urgent | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 8.3 | CC: | fdeutsch, ferferna, fge, jiji, jishi, network-qe, sgordon, thaller, till | ||||
| Target Milestone: | rc | Keywords: | Regression, Triaged, ZStream | ||||
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1932972 (view as bug list) | Environment: | |||||
| Last Closed: | 2021-05-18 15:18:24 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1931376, 1932972, 1936432 | ||||||
| Attachments: |
|
||||||
Note that unlike the udev rule, applying echo "unmanaged-devices=interface-name:veth*" NetworkManager configuration explicitly works around the issue. Making device unmanaged via udev rule (or "device.managed" setting in NetworkManager.conf), can be overruled by the user. On the other hand, "keyfile.unmanaged-devices" in NetworkManager.conf and NM_CONTROLLED=no in ifcfg cannot be overruled. The device is always unmanaged. But the solution here is not to use "keyfile.unmanaged-devices". The solution is to see who tells NetworkManager to use this device (and, if that is undesirable, prevent it from doing that). (In reply to Thomas Haller from comment #2) > Making device unmanaged via udev rule (or "device.managed" setting in > NetworkManager.conf), can be overruled by the user. > > On the other hand, "keyfile.unmanaged-devices" in NetworkManager.conf and > NM_CONTROLLED=no in ifcfg cannot be overruled. The device is always > unmanaged. > > But the solution here is not to use "keyfile.unmanaged-devices". > > The solution is to see who tells NetworkManager to use this device (and, if > that is undesirable, prevent it from doing that). I can confirm that Nmstate is telling NetworkManager to use that device. I am looking how it would be possible to prevent this. (In reply to Fernando F. Mancera from comment #3) > (In reply to Thomas Haller from comment #2) > > Making device unmanaged via udev rule (or "device.managed" setting in > > NetworkManager.conf), can be overruled by the user. > > > > On the other hand, "keyfile.unmanaged-devices" in NetworkManager.conf and > > NM_CONTROLLED=no in ifcfg cannot be overruled. The device is always > > unmanaged. > > > > But the solution here is not to use "keyfile.unmanaged-devices". > > > > The solution is to see who tells NetworkManager to use this device (and, if > > that is undesirable, prevent it from doing that). > > I can confirm that Nmstate is telling NetworkManager to use that device. I > am looking how it would be possible to prevent this. Is the reproducer complete or are there some nmstate commands missing? I mean, if you really want, you can also configure "keyfile.unmanaged-devices" to force the device as being unmanaged. In general that behavior is not very useful, because it means an explicit user's request like `nmcli device set $IFNAME managed yes` and `nmcli device connect $IFNAME` will fail. If the user doesn't want NetworkManager to manage a device, then it suffices to set the udev rule (or "device.managed=0") and simply not tell NetworkManager to manage it. Hence, it would be better to investigate why nmstate wants to manage the device and don't do that. I've found why Nmstate is setting the port as managed. The commit causing this is https://github.com/nmstate/nmstate/pull/1220/commits/85a1c34e72f47cf6fb555ce6d1c71d677133ea98 which solves: https://bugzilla.redhat.com/show_bug.cgi?id=1855189 Reverting this commit will fix this issue but will break the existing one. The solution will be to see if the unmanaged device is up or down before creating the profile, but on nmstate-0.3 that is not possible if the interface is unmanaged because we are using NetworkManager to query the network state. (In reply to Fernando F. Mancera from comment #6) > I've found why Nmstate is setting the port as managed. The commit causing > this is > https://github.com/nmstate/nmstate/pull/1220/commits/ > 85a1c34e72f47cf6fb555ce6d1c71d677133ea98 which solves: > https://bugzilla.redhat.com/show_bug.cgi?id=1855189 > > Reverting this commit will fix this issue but will break the existing one. I mean the one already fixed. > The solution will be to see if the unmanaged device is up or down before > creating the profile, but on nmstate-0.3 that is not possible if the > interface is unmanaged because we are using NetworkManager to query the > network state. Upstream PR: https://github.com/nmstate/nmstate/pull/1529 The patch failed our verification. While it works fine when the bridge has no physical NIC (managed) attached, when the bridge is configured on top of eth0, all veths get disconnected on apply. My original U/S verification tried only the former, but we failed our D/S tests on the latter. RHEL 8.4 patch posted upstream: https://github.com/nmstate/nmstate/pull/1531 Created attachment 1760611 [details]
pre-tested.log
Tested:
nmstate-1.0.3-0.20210302073301710589.pr1531.6.g3e5bfca.el8.noarch
nispor-1.0.1-4.el8.x86_64
NetworkManager-1.30.0-2.el8.x86_64
Verified with versions: nmstate-1.0.2-5.el8.noarch nispor-1.0.1-4.el8.x86_64 NetworkManager-1.30.0-2.el8.x86_64 For bond, vrf and linux-bridge, their unmanaged subordinate ports will keep as unmanged until you explicitly list these ports under 'interfaces' key. Currently team and ovs don't full support if itself or its subordinate port is not managed by NM. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1748 |
Description of problem: Starting with nmstate 0.3, any veth iface attached to a bridge created through nmstate becomes managed when the configuration of the bridge gets reapplied. This happens despite there are udev rules making all veths unmanaged. Due to this, VM's that get attached to a Linux bridge in OpenShift Virtualization get disconnected as soon as nmstate reapplies the config (on configuration change or mere restart of knmstate). This was not the case in nmstate 0.2. Version-Release number of selected component (if applicable): nmstate 0.3.4 NetworkManager 1.26.0 How reproducible: Always. Steps to Reproduce: echo 'interfaces: - bridge: options: stp: enabled: false ipv4: dhcp: false enabled: false ipv6: enabled: false name: upg-br-mark state: up type: linux-bridge ' > br.yaml nmstatectl set br.yaml ip link add veth23da3115 type veth peer name veth_x ip netns add test ip link set veth_x netns test ip link set veth23da3115 master br1 ip link | grep br1 nmcli d | grep veth23da3115 # veth23da3115 ethernet unmanaged -- nmstatectl set br.yaml # With nmstate 0.3 it becomes managed nmcli d | grep veth23da3115 # veth23da3115 ethernet connected veth23da3115 # With nmstate 0.2 it stays unmanaged nmcli d | grep veth23da3115 # veth23da3115 ethernet unmanaged -- Actual results: Veth becomes managed and then can be randomly disconnected by NetworkManager. Expected results: Veths should remain unmanaged. Additional notes: This udev rule seems to be ignored: [root@network06-nwrdz-worker-0-lf2mw core]# cat /usr/lib/udev/rules.d/85-nm-unmanaged.rules ... # Virtual Ethernet device pair. Often used to communicate with a peer interface # in another net namespace and managed by libvirt, Docker or the like. ENV{ID_NET_DRIVER}=="veth", ENV{NM_UNMANAGED}="1" ...