Bug 2151040

Summary:	Prevent reaching network-online.target/NetworkManager-wait-online while nm-cloud-setup is still working
Product:	Red Hat Enterprise Linux 9	Reporter:	Thomas Haller <thaller>
Component:	NetworkManager	Assignee:	Wen Liang <wenliang>
Status:	VERIFIED ---	QA Contact:	David Jaša <djasa>
Severity:	medium	Docs Contact:	Jaroslav Klech <jklech>
Priority:	medium
Version:	9.1	CC:	bgalvani, cfeist, desktop-qa-list, fdanapfe, ferferna, fpokryvk, jklech, lrintel, nm-team, oalbrigt, rkhan, sfaye, sukulkar, thaller, till, vbenes, wenliang
Target Milestone:	rc	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	NetworkManager-1.43.10-1.el9	Doc Type:	Known Issue
Doc Text:	.The `nm-cloud-setup` service removes manually-configured secondary IP addresses from interfaces Based on the information received from the cloud environment, the `nm-cloud-setup` service configures network interfaces. Disable `nm-cloud-setup` to manually configure interfaces. However, in certain cases, other services on the host can configure interfaces as well. For example, these services could add secondary IP addresses. To avoid that `nm-cloud-setup` removes secondary IP addresses: . Stop and disable the `nm-cloud-setup` service and timer: + ---- # systemctl disable --now nm-cloud-setup.service nm-cloud-setup.timer ---- . Display the available connection profiles: + ---- # nmcli connection show ---- . Reactive the affected connection profiles: + ---- # nmcli connection up <profile_name> ---- As a result, the service no longer removes manually-configured secondary IP addresses from interfaces.	Story Points:	---
Clone Of:	2132754	Environment:
Last Closed:		Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	2132754
Bug Blocks:

Description Thomas Haller 2022-12-05 23:58:42 UTC

+++ This bug was initially created as a clone of Bug #2132754 +++

nm-cloud-setup reconfigures the network. It does so frequently and automatically. It is also enabled by default in RHEL cloud images.

When doing that, it removes IP addresses that are not expected. That causes problems, for services that add IP addresses on the interface (by not using NetworkManager API but netlink/iproute2 directly).

The solution will be two fold:

1) bug 2132754: nm-cloud-setup uses Reapply() D-Bus call. The Reapply() call will get a new flag "preserved-external-ip", so that NetworkManager leaves externally configured IP addresses/routes on the interface alone.

2) this rhbz. When the device/connection activates, it should stay in "activating" state until nm-cloud-setup completed and configured additional addresses. This will also delay "NetworkManager-wait-online.service" and "network-online.target" and allows the user to order themselves after nm-cloud-setup is done.



Note that we could solve the ordering partially by only delaying network-online.target. For that, there could be a service which prevents "network-online.target" to be reached while nm-cloud-setup is still running. However, it would be nicer if you can also see in `nmcli device` that the device is still in "activating" state. We have a mechanism for that, the dispatcher "pre-up" event. We need to install a pre-up script which blocks activation while nm-cloud-setup is running. The pre-up script runs pretty late in the activation, when the interface is otherwise mostly configured.

One problem is that nm-cloud-setup does not run per-interface. It runs and configures all interfaces. If then a pre-up event comes for eth0 and eth1, then they both need to block until nm-cloud-setup is complete. That might be slightly compliated to get right (but maybe not. A `systemctl restart nm-cloud-setup.service` in the pre-up dispatcher script might be sufficient).

Another downside is that eth0 and eth1 both need to wait for nm-cloud-setup to complete. Maybe eth0 could complete faster. But that seems not a big problem.


I think all this needs is a blocking "pre-up" script that calls `systemctl restart nm-cloud-setup.service` and waits for it to complete.

Comment 4 David Jaša 2023-08-01 13:23:24 UTC

VERIFIED with NetworkManager-1.43.11-1.el9.x86_64: an address added via 'ip addr add ...' before nm-cloud setup is run is present afterwards as well.