Hide Forgot
on AWS images, there is nm-cloud-setup running that reconfigures the network. Does systemctl disable nm-cloud-setup.service systemctl disable nm-cloud-setup.timer help?
The above suggestion to disable nm-cloud-setup is working. I am posting my replication results in RHEL 8.4 AMI here. I launched an instance with RHEL 8.4 AMI with 2 network interfaces attached # ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000 link/ether 0a:6b:6a:73:8e:0c brd ff:ff:ff:ff:ff:ff inet 172.31.45.194/20 brd 172.31.47.255 scope global noprefixroute eth0 valid_lft forever preferred_lft forever 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc fq_codel state UP group default qlen 1000 link/ether 0a:49:e2:8c:9f:b2 brd ff:ff:ff:ff:ff:ff inet 172.31.32.249/20 brd 172.31.47.255 scope global noprefixroute eth1 valid_lft forever preferred_lft forever inet6 fe80::4787:47f3:14c0:4d20/64 scope link noprefixroute valid_lft forever preferred_lft forever # ip r default via 172.31.32.1 dev eth0 proto dhcp metric 100 default via 172.31.32.1 dev eth1 proto dhcp metric 101 172.31.32.0/20 dev eth0 proto kernel scope link src 172.31.45.194 metric 100 172.31.32.0/20 dev eth1 proto kernel scope link src 172.31.32.249 metric 101 # nmcli connection show NAME UUID TYPE DEVICE System eth0 5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 ethernet eth0 Wired connection 1 2736fb27-69b3-3307-8e57-f0fef05a7638 ethernet eth1 ens3 02cc87e6-dc7c-4c9e-a530-f908611c7f5e ethernet -- # sudo systemctl disable nm-cloud-setup.service Removed /etc/systemd/system/NetworkManager.service.wants/nm-cloud-setup.service. # sudo systemctl disable nm-cloud-setup.timer Removed /etc/systemd/system/timers.target.wants/nm-cloud-setup.timer. # nmcli connection modify "Wired connection 1" +ipv4.routes "10.20.38.0/24 172.31.32.1" # nmcli connection up "Wired connection 1" Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/3) # ip r default via 172.31.32.1 dev eth0 proto dhcp metric 100 default via 172.31.32.1 dev eth1 proto dhcp metric 101 10.20.38.0/24 via 172.31.32.1 dev eth1 proto static metric 101 <---- The Route is successfully added 172.31.32.0/20 dev eth0 proto kernel scope link src 172.31.45.194 metric 100 172.31.32.0/20 dev eth1 proto kernel scope link src 172.31.32.249 metric 101
> However, I wonder whether both the services were introduced in RHEL8.4 AMIs since customer did not face the issue in RHEL8.3 AMI. Yes, this is new in RHEL-8.4 AMI. This happens here: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/blob/654bb92a6968d9cee03b0c70d7797b057734e2a3/src/nm-cloud-setup/main.c#L349 I guess, nm-cloud-setup should merge the routes/addresses/rules from the profile too, before adding the automatically added ones. That would make sense.
Possible fix: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/893
nm-cloud-setup is supposed to configure networking in a cloud environment automatically. When a user has special requirements (like configuing manual routes), then they are already in a position where they explicitly reconfigure the networking. In such a case it is completely OK to disable nm-cloud-setup, if the provided automatism doesn't do what the user wants. Still, - The out of the box behavior should be better, which I think https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/893 should improve. This change probably will land in rhel-8.6, but likely won't land in rhel-8.5 (do 8.5 Z stream releases result in new AMI images to be generated?). Anyway, while the patch probably would help with this particular case, it's not possible that the automatic behavior is optimal for everybody. I am not familiar how cloud images are provided to AWS. Could you please ask your question to the cloud-team? - I am not aware which KB articles exist (if any). In general, this is something that should be well documented. It should also be documented that it is fully OK to disable nm-cloud-setup and configure whatever special requirements the user has. I wouldn't call that a "workaround". `man nm-cloud-setup` ([1]) should also explain it, but if something is unclear there, we'd like to improve the manual. [1] https://networkmanager.pages.freedesktop.org/NetworkManager/NetworkManager/nm-cloud-setup.html
> This change probably will land in rhel-8.6, but likely won't land in rhel-8.5 (do 8.5 Z stream releases result in new AMI images to be generated?). Sorry, I confused the release numbers :( https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/893 likely will make it into rhel-8.5, but not 8.4 (or 8.4.z).
fixed on upstream nm-1-32 branch (https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/893).
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: NetworkManager security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:4361