Bug 2179537

Summary: NetworkManager doesn't run the dispatcher scripts when the NTP server changes after a DHCP renewal
Product: Red Hat Enterprise Linux 9 Reporter: Juanma Sanchez <juasanch>
Component: NetworkManagerAssignee: Beniamino Galvani <bgalvani>
Status: VERIFIED --- QA Contact: Filip Pokryvka <fpokryvk>
Severity: medium Docs Contact:
Priority: low    
Version: 9.1CC: bgalvani, fpokryvk, justinjereza, lrintel, rkhan, sfaye, sukulkar, till, vbenes
Target Milestone: rcKeywords: Triaged
Target Release: 9.3   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: NetworkManager-1.43.6-1.el9 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Juanma Sanchez 2023-03-18 11:49:58 UTC
Description of problem:

  When the DHCP lease is renewed and there's a change in the NTP server, the
  dispatcher scripts are not run and therefore chrony is not updated
  to use the new NTP server.

  The only workaround is to bring down/up the connection via nmcli. After
  the connection is brought back up, chrony shows the updated NTP server.


Version-Release number of selected component (if applicable):              
                                                                           
  This has been tested on NetworkManager-1.36.0-7.el8_6 and                
  NetworkManager-1.40.0-5.el8_7, both behave as described above.

  When testing the same on NetworkManager-1.32.10-5.el8_5, a "dhcp4-change"
  event is triggered when the NTP-server option changes, the dispatcher    
  scripts are run, and chrony is updated successfuly with the new NTP server.

  Looks like a problem introduced in RHEL8.6 after NM was updated from 1.32 to 1.36


How reproducible:                                                          

  Always


Steps to Reproduce:                                                        

  * create a pair of veth, add a new ns for the peer, configure an IP on the
    peer, and build a DHCP NM connection on the client

    $ sudo ip link add dhcpifc type veth peer name dhcpifd
    $ sudo ip netns add dhcps
    $ sudo ip link set dhcpifd netns dhcps

    $ sudo nmcli con add con-name dhcpifc type ethernet \
      ifname dhcpifc ipv6.method disabled ipv4.method auto \
      ipv4.ignore-auto-routes yes

    $ sudo ip netns exec dhcps ip add add 10.9.9.1/24 dev dhcpifd
    $ sudo ip netns exec dhcps ip link set dhcpifd up


  * Run a simple dnsmasq dhcp server with a short lease time on the peer interface:

      $ sudo ip netns exec dhcps dnsmasq -d -C- <<EOF
      interface=dhcpifd
      port=0
      dhcp-range=10.9.9.100,10.9.9.110,255.255.255.0,60s
      dhcp-option=option:ntp-server,10.9.9.12
      EOF


  * Bring the DHCP connection up. Wait until the DHCP settings are configured

    $ sudo nmcli con dhcpifc up


  * Check the chrony sources. The NTP server provided initially is configured

    $ sudo chronyc -n sources | grep 10.9
    ^? 10.9.9.12                     0   7     0     -     +0ns[   +0ns] +/-    0ns


  * Before the DHCP is renewed, modify the NTP server option and re-run dnsmasq

      $ sudo ip netns exec dhcps dnsmasq -d -C- <<EOF
      interface=dhcpifd
      port=0
      dhcp-range=10.9.9.100,10.9.9.110,255.255.255.0,60s
      dhcp-option=option:ntp-server,10.9.9.10
      EOF


  * After the renewal, re-check the chrony sources. The new NTP server has not been configured

    $ sudo chronyc -n sources | grep 10.9
    ^? 10.9.9.12                     0   8     0     -     +0ns[   +0ns] +/-    0ns

Actual results:

  Chrony is not updated to use new NTP server provided

Expected results:

  Chrony should be updated to use new NTP server provided

Additional info:

  This is easily reproducible on RHEL8.6+ and RHEL9 using the mentioned NetworkManager versions.
  If there's log/trace required please let me know and I'll upload it.

Comment 11 Justin Jereza 2023-05-23 02:35:08 UTC
This affects me as well by not running scripts in `/etc/NetworkManager/dispatcher.d/` on every `dhcp4-change` event. My use case is removing classless static routes issued by DHCPv4 since NetworkManager doesn't seem to have a setting for retaining the default route but ignoring classless static routes.

I've put the following script in that directory on both RHEL 9 and Fedora 38:

```bash
#!/usr/bin/sh

export LC_ALL=C

interface=$1
action=$2

echo "[ $(date '+%F %T.%N') ] NM-dispatch: ($interface) $action" >> /tmp/nm-dispatch.log

exit 0
```

Every `dhcp4-change` event logged in `journalctl --unit=NetworkManager.service` has a corresponding entry in `/tmp/nm-dispatch.log` on Fedora 38 whereas it only appears on RHEL 9 when interfaces go up and down.