Bug 1811829 - Dispatcher "up" event happens before IPv6 global link is up, so chrony leaves IPv6 servers offline
Summary: Dispatcher "up" event happens before IPv6 global link is up, so chrony leaves...
Keywords:
Status: NEW
Alias: None
Product: Fedora
Classification: Fedora
Component: NetworkManager
Version: 39
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Lubomir Rintel
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-03-09 21:10 UTC by Ferdinand
Modified: 2024-02-19 08:12 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-30 17:16:09 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Ferdinand 2020-03-09 21:10:52 UTC
Description of problem:
This bug is between NetworkManager and chrony.

On a dual stack IPv4+IPv6 connection, when reconnecting (eg. resuming after standby), the dispatcher "up" event is sent as soon as IPv4 is available. At that time, the IPv6 global link might not yet be available. When chrony listens for dispatcher events, it checks (through its onoffline command) if it should put servers in an on- or offline state.

If the NTP servers configured for chrony are on IPv6, and the "up" event happens before IPv6 is ready, chrony does not reconnect to these servers, and leaves them in the offline state.

This leaves a user with only IPv6 NTP servers configured without an NTP time source, which is rather unexpected.

Version-Release number of selected component (if applicable):
NetworkManager-1.20.10-1.fc31, chrony-3.5-4.fc31

1. Dual stack: network interface reconnects.
2. IPv4 connection ready, nm dispatcher sends "up", before IPv6 is ready.
3. chrony checks connection, doesn't see IPv6 connection.
4. chrony only returns IPv4 servers to online state.


Would it make sense for NetworkManager to send an "up" event, not just for the interface, but for each IPv4/IPv6 link that's up?

Otherwise, can NetworkManager tell if IPv6 dhcp is underway, and delay the "up" event until it completes?

Is there some other elegant solution?

Additional info:
The default Fedora NTP pool 2.fedora.pool.ntp.org returns IPv6 addresses as well, so chrony in the default setting might also be left completely offline after resuming from standby.

Workaround: If IPv6 is always expected to come back shortly after IPv4, it would work to add a short delay in the chrony dispatcher script. But then, how long should this delay be? Could chrony trust the "up" event, and use the explicit "online" command instead? Was this how chrony did it before "onoffline" was added?

Comment 1 Miroslav Lichvar 2020-03-10 07:22:53 UTC
Before the onoffline command (in Fedora 27 and earlier), the chrony dispatcher script used to check for a default IPv4 route. If it was present, it switched all sources (including IPv6) to the online status using the "online" command. If no default IPv4 route was found, each source was switched individually according to a "ip route get $ADDR" command. So I think this issue would show up only on IPv6-only systems.

Comment 2 Miroslav Lichvar 2020-03-17 13:42:56 UTC
I did some experiments in different network configurations. It seems without DHCPv6 the global IPv6 address may not even be configured at the time of the "up" and "connectivity-change" events. With DHCPv6 it seems to be more reliable. The IPv6 address is configured and there is sometimes a "dhcp6-change" event. But the address is not usable yet (tentative state) when the dispatcher script is executed.

I suspect the NM dispatcher system wasn't designed for this use case. Maybe there should be a new event specified for "reachable" network?

Comment 3 Ben Cotton 2020-11-03 16:27:34 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 4 Ben Cotton 2020-11-24 17:40:49 UTC
Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 5 Miroslav Lichvar 2020-11-26 08:35:14 UTC
I still see this on Fedora 33. The dispatcher script is not called when IPv6 is "up" (applications can connect to IPv6 addresses).

Comment 6 Ben Cotton 2021-11-04 16:55:03 UTC
This message is a reminder that Fedora 33 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 33 on 2021-11-30.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '33'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 33 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 7 Ben Cotton 2021-11-30 17:16:09 UTC
Fedora 33 changed to end-of-life (EOL) status on 2021-11-30. Fedora 33 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 8 Ben Cotton 2022-02-08 21:23:58 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 36 development cycle.
Changing version to 36.

Comment 9 Ben Cotton 2023-04-25 16:40:07 UTC
This message is a reminder that Fedora Linux 36 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 36 on 2023-05-16.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '36'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version. Note that the version field may be hidden.
Click the "Show advanced fields" button if you do not see it.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 36 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 10 Fedora Release Engineering 2023-08-16 07:04:32 UTC
This bug appears to have been reported against 'rawhide' during the Fedora Linux 39 development cycle.
Changing version to 39.

Comment 11 peter+fedoraproject.org 2024-02-17 19:04:52 UTC
This bug is still current in Fedora 39.

My network is IPv6 only, with NAT64 & DNS64 to reach legacy IPv4-only servers. All the NTP servers therefore have IPv6 addresses - any IPv4 addresses provided by the pool are seen locally as IPv6 addresses prefixed by 64:ff9b::.

chronyd works fine at startup, but on resuming after standby it no longer sees the IPv6 servers and loses synchronisation.

The original poster presents this as a bug between Network Manager and chronyd. I argue that both should work well together - or separately - on an IPv6-only system.

Network Manager should prompt any service it is responsible for (not just chronyd) when IPv6 is *useable* after standby (it is a network manager, after all). Comment 2 above asks whether NM has an appropriate event defined: well, if not, then obviously it should provide one; if NM *does* provide such an event, Fedora should use it with chronyd.

Equally, chronyd should not need to be prompted by a network manager. It should take appropriate action, maybe re-interrogating the pool for an up-to-date server list when it notices a large time gap after standby (it is a time daemon, after all). Whatever the action on resume, it should be resilient to network changes.

Whether it's a Fedora matter or requires upstream work, could Fedora/Red Hat address this issue, now 4 years old, and make sure that any fix is tested on BOTH dual-stack AND IPv6-only systems. Thanks.

Comment 12 peter+fedoraproject.org 2024-02-17 19:08:40 UTC
And can we make this at least medium severity?

It's non-critical loss of function as far as I'm concerned.

Comment 13 Beniamino Galvani 2024-02-19 08:12:13 UTC
Please report the bug upstream: https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues


Note You need to log in before you can comment on or make changes to this bug.