Bug 1867067

Summary: nut-server can fail to start on boot (starts before network is up)
Product: [Fedora] Fedora EPEL Reporter: Tim Jackson <rpm>
Component: nutAssignee: Michal Hlavinka <mhlavink>
Status: NEW --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: epel8CC: alexducast, brainslug, dkaylor, dr, james, kritek, mhlavink, orion
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Tim Jackson 2020-08-07 10:06:17 UTC
Description of problem:
A correctly-configured nut-server can fail to start first time on boot.

Version-Release number of selected component (if applicable):
nut-2.7.4-26.el8.x86_64

How reproducible:
Always (on my system, but may be timing-dependent)


Steps to Reproduce:
1. Configure nut-server, and enable it to start on boot
2. Reboot system and connect via terminal/SSH
3. Check if nut-server is running. Observe that it is not.
4. systemctl restart nut-server
5. Observe that nut-server is now running

Actual results:
nut-server is not running. System log shows something like this:

08:46:10 HOSTNAME systemd[1]: Started Network UPS Tools - power devices information server.
08:46:11 HOSTNAME upsd[1534]: fopen /var/run/nut/upsd.pid: No such file or directory
08:46:11 HOSTNAME upsd[1534]: not listening on [REDACTED_LAN_IP] port 3493
08:46:11 HOSTNAME upsd[1534]: not listening on [REDACTED_LAN_IP] port 3493
08:46:11 HOSTNAME upsd[1534]: listening on 127.0.0.1 port 3493
08:46:11 HOSTNAME upsd[1534]: listening on 127.0.0.1 port 3493
08:46:11 HOSTNAME upsd[1534]: no listening interface available
08:46:11 HOSTNAME upsd[1534]: no listening interface available
08:46:11 HOSTNAME upsd[1534]: Network UPS Tools upsd 2.7.4
08:46:11 HOSTNAME systemd[1]: nut-server.service: Main process exited, code=exited, status=1/FAILURE
08:46:11 HOSTNAME systemd[1]: nut-server.service: Failed with result 'exit-code'.

After "systemctl restart nut-server", the service starts correctly.

Expected results:
nut-server runs on start


Additional info:
It looks like nut-server is starting before the network is ready. The systemd unit file probably needs to include "After=network.target"

Comment 1 Tim Jackson 2020-08-10 19:40:54 UTC
I realised that the unit file already has "After=network.target", so that alone doesn't solve the problem. However, I have more information about this:

- It's apparently triggered by the configuration of an IP address other than 127.0.0.1 ("LISTEN x.y.z.a" configuration option in upsd.conf) and occurs when the x.y.z.a interface is not ready at the time upsd starts up

- The solution is to change "After=network.target" to "After=network-online.target" in nut-server.service
 => According to https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ , this is less desirable than "After=network.target", but as I understand it from that article, the primary solution to avoid it for a server that can be configured to listen on an arbitrary address seems to be for the server to use the IP_FREEBIND socket option (which upsd apparently does not, at least by grepping the code). So, unless and until upsd upstream changes this, waiting for "network-online.target" seems to be the best option.

- Along the way I found that, *if* listening on all interfaces is an acceptable solution, "LISTEN 0.0.0.0" works, and does not suffer from this issue. => Workaround for some use cases

Comment 2 Brainslug 2021-01-12 22:42:25 UTC
I ran into exactly the same issue today and was about to file a bug report when I found this one. 
Thanks to OP for saving me some time writing it up, I hope this will be fixed soon.

Comment 3 Daniel Roesen 2021-03-24 11:47:22 UTC
Same here - same problem, and came to the same conclusion of changing the After to "network-online.target" as the least offending solution.

Comment 4 Rick Dicaire 2024-07-28 18:08:43 UTC
Still an issue with Fedora 40. Including trying this from /usr/lib/systemd/system/nut-server.service:

# The `upsd` is a networked service (even if bound to a `localhost`)
# so it requires that the OS has some notion of networking already.
# Extending the unit does not require *this* file to be edited, you
# can instead drop in an additional piece of configuration, e.g. to
# require that NUT data server only starts after external networking
# is configured (usable IP addresses appear in the system) you can
# add a `/etc/systemd/system/nut-server.service.d/network.conf` with:
#   [Unit]
#   Requires=network-online.target
#   After=network-online.target

Comment 5 Rick Dicaire 2024-07-28 18:37:32 UTC
I should also mention my LISTEN line in /etc/ups/upsd.conf is:

LISTEN 0.0.0.0 3493

Comment 6 Rick Dicaire 2024-07-30 20:32:34 UTC
UPDATE: With help from Nut Mailing List, these are the steps I used to get nut working:

1. Configure the various files in /etc/ups
2. Edit /usr/lib/systemd/system/nut.target and add:
     network.target network-online.target
   to Wants and After lines.
3. Run systemctl daemon-reload.
4. Run systemctl enable nut.target.
5. Run systemctl start nut-driver-enumerator.
6. Reboot. Check that upsmon is talking to upsd. Check from remote host that upsc ups_name@host works.