Bug 1188664

After some debugging, the issue is that beah starts before network is up, and before /etc/resolv.conf contains nameservers. This leads to beah resolving from localhost DNS server (which is not running).

The same issue applies to grosse harness (both are written in python using same resolving code).

Looking to: http://www.freedesktop.org/software/systemd/man/systemd.unit.html
There should be probably both After= and Requires= (or Wants=) in unit files.

I've checked if NetworkManager-wait-online.service is enabled, and it's not, so we're facing race condition here and reason why it can be seen on arm is because the system is slow :) On another arm system, I'm not seeing this issue.

# systemctl is-enabled NetworkManager-wait-online.service
disabled

# systemctl status NetworkManager-wait-online.service 
NetworkManager-wait-online.service - Network Manager Wait Online
   Loaded: loaded (/usr/lib/systemd/system/NetworkManager-wait-online.service; disabled)
   Active: inactive (dead)

Content of current beah-beaker-backend.service file:
[Unit]
Description=The Beaker backend server.
After=network.target NetworkManager-wait-online.service time-sync.target

[Service]
Type=simple
ExecStart=/usr/bin/beah-beaker-backend

[Install]
WantedBy=multi-user.target

Workaround for beah is to put following lines to kickstart:
%post
systemctl enable NetworkManager-wait-online.service
%end

Thanks for the explanation Pavel - we'll aim to get this into the next harness release.

We can't just unconditionally require and/or enable NetworkManager-wait-online.service because the recipe might not be using NetworkManager...

The real fix is to make beah use a sane resolver (i.e. the glibc one) that actually obeys changes in /etc/resolv.conf instead of Twisted's flawed reimplementation. Sigh.

Just to make decision easier. NM is default in RHEL-7, it's even in core group so test that doesn't want to use NM needs to already take extra steps disabling it (either masking it or disabling it in ifcfg script), so additional extra step in masking NetworkManager-wait-online.service in such tests shouln't be problem.

I'm aware that this may break some tests, but noting this change in some release notes should be ok.

After reading http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ I think the right answer is actually for the beah services to have:

After=network-online.target
Wants=network-online.target

That way it is independent of the network management daemon, *and* it doesn't impact any other services on the system (which enabling NetworkManager-wait-online.service would).

http://gerrit.beaker-project.org/4301

Test builds are here:
http://galangal.usersys.redhat.com/~dcallagh/bz1188664/

To try it in your RHEL7 recipes:
<repos>
  <repo name="beaker-harness-bz1188664" url="http://galangal.usersys.redhat.com/~dcallagh/bz1188664/" />
</repos>

Beaker 21.0 has been released.