Bug 1072284 - beah runs tasks while systemd-readahead is still collecting readahead information
Summary: beah runs tasks while systemd-readahead is still collecting readahead informa...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Beaker
Classification: Community
Component: beah
Version: 0.15
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified vote
Target Milestone: 0.16.2
Assignee: Amit Saha
QA Contact: tools-bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-03-04 09:57 UTC by Alois Mahdal
Modified: 2018-02-06 00:41 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-04-16 07:12:41 UTC


Attachments (Terms of Use)

Description Alois Mahdal 2014-03-04 09:57:36 UTC
Before first test is launched, system is expected to be in a known stable state.  However, with recently launched test on RHEL7, it turns out that systemd-readahead-collect.service was running at the moment when first test started.

Purpose of this service is to collect data (FS access patterns) during boot which is then to be used by systemd-readahead-replay to pre-load often accessed files.  This, the service  is expected to run only during boot time.

This was found on RHEL7 machines when running /distribution/test_inspector with a new test[1].  2/3 receipts generated (supposedly) false positives due to this service running during first of tests but not during the last one.

  [1]: https://beaker.engineering.redhat.com/jobs/604206

~

From the service manpage[2]:

> systemd-readahead-collect.service is a service that
> collects disk usage patterns at boot time.
> systemd-readahead-replay.service is a service that
> replays this access data collected at the subsequent
> boot.  Since disks tend to be magnitudes slower than
> RAM, this is intended to improve boot speeds by
> pre-loading early at boot all data on disk that
> is known to be read for the complete boot process.
>
> systemd-readahead-done.service is executed a short
> while after boot completed and signals
> systemd-readahead-collect.service to end data
> collection. On this signal, this service will then
> sort the collected disk accesses and store information
> about them in /.readahead.

  [2]: http://www.freedesktop.org/software/systemd/man/systemd-readahead-replay.service.html

Comment 2 Dan Callaghan 2014-03-04 22:54:18 UTC
We should be able to add After=systemd-readahead-done.service to the beah.service unit. I guess that would be reasonable, since beah is not really a normal service (it starts doing a whole bunch of work immediately on boot, but it's not actually part of the boot process).

Comment 3 Amit Saha 2014-03-12 05:02:30 UTC
http://gerrit.beaker-project.org/#/c/2908/(In reply to Dan Callaghan from comment #2)
> We should be able to add After=systemd-readahead-done.service to the
> beah.service unit. I guess that would be reasonable, since beah is not
> really a normal service (it starts doing a whole bunch of work immediately
> on boot, but it's not actually part of the boot process).

http://gerrit.beaker-project.org/#/c/2908/

Comment 4 Dan Callaghan 2014-03-13 01:44:12 UTC
(In reply to Dan Callaghan from comment #2)

On second thought, I don't think this will work as is. It introduces a dependency cycle in the ordering: default.target -> beah-*.service -> systemd-readahead-done.service -> systemd-readahead-done.timer -> default.target.

One possibility would be to just make beah do nothing for 30 seconds after it's started, to give readahead collection time to stop.

Another possibility would be to introduce some new target (default-with-beah.target?) which depends on default.target, make beah-*.service WantedBy that new target, and change the system boot target to be default-with-beah.target instead of default.target. Then we can add After=systemd-readahead-done.service and it won't create a dependency cycle. The downside is we have to fiddle with the system boot target which is not very nice.

Any other ideas?

Comment 5 Dan Callaghan 2014-03-13 07:16:34 UTC
Alternatively, we could just disable readahead collection for Beaker recipes.

We already do this on RHEL6 because of bad interactions between readahead, auditd, and beah.

And readahead collection really doesn't make much sense consdering that a typical recipe is installed once, boots up, runs tasks, and then finishes. The readahead replay would never even be used in that case.

Comment 6 Dan Callaghan 2014-03-23 23:45:55 UTC
We haven't heard any objections about just disabling systemd-readahead so let's go ahead and do that.

We need to make sure there's a ks_meta to opt out of the disabling, and that it's consistent with how the RHEL6 readahead disabling works.

Comment 7 Amit Saha 2014-03-25 03:49:07 UTC
It turns out that it is indeed possible to achieve what we originally wanted here without resorting to an alternative boot target.

So, I  have uploaded a fresh patch for Beah's systemd unit files to that effect: 

Last patch here: http://gerrit.beaker-project.org/#/c/2908/

Lennart Pottering's reply to my query on systemd mailing list explaining how this is possible:

http://lists.freedesktop.org/archives/systemd-devel/2014-March/018162.html

This change should start the beah services after the readahead collection has completed.

Comment 8 Amit Saha 2014-03-28 06:27:29 UTC
Tip for verification:

* Provision a job with RHEL 7 / Fedora 18+
* Login to the system (by running a reservesys/ task, for e.g.) and check that the beah services (beah-srv, beah-beaker-backend and beah-fwd-backend) started *after* the systemd-readahead-collect service exited.

Relevant command:

$ systemctl status beah-srv beah-beaker-backend beah-fwd-backend systemd-readahead-collect

Comment 11 Dan Callaghan 2014-04-16 07:12:41 UTC
Beah 0.7.4 has been released.

Comment 12 Dan Callaghan 2014-09-30 06:23:54 UTC
This has not always worked properly and has caused us various problems since it was implemented (the latest is bug 1147807), so we are going to revert the dependency ordering hack in beah. Beaker will just disable systemd-readahead the same as it disables the readahead service on RHEL6.


Note You need to log in before you can comment on or make changes to this bug.