Bug 1135475
| Summary: | some systemd services fail to start due to the livesys script (race condition) | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Kamil Páral <kparal> | ||||||
| Component: | spin-kickstarts | Assignee: | Jeroen van Meeuwen <vanmeeuwen+fedora> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 21 | CC: | admiller, awilliam, bruno, cfergeau, kevin, mschmidt, vanmeeuwen+fedora, zbyszek | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2014-09-02 16:54:12 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
|
Description
Kamil Páral
2014-08-29 12:19:21 UTC
Created attachment 932660 [details]
systemd debug journal from F21 Nightly 2014-08-26 - OK
This is an example of a boot where spice-vdagentd.service starts correctly. You can see the ordering of the service and the debugging printout as described in reproducer step 11.
Created attachment 932661 [details]
systemd debug journal from F21 Nightly 2014-08-27 - FAIL
This is an example of a boot where spice-vdagentd.service doesn't start correctly. You can see the ordering of the service and the debugging printout as described in reproducer step 11-12.
(In reply to Kamil Páral from comment #0) > # make it so that we don't do writing to the overlay for things which > # are just tmpdirs/caches > mount -t tmpfs -o mode=0755 varcacheyum /var/cache/yum > mount -t tmpfs vartmp /var/tmp The mounts could be moved to separate *.mount units, similar to the existing tmp.mount. They could use "ConditionKernelCommandLine=rd.live.image". They would be ordered before local-fs.target, so they wouldn't race with the startup of services. Just a note for those unfamiliar with fedora releng scripts - you can see the livesys script contents here: https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-live-base.ks I can reproduce the bug with Fedora-Live-Workstation-x86_64-21-20140827.iso. Hm, with the recent changes in systemd, this looks quite bad, because 'systemctl status' says the systems is 'degraded', verious serious situation :). I imagine that the race is the following: - systemd creates /var/tmp/systemd-spice-vdagentd.service-XXXXXX/tmp (or var_tmp), - livesys script mounts tmpfs on /var/tmp - systemd tries to use /var/tmp/systemd-spice-vdagentd.service-XXXXXX/tmp and fails The time window is certainly small, but apparently you can hit it. This would also explain why you see a problem with just one service: it is just the one which coincides with the livesys script. The solution proposed my Michal in comment #3 is the right one. -- Alternatively, since /etc/fstab is not copied from the live image to the installed system, those file entries might be added in fstab directly. This will be less work, I imagine. BTW, /etc/fstab could be pruned a bit. $ cat /etc/fstab /dev/root / ext4 defaults,noatime 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 <--- not needed tmpfs /dev/shm tmpfs defaults 0 0 <--- not needed proc /proc proc defaults 0 0 <--- not needed sysfs /sys sysfs defaults 0 0 <--- not needed Thanks Michal and Zbigniew for your help. I tried to apply the following patch to the kickstart: ============================== diff --git a/fedora-live-base.ks b/fedora-live-base.ks index 9929fb7..335b4a2 100644 --- a/fedora-live-base.ks +++ b/fedora-live-base.ks @@ -156,12 +156,6 @@ if ! strstr "\`cat /proc/cmdline\`" nopersistenthome && [ -n "\$homedev" ] ; the action "Mounting persistent /home" mountPersistentHome fi -# make it so that we don't do writing to the overlay for things which -# are just tmpdirs/caches -mount -t tmpfs -o mode=0755 varcacheyum /var/cache/yum -mount -t tmpfs vartmp /var/tmp -[ -x /sbin/restorecon ] && /sbin/restorecon /var/cache/yum /var/tmp >/dev/null 2>&1 - if [ -n "\$configdone" ]; then exit 0 fi @@ -296,6 +290,11 @@ rm -f /core* # convince readahead not to collect # FIXME: for systemd +cat >> /etc/fstab << EOF +vartmp /var/tmp tmpfs defaults 0 0 +varcacheyum /var/cache/yum tmpfs mode=0755 0 0 +EOF + %end ============================== and I built a LiveCD. Everything seems to work correctly. Reassigning to spin-kickstarts, so they can pick up the patch (make it saner, I didn't add any comments or anything, maybe move the code around, it was just a quick hack from me) or use a different approach. Note: When we're adjusting this, it could be a good idea to use the same approach for dnf cache directory: > varcachednf /var/cache/dnf tmpfs mode=0755 0 0 Note2: > BTW, /etc/fstab could be pruned a bit. I think those items are created automatically by livecd-creator. I haven't found it defined in the kickstart anywhere. Thanks, Adam. |