Bug 223722

Summary: RFE: add functionality to persist temporary state back to original location
Product: [Fedora] Fedora Reporter: Ville Skyttä <ville.skytta>
Component: initscriptsAssignee: Lukáš Nykrýn <lnykryn>
Status: CLOSED CANTFIX QA Contact: Brock Organ <borgan>
Severity: medium Docs Contact:
Priority: medium    
Version: rawhideCC: fedora-jasper, gordan, herrold, initscripts-maint-list
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-05-16 07:43:41 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Persist temporary state back to disk on shutdown/reboot
none
/etc/rc.d/rc.sysinit patch
none
/etc/init.d/halt patch
none
/etc/sysconfig/savestruct/exclude.list example
none
/usr/local/sbin/savestruct none

Description Ville Skyttä 2007-01-21 21:38:53 UTC
The temporary state stuff in initscripts can currently be easily used to mount
things that need to be written to to a tmpfs.  It would be even better if there
was a clean way of getting it dropped back to its original persistent location
on shutdown.  This would help in creating standalone configurations where system
write activity is not flushed to disk all the time and thus make it easier for
example to spin down the harddisk, but still having those writes to not go to
/dev/null on shutdown.

I'm currently using the attached script as /etc/rc[06].d/S00perist-stateless and
READONLY=no TEMPORARY_STATE=yes in /etc/sysconfig/readonly-root on FC6 with
great success.  However it would be nice and probably result in less code
duplication and fragility if this functionality could be included in initscripts
out of the box, for example, by having the temporary state
mounting/unmounting+persisting moved from rc.sysinit to /sbin/temporary-state
which could be called like "/sbin/temporary-state setup" to set up the temporary
state area in tmpfs, and "/sbin/temporary-state persist" to unmount the tmpfs's
and copy files there to their original fs location.

Comment 1 Ville Skyttä 2007-01-21 21:38:54 UTC
Created attachment 146146 [details]
Persist temporary state back to disk on shutdown/reboot

Comment 2 Jon Stanley 2008-04-23 20:30:32 UTC
Adding FutureFeature keyword to RFE's.

Comment 3 LarryA 2009-04-16 14:22:04 UTC
This is becoming more important to include as a feature because Solid State disk drives are becoming more affordable from multiple vendors, installed memory can be large, and availability of newer portable machines that have no spinning-disk storage.  Supply side changes will accommodate creative and complex uses of tempfs.

Comment 4 Gordan Bobic 2009-06-10 11:34:21 UTC
I think running this as an init script isn't quite low enough a level - it won't kick in early enough. Here is a slightly different approach.

We put write-heavy yet storage-light areas that don't need to be persisted on tmpfs. Prime candidates I can think of for this approach are:
/tmp
/var/tmp
/var/run
/var/lock
/var/log (a bit extreme, but on a laptop you probably don't need them and on a server you can syslog them to a logging server with a mechanical disk).

The list is, of course, non-exhaustive and the functionality should be flexible enough to be applied automagically to any tmpfs mounted subtree.

The key problem here is that /var/lock, /var/run, and /var/log have a directory sub-structure that doesn't get re-created at startup time. This structure is required for correct operation of these directories.
/tmp and /var/tmp should never need to be persisted at all, so this doesn't apply to them.

So, something that saves the directory structure (and only directory structure, without including files) is required, which will check the current structure in tmpfs and save it at shutdown only if it has changed, and something that restores that structure at startup once the tmpfs has been mounted to those directories.

The saving needs to be performed in /etc/init.d/halt before the file systems are umounted, and restore needs to be performed in /etc/rc.d/rc.sysinit as soon as the rootfs is mounted rw. /tmp and /var/tmp should never be saved.

Some directories should be settable to never be saved from a subtree (e.g. /var/run/xdmctl/dmctl).

Attached are the following:
/etc/rc.d/rc.sysinit.patch
/etc/init.d/halt.patch
/usr/local/sbin/savestruct
/etc/sysconfig/savestruct/exclude.list

savestruct saves a .tar.bz2 file containing the directory structure of a subtree into /etc/sysconfig/savestruct/path.tar.bz2, e.g.

# savestruct /var/lock

will created a /etc/sysconfig/savestruct/_var_lock.tar.bz2. This is called in /etc/init.d/halt for every tmpfs mount except proc, sys, dev (something like this already happens in there by default, see the patch for details).

savestruct will refuse to save /tmp and /var/tmp structures.

This .tar.bz2 will in turn get restored in rc.sysinit.

Note: savestruct should be called for each directory we want to move to tmpfs before tmpfs mounting it and/or adding it to /etc/fstab. Once all the directories are saved, edit /etc/fstab to mount tmpfs to the required paths, and simply reboot. From then on, everything will happen automatically with some minor reporting going on during startup/shutdown in line with what is already there.

Comment 5 Gordan Bobic 2009-06-10 11:36:09 UTC
Created attachment 347205 [details]
/etc/rc.d/rc.sysinit patch

Add restoring saved directory structures (if they exist) to tmpfs mounted paths.

Comment 6 Gordan Bobic 2009-06-10 11:37:27 UTC
Created attachment 347206 [details]
/etc/init.d/halt patch

Add directory structure saving for tmpfs mounted paths.

Comment 7 Gordan Bobic 2009-06-10 11:39:00 UTC
Created attachment 347207 [details]
/etc/sysconfig/savestruct/exclude.list example

Example of the exclude file. The strings listed will be grepped out of the list of directories to be saved.

Comment 8 Gordan Bobic 2009-06-10 11:43:26 UTC
Created attachment 347208 [details]
/usr/local/sbin/savestruct

Script to save a directory-only substructure of a path. Called from /etc/init.d/halt. Currently referenced from /usr/local/sbin since it's a prototype, should be moved to /usr/sbin if the method is accepted. It saves the structure as a .tar.bz2 in /etc/sysconfig/savestruct/, and is fairly generic.

It will _refuse_ to save /tmp and /var/tmp.

It will NOT save a new .tar.bz2 file if the stored path structure hasn't changed, in order to reduce the number of writes as much as possible. It uses /tmp for 2 temporary files, but this is on the premise that /tmp is mounted on tmpfs so those writes won't affect SSD life.

Comment 9 Gordan Bobic 2009-06-10 11:50:01 UTC
By my count, about 70 files in total get created in the following directories at startup on an average laptop:

/tmp
/var/tmp
/var/lock
/var/log
/var/run

That means at least 70 file writes + 70 directory writes = 140 writes - minimum.

This seems like a worthwhile saving to me. If each of those went to the same block, that would mean that block would only last about 70 reboots on a typical 10,000 erase cycle flash if there were no wear leveling implemented. My proposed patches get this down to effectively 0.

Comment 10 Gordan Bobic 2009-06-10 11:51:35 UTC
Forgot to mention - my patches are against RHEL5.3 init scripts as provided by package initscripts-8.45.25-1.

Comment 11 Gordan Bobic 2010-06-08 14:29:59 UTC
Just thought I'd mention that the patches I attached previously written for RHEL 5.3 apply cleanly for me on RHEL 5.4, RHEL 5.5 and RHEL6 beta and still work as expected.

Comment 12 Fedora Admin XMLRPC Client 2013-09-04 14:50:25 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 13 Lukáš Nykrýn 2014-05-16 07:43:41 UTC
This is no longer applicable in fedora. In near futuru I would like to completely rewrite the readonly scripts to cooperate better with systemd.