Bug 445218

Summary: RFE: Support persistence of /home across Fedora versions
Product: [Fedora] Fedora Reporter: Kai Engert (:kaie) (inactive account) <kengert>
Component: LiveCDAssignee: Jeremy Katz <katzj>
Status: CLOSED NEXTRELEASE QA Contact:
Severity: low Docs Contact:
Priority: low    
Version: 9CC: dcantrell
Target Milestone: ---Keywords: FutureFeature
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-06-18 10:56:43 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kai Engert (:kaie) (inactive account) 2008-05-05 15:19:53 UTC
I would like to see that /home can be kept on a live USB stick, even when I
later upgrade to a newer Fedora Live release.


The recent new Persistence feature of Live CDs makes it possible to carry over
changes between boot sessions.

But, what happens in 6 months when I want to upgrade my USB live stick to the
next release of Fedora?

I assume the livecd-overlay is strictly bound to the base filesystem.
Let's assume I will upgrade the base filesystem to a newer live cd.
I don't know what will happen. I will assume there are conflicts and it won't
work at all.

But let's assume for a moment that the overlay filesystem is really smart and
will continue to work on top of a newer base.
Then I think it will still cause problems.
For example, the changes to the yum database of the old live cd obviously will
conflict with the yum database of the new live cd.


I conclude:

The current persistence feature is not immediately helpful for keeping /home on
live upgrades on a single stick.


I propose:

During boot of a Fedora Live CD, the init scripts should search for a partition
with a label "fedoralivehome". This could be an ext3 filesystem, it could live
on a separate partition on the same USB stick.

If such a partition is found, it will be mounted as /home.
(If not found, proceed as usual)


I had made this proposal on the fedora-livecd mailing list on 2008-02-20.
I had been told that the persistence feature would help, but I suspect it does not.

My proposal is to combine the new Persistence feature with this separate
proposal, I believe they could go hand in hand.


This bug requests that the init scripts get changed to enable such an automatic
mount of /home


I would like to mention, that a manual solution to this bug exists.
(Probably, have not tried yet)

Using the persistence feature, it should be possible to manually edit /etc/fstab
and use an entry to mount /home to a separate partition.

When upgrading later to a newer Fedora Live release (on a USB stick), the licecd
base and the overlay should probbaly get recreated. The manual change to
/etc/fstab would have to be done again. But the /home partition can be kept
unchanged.

Comment 1 Bug Zapper 2008-05-14 10:40:05 UTC
Changing version to '9' as part of upcoming Fedora 9 GA.
More information and reason for this action is here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 2 Jeremy Katz 2008-05-29 20:33:11 UTC
So the question(s) I keep coming back to with this are
1) What should the filesystem be labeled?  Something that's per-release,
per-image, or just general use with any live image?  Each has their ups and downs
2) Would you want to do /home or just /home/fedora?
3) How do you do this *and* supported an encrypted partition?  As that's when it
gets really interesting.  Unfortunately, luks headers don't have a label and
just have a UUID :-/

Comment 3 Kai Engert (:kaie) (inactive account) 2008-05-30 12:38:33 UTC
Please let me note, I'm not proposing that separate /home partition as an
alternative solution to the current persistence, but as an additional parallel
solution.

I think persistence is great for release related filesystem changes, and /home
should be kept in the upgrade stick sceneario.


> 1) What should the filesystem be labeled?  Something that's per-release,
> per-image, or just general use with any live image?  Each has their ups and downs

In my opinion, it should be a "general" label like "fedoralivehome".
I'd expect a single usb stick to carry a single live OS, only.

If only a single USB stick with live OS is plugged in at boot time, this should
work. That's what I'd expect as a usual configuration. If users plug in multiple
sticks, each carrying a filesystem with that label, well, bad luck.


> 2) Would you want to do /home or just /home/fedora?

I'd personally do /home


> 3) How do you do this *and* supported an encrypted partition?  As that's when it
> gets really interesting.  Unfortunately, luks headers don't have a label and
> just have a UUID :-/

I think creating that separate /home requires manual interaction anyway.
If someone wants to create an encrypted partition, he must do so manually, and
will probably edit /etc/crypttab accordingly.

So, if the encrypted partition gets added to /dev/mapper/ early during startup,
would it work if the encapsulated filesystem contained the label? Would that
/dev/mapper partition be found?


Comment 4 Jeremy Katz 2008-06-01 21:18:44 UTC
(In reply to comment #3)
> Please let me note, I'm not proposing that separate /home partition as an
> alternative solution to the current persistence, but as an additional parallel
> solution.

Definitely!  And I'm entirely on-board with trying to figure out the best way to
make it work.  Then I could stop having my custom livecd config which does some
of this stuff with a big stick ;)  But at the same time, I think that doing it
with a encrypted /home is the only thing that's really "safe" to do --
otherwise, the amount of possibility of losing a stick with sensitive data
becomes very high.  Higher even than having your laptop stolen

> > 3) How do you do this *and* supported an encrypted partition?  As that's when it
> > gets really interesting.  Unfortunately, luks headers don't have a label and
> > just have a UUID :-/
> 
> I think creating that separate /home requires manual interaction anyway.
> If someone wants to create an encrypted partition, he must do so manually, and
> will probably edit /etc/crypttab accordingly.

But by this argument, creating the separate /home requires manual interaction
and you can just edit /etc/fstab accordingly.  
 
> So, if the encrypted partition gets added to /dev/mapper/ early during startup,
> would it work if the encapsulated filesystem contained the label? Would that
> /dev/mapper partition be found?

Then we'd have to unlock any encrypted partitions that we find on every boot. 
Which, while it would work, could be bad.  Especially if you're using a computer
that's not yours which has some encrypted partitions on it.  And you have to do
the unlocking to have any idea of what's actually on the encapsulated filesystem.

But I'll play around with some things this week

Comment 5 David Zeuthen 2008-06-01 21:30:49 UTC
(In reply to comment #4)
> Then we'd have to unlock any encrypted partitions that we find on every boot. 
> Which, while it would work, could be bad.  Especially if you're using a computer
> that's not yours which has some encrypted partitions on it.  And you have to do
> the unlocking to have any idea of what's actually on the encapsulated filesystem.
> 
> But I'll play around with some things this week

On the root file system, you can store the UUID's of the LUKS devices you need
to unlock. For example you could have this in the /etc/fstab file using the
comment option like this: comment:LUKS:<uuid>. Like this

UUID=52586ab0-e339-44d3-9e80-f0cd2e3d3db7 /home                   ext3   
defaults,comment=LUKS:830fe109-4981-4959-ad16-289ae322eb7d:        1 2

Should be fairly trivial to teach /etc/rc.sysinit, or even 'mount -a', about this.

Comment 6 Jeremy Katz 2008-06-01 21:44:29 UTC
(In reply to comment #5)
> (In reply to comment #4)
> > Then we'd have to unlock any encrypted partitions that we find on every boot. 
> > Which, while it would work, could be bad.  Especially if you're using a computer
> > that's not yours which has some encrypted partitions on it.  And you have to do
> > the unlocking to have any idea of what's actually on the encapsulated
filesystem.
> > 
> > But I'll play around with some things this week
> 
> On the root file system, you can store the UUID's of the LUKS devices you need
> to unlock. For example you could have this in the /etc/fstab file using the
> comment option like this: comment:LUKS:<uuid>. Like this
> 
> UUID=52586ab0-e339-44d3-9e80-f0cd2e3d3db7 /home                   ext3   
> defaults,comment=LUKS:830fe109-4981-4959-ad16-289ae322eb7d:        1 2
> 
> Should be fairly trivial to teach /etc/rc.sysinit, or even 'mount -a', about this.

Sure, but the tricky thing is it'd be nice to have the *stock* livecd with no
modifications able to use a persistent and encrypted /home.  It's easy enough to
handle persistent non-encrypted /home -- but the lack of labeling in luks
outside of the uuid makes it a bit trickier

Comment 7 Kai Engert (:kaie) (inactive account) 2008-06-02 09:58:02 UTC
(In reply to comment #4)
> > I think creating that separate /home requires manual interaction anyway.
> > If someone wants to create an encrypted partition, he must do so manually, and
> > will probably edit /etc/crypttab accordingly.
> 
> But by this argument, creating the separate /home requires manual interaction
> and you can just edit /etc/fstab accordingly.  

Yes, I agree.

I think it would be a good first step to have the plain unencrypted /home
working, out of the box.
I think that would work for many people.
It's sufficient for occasional testing and for "sysadmin sticks" or for "test
fedora release sticks", where you want to keep your downloads around, and keep
your settings, but where you won't do any confidential work.

I think supporting encryption out-of-the-box seems to be tricky. For the time
being power users can set it up themselves.


So, if we're taling about out-of-the-box, then maybe the plain /home partition
could be integrated into the live-cd-tools, too.

Instead of using a real partition (which would again require some manual
preparation by the user), we could just use a loopback file.

In other words:
- assume the standard stick has a single big partition
- introduce 3 new command line options:
  --create-home-file <filename-on-stick|default>
  --home-size <size-in-mb>
  --use-home-file <filename-on-stick|default>

- when installing the live iso to disk, the user can specify either
  --create-home-file ... --home-size ...
  or
  --use-home-file ...

- the installer script sets up something to 
  mount -o loop /home during startup

- on first install, the user can specify --create-home-file,
  on updates the user can specify --use-home-file (to keep the old home)


- actually, using the above approach I guess it would be
  possible to support a new flag --encrypt-home
  If given, when combined with --create-home-file,
  the installer can execute luksFormat, luksOpen, and mkfs.ext3 inside
  In both --create-home-file or --use-home-file, the installer
  sets up the magic to loop mount that file


But I propose to start with support for plain partitions, and do the
encrypt-home solution in a subsequent step.


Comment 8 Jeremy Katz 2008-06-02 14:42:00 UTC
Hmmm, the idea of doing it as a loopback file rather than as an actual partition
is interesting.  And it makes it pretty trivial to support encryption -- we can
then look at the file with blkid and see if we need to unlock it or not.

The downside is you then can't have your stick-of-persistent-settings being used
with a CD (automatically; syntax sugar like we already do for the persistent
snapshot could be done).  But maybe that's good enough.

Comment 9 Kai Engert (:kaie) (inactive account) 2008-06-02 15:00:11 UTC
We could use the initial proposal from this bug for live CDs (separate plain
labeled partition with hardcoded name fedoralivehome) and usb stick installs
that didn't specify --home parameters.

We could use the loopback file for everything that needs to be smarter (on same
partition, using loopback file, either plain or encrypted).


Comment 10 Jeremy Katz 2008-06-04 21:32:48 UTC
I've pushed a patch now that implements this by using the loopback file by
default, but also making it so that you can specify a partition by label or uuid
on the kernel command line to use instead (persistendhome=LABEL=foo).  Which
also will support an encrypted partition as you could do
persistenthome=UUID=<uuidhere>.  To make it a little nicer to use a partition
that you may have, I've also tweaked livecd-iso-to-disk so that you can add an
extra option to the syslinux.cfg.

Comment 11 Kai Engert (:kaie) (inactive account) 2008-07-18 19:27:50 UTC
Jeremy, that sounds great, thanks for working on it!
I wonder where you pushed it to?
The licecd-tools package in rawhide does not seem to contain that new option?

Comment 12 Jeremy Katz 2008-07-18 19:48:21 UTC
It's only in git at the moment; I need to do a new livecd-tools build for
rawhide, but I've been holding off to try to get a few other patches in first

Comment 13 Bug Zapper 2009-06-10 00:37:19 UTC
This message is a reminder that Fedora 9 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 9.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '9'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 9's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 9 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 14 Kai Engert (:kaie) (inactive account) 2009-06-18 10:56:43 UTC
Thank you very much for your great work on this. I set up a live stick with f10, then ugpraded it to f11, and it kept the home dir. Worked very well for me!

I guess we can close this.

(setting to nextrelease, as this bug is f9, and I can see it fixed in f10)