Bug 1074358

Is there an possibility to regenerate those iniramfses without the rescue image?

Proposing as a Beta blocker on the basis of messing with existing installs; to me this seems worse than just not having working dual boot, as it prevents you being able to boot the existing OSes using the usual methods, you additionally have to fix their initramfs'es with some kind of rescue tool to boot them.

No clear criterion for this yet, though we are in the process of drafting one on test@.

Discussed at 2014-10-01 blocker review meeting - http://meetbot.fedoraproject.org/fedora-blocker-review/2014-10-01/f21-blocker-review.2014-10-01-15.58.log.txt . Rejected as a blocker, as on a close reading, it doesn't violate any existing or proposed criteria: the proposed dual-boot-with-existing-Fedora criterion does not cover shared /boot scenarios .

According to the installer team, we don't really intend to support the sharing of /boot in this way - it's not safe to expect two different distribution releases will be able to agree on what the things in /boot should look like:

<dlehman> my opinion? we shouldn't allow use of preexisting /boot filesystem
<dlehman> adamw: https://git.fedorahosted.org/cgit/anaconda.git/tree/pyanaconda/packaging/__init__.py?h=f21-branch#n345
<adamw> dlehman: so, it's going to Do Stuff for all kernels it finds in /boot, basically?
<dlehman> yes
* dlehman originally tried to write it to use the contents of rpms we installed, but live
<dlehman> reason #376 why I don't like live install
<adamw> dlehman: so what i'm suspecting is that the old kernels would actually work with the newer fedora, but they're broken with the older one because of this linux / linux16 change i didn't get time to grok yet?
<dlehman> sounds likely

The most likely thing to happen here is that we disallow/strongly warn on use of shared /boot. There are ways we could attempt to cope with it, but they're complex and fragile and still might not make people happy.

Just re-discovered this one, for the record based on later investigations into linux/linux16 I don't think that has anything to do with it, it's probably just breaking when dracut for a later release generates a broken initramfs for an earlier release (or mixes arches, or whatever).

I feel like possibly we could try to filter this somewhat - at least don't operate on kernels for other arches, maybe don't operate on kernels of different release versions - but just not allowing shared /boot would of course avoid the necessity.

Someone else hit this on test@ and was...not happy:

https://lists.fedoraproject.org/pipermail/test/2014-December/124329.html

It does seem like we should *either* disallow shared /boot or come up with some kind of filtering for this for F22, it's a bit dangerous.

I just hit this.  Gritting my teeth and leaving arguments about the supportability/desirability of shared /boot aside, why is anaconda doing anything with the initramfs files for any kernel other than the on which it is installing?  Is this an unintentional side effect which can presumably be fixed, or is there some benefit to this behavior that I'm not seeing?

chattr +i on the existing initramfs files appears to be a workaround.  Of course, one does need to know to do this in advance.

It does this intentionally, it's not a side-effect. I don't have the precise explanation of why and I don't really feel like digging through all the code to explain it, but it's easy to see that it's intentional, because there's a function written precisely f or the purpose of regenerating initramfs'es.

def recreateInitrds(self, force=False):
        """ Recreate the initrds by calling new-kernel-pkg

            This needs to be done after all configuration files have been
            written, since dracut depends on some of them.

(In reply to Adam Williamson (Red Hat) from comment #7)
> It does this intentionally, it's not a side-effect.

Hmm.  Looking at _updateKernelVersionList in __init__.py, and the commit that introduced it (https://git.fedorahosted.org/cgit/anaconda.git/commit/pyanaconda/packaging/__init__.py?id=58e9cc22fc49bf7a99bf5c9dd0c927ce8a5cb7d7), I don't see any particular evidence of a desire to mess with initramfs files from other operating systems.  I think it's equally possible that it's an unintended side effect.

Adding David Shea who wrote _updateKernelVersionList for his perspective ...

That was covered in the IRC log in c#3:

<adamw> dlehman: so, it's going to Do Stuff for all kernels it finds in /boot, basically?
<dlehman> yes
* dlehman originally tried to write it to use the contents of rpms we installed, but live
<dlehman> reason #376 why I don't like live install

sorry, I slightly misread your question. It is intentional that anaconda re-generates initrds at all. It doesn't actually want to regenerate any initrds other than the ones it is itself installing, the fact that it in fact touches others is indeed not desired, but solving that is not easy.

Mind you, a thought occurs on that. pyanaconda/packaging/livepayload.py subclasses Payload. Couldn't the generic Payload class's _kernelVersionList() do the thing where it gets its list from the contents of RPMs we installed, and the LiveImagePayload class could override it with one which derived its list from the kernels present in the live image (so it too wouldn't consider kernels other than ones it was itself deploying)?

(In reply to Adam Williamson (Red Hat) from comment #10)
> Mind you, a thought occurs on that. pyanaconda/packaging/livepayload.py
> subclasses Payload. Couldn't the generic Payload class's
> _kernelVersionList() do the thing where it gets its list from the contents
> of RPMs we installed, and the LiveImagePayload class could override it with
> one which derived its list from the kernels present in the live image (so it
> too wouldn't consider kernels other than ones it was itself deploying)?

I'm not at all familiar with anaconda's innards, but it did occur to me that when _updateKernelVersionList finds an initramfs file it could look for a corresponding modules directory in /mnt/sysimage/lib/modules.  If it doesn't find one, it could skip that version.

y'know, I'm sure some awkward bugger can think of some problem with it, but that sounds pretty viable to me. Or in fact it could just find the kernel versions based on /lib/modules full stop.

so completely untested, something like this:

diff --git a/pyanaconda/packaging/__init__.py b/pyanaconda/packaging/__init__.py
index 74096a2..478ec42 100644
--- a/pyanaconda/packaging/__init__.py
+++ b/pyanaconda/packaging/__init__.py
@@ -315,15 +315,14 @@ class Payload(object):
 
         files = glob(iutil.getSysroot() + "/boot/vmlinuz-*")
         files.extend(glob(iutil.getSysroot() + "/boot/efi/EFI/%s/vmlinuz-*" % self.instclass.efi_dir))
+        dirs = glob(iutil.getSysroot() + "/lib/modules/*")
 
-        versions = sorted((f.split("/")[-1][8:] for f in files if os.path.isfile(f)), cmp=cmpfunc)
-        log.debug("kernel versions: %s", versions)
+        versions = sorted((d.split("/")[-1][8:] for d in dirs if os.path.isdir(d)), cmp=cmpfunc)
+        rescue = sorted((f.split("/")[-1][8:] for f in files if os.path.isfile(f)) and "-rescue-" in f, cmp=cmpfunc)
 
         # Store regular and rescue kernels separately
-        self._kernelVersionList = (
-                [v for v in versions if "-rescue-" not in v],
-                [v for v in versions if "-rescue-" in v]
-                )
+        self._kernelVersionList = (versions, rescue)
+        log.debug("kernel versions: %s", self._kernelVersionList)

bah, it doesn't need the [8:] any more, make that:

diff --git a/pyanaconda/packaging/__init__.py b/pyanaconda/packaging/__init__.py
index 74096a2..e46ab80 100644
--- a/pyanaconda/packaging/__init__.py
+++ b/pyanaconda/packaging/__init__.py
@@ -315,15 +315,14 @@ class Payload(object):
 
         files = glob(iutil.getSysroot() + "/boot/vmlinuz-*")
         files.extend(glob(iutil.getSysroot() + "/boot/efi/EFI/%s/vmlinuz-*" % self.instclass.efi_dir))
+        dirs = glob(iutil.getSysroot() + "/lib/modules/*")
 
-        versions = sorted((f.split("/")[-1][8:] for f in files if os.path.isfile(f)), cmp=cmpfunc)
-        log.debug("kernel versions: %s", versions)
+        versions = sorted((d.split("/")[-1] for d in dirs if os.path.isdir(d)), cmp=cmpfunc)
+        rescue = sorted((f.split("/")[-1][8:] for f in files if os.path.isfile(f)) and "-rescue-" in f, cmp=cmpfunc)
 
         # Store regular and rescue kernels separately
-        self._kernelVersionList = (
-                [v for v in versions if "-rescue-" not in v],
-                [v for v in versions if "-rescue-" in v]
-                )
+        self._kernelVersionList = (versions, rescue)
+        log.debug("kernel versions: %s", self._kernelVersionList)
 
     ###
     ### METHODS FOR QUERYING STATE

(In reply to jiri vanek from comment #1)
> Is there an possibility to regenerate those iniramfses without the rescue
> image?

just for record, running dracut in chrooted affected system via eg live disk, is working fine.

(In reply to Adam Williamson (Fedora) from comment #14)
> bah, it doesn't need the [8:] any more, make that:
> 
> diff --git a/pyanaconda/packaging/__init__.py
> b/pyanaconda/packaging/__init__.py
> index 74096a2..e46ab80 100644
> --- a/pyanaconda/packaging/__init__.py
> +++ b/pyanaconda/packaging/__init__.py
> @@ -315,15 +315,14 @@ class Payload(object):
>  
>          files = glob(iutil.getSysroot() + "/boot/vmlinuz-*")
>          files.extend(glob(iutil.getSysroot() + "/boot/efi/EFI/%s/vmlinuz-*"
> % self.instclass.efi_dir))
> +        dirs = glob(iutil.getSysroot() + "/lib/modules/*")
>  
> -        versions = sorted((f.split("/")[-1][8:] for f in files if
> os.path.isfile(f)), cmp=cmpfunc)
> -        log.debug("kernel versions: %s", versions)
> +        versions = sorted((d.split("/")[-1] for d in dirs if
> os.path.isdir(d)), cmp=cmpfunc)
> +        rescue = sorted((f.split("/")[-1][8:] for f in files if
> os.path.isfile(f)) and "-rescue-" in f, cmp=cmpfunc)
>  
>          # Store regular and rescue kernels separately
> -        self._kernelVersionList = (
> -                [v for v in versions if "-rescue-" not in v],
> -                [v for v in versions if "-rescue-" in v]
> -                )
> +        self._kernelVersionList = (versions, rescue)
> +        log.debug("kernel versions: %s", self._kernelVersionList)
>  
>      ###
>      ### METHODS FOR QUERYING STATE

I tried this from an F21 live CD, and I got a traceback at this line:

  rescue = sorted((f.split("/")[-1][8:] for f in files if os.path.isfile(f)) and "-rescue-" in f, cmp=cmpfunc)

  NameError: global name 'f' is not defined

Does the [8:] need to be removed from this line as well?

no, that's not it. What the [8:] does is cut the 'linux-' off the front of 'linux-3.17-3.fc21' or whatever, so you wind up with '3.17-3.fc21'. Not sure what's going wrong there at first glance, I can poke it a bit more but I'd rather get dlehman/davidshea to sign off on the basic idea before spending too much time on it.

(In reply to Adam Williamson (Red Hat) from comment #10)
> Mind you, a thought occurs on that. pyanaconda/packaging/livepayload.py
> subclasses Payload. Couldn't the generic Payload class's
> _kernelVersionList() do the thing where it gets its list from the contents
> of RPMs we installed, and the LiveImagePayload class could override it with
> one which derived its list from the kernels present in the live image (so it
> too wouldn't consider kernels other than ones it was itself deploying)?

Yeah, that sounds like the best way to go about it.

For the record, David Shea's proposed fix for this had some issues and has been reverted; there is now a proposed patch to disallow shared /boot . If anyone has a really good argument for allowing shared /boot that isn't mentioned above, please let us know.

(In reply to Adam Williamson (Red Hat) from comment #19)
> For the record, David Shea's proposed fix for this had some issues and has
> been reverted; there is now a proposed patch to disallow shared /boot . If
> anyone has a really good argument for allowing shared /boot that isn't
> mentioned above, please let us know.

In addition to using a shared /boot to allow clean installs (as the Fedora Docs recommend) without losing the previous install as I explained above, I also keep things like a gparted live iso and memtest in my /boot should anything need fixing or testing without the system being live.  I also do my installs by copying the pxeboot install kernel and initrd files into /boot to do an install over the network on machines with no dvd/cd drives.  A failed install that has already reformatted /boot will remove the possibility of booting anything else to try to fix the problem.

I kept meaning to suggest that the default disk setup for Fedora have dual root partitions and directly support what I've been doing manually.  This allows the best features of both an upgrade that keeps existing configuration and a clean install that removes the cruft and starts from a new file system.  Forcing a format on /boot at install seems to me to remove this possibility.

(In reply to Adam Williamson (Red Hat) from comment #19)
> For the record, David Shea's proposed fix for this had some issues and has
> been reverted; there is now a proposed patch to disallow shared /boot . If
> anyone has a really good argument for allowing shared /boot that isn't
> mentioned above, please let us know.

Similar to Tom Shield, I too like to do a clean install with the option to revert back to the previous one.  Having a shared /boot is something many people are used to doing I imagine.  And given Fedora is a fairly bleeding edge distro (in a positive sense) we may need to revert back at times.

(In reply to Adam Williamson (Red Hat) from comment #19)
> For the record, David Shea's proposed fix for this had some issues and has
> been reverted; there is now a proposed patch to disallow shared /boot . If
> anyone has a really good argument for allowing shared /boot that isn't
> mentioned above, please let us know.

Aaargh!  Please don't do this.  Throw up all the warnings that you want, but this will make it almost impossible for me to install Fedora on my system.  Almost all of my disk space is used by either my Windows partitions or bcache, with only about 1GB left over for /boot.

What about the idea of looking in the newly installed system's /lib/modules directory to determine which initramfs files should be rebuilt?

Everyone can cease panicking now, we're back to trying to fix it:

https://lists.fedorahosted.org/pipermail/anaconda-patches/2015-January/015699.html

I'll try and get some testing done on that today.

*** Bug 1217781 has been marked as a duplicate of this bug. ***