Created attachment 984683 [details] boot journalctl output Description of problem: Latest Kernel update 3.18.3 is not detecting the two LVM partitions that host my /(root) and /home folders - drops to emergency console eventually. Booting to the prior kernel 3.17.8 works. Additional info: See attached "sos" report
Did that happen on a pure update of the kernel packages _or_ after a Fedora update from Fedora 20? FWIW: the latter happened to me updating from Fedora 20 and I had to reboot a previous, still working kernel and localinstall 3.18.3 to fix it.
I see few of these: lvm: error while loading shared libraries: /lib64/libbz2.so.1: cannot read file data: Error 21 I wonder why is libbz2 needed? :-/ prajnoha, is there a missing dependency? Or just missing lib in initramdisk?
(In reply to Marian Csontos from comment #2) > I see few of these: > lvm: error while loading shared libraries: /lib64/libbz2.so.1: cannot read > file data: Error 21 We're not using libbz2, it must be a dependency of the library we're using (libblkid, libudev, libselinux, libreadline, ...). > > I wonder why is libbz2 needed? :-/ > prajnoha, is there a missing dependency? Or just missing lib in initramdisk? Neither of the two I think - if the library was missing, it would display an error like "cannot be found". This is "cannot read file data" which points to some kind of corruption. I would probably try running fsck and regenerating the initramfs (dracut --force) here.
Seems like the initramfs is incorrectly generated - if it's still possible to boot with the older kernel, do that and then "dracut --force /boot/initramfs-<the new kernel version>.img <the new kernel version>". For proper kernel version string to use, check /usr/lib/modules directory.
(In reply to Heinz Mauelshagen from comment #1) > Did that happen on a pure update of the kernel packages _or_ > after a Fedora update from Fedora 20? During a pure update - not a move from Fedora 20 to F21 > FWIW: the latter happened to me updating from Fedora 20 and I had to > reboot a previous, still working kernel and localinstall 3.18.3 to fix > it. I will try this if an fsck and rebuild of initramfs does not work.
(In reply to Peter Rajnoha from comment #4) > Seems like the initramfs is incorrectly generated - if it's still possible > to boot with the older kernel, do that and then "dracut --force > /boot/initramfs-<the new kernel version>.img <the new kernel version>". For > proper kernel version string to use, check /usr/lib/modules directory. I will try this. Although I did try and rebuild the initramfs using mkinitrd (I forgot Fedora is uses dracut), is there a difference between the two tools - i.e. do they make different initramfs files?
I have tried: 1. Rebuild of the initramfs using dracut 2. fsck (except on /(root) - not sure how to when it needs to be unmounted and it is in an LVM volume.) Given that I can boot into my machine using the previous kernel, I am assume that implies the file systems are clean anyway! 3. Reinstalling kernel 3.18.3 (yum remove/install) Now of these has solved the problem. SSD disk partitioning: /dev/sda1 efi /dev/sda2 /boot /dev/sda3 LVM2 containing / and /home /dev/sda4 LVM2 containing /virtStore (virtual machine storage)
(In reply to Bruce Wolfe from comment #7) > I have tried: > 1. Rebuild of the initramfs using dracut > 2. fsck (except on /(root) - not sure how to when it needs to be unmounted > and it is in an LVM volume.) Given that I can boot into my machine using the > previous kernel, I am assume that implies the file systems are clean anyway! > 3. Reinstalling kernel 3.18.3 (yum remove/install) > > Now of these has solved the problem. > > SSD disk partitioning: > /dev/sda1 efi > /dev/sda2 /boot > /dev/sda3 LVM2 containing / and /home > /dev/sda4 LVM2 containing /virtStore (virtual machine storage) Have you tried a "yum localinstall kernel-3.18.3-201.fc21.x86_64" yet? To check your root device, boot from a rescue media.
(In reply to Heinz Mauelshagen from comment #8) > (In reply to Bruce Wolfe from comment #7) > > I have tried: > > ... > > Have you tried a "yum localinstall kernel-3.18.3-201.fc21.x86_64" yet? > Yes - no change. > To check your root device, boot from a rescue media. Done - as expected, clean. I also noticed some dev/mapper and lvm updates in the latest set up updates, so applied them without any change either. Still cannot boot into kernel 3.18.3, no change in error reporting.
Following up on a suggestion from ask.fedora.org Regenerating initramfs again, but this time adding the --no-hostonly as an option and it appears that this solves the problem. I can now successfully boot into kernel 3.18.3 Not sure if you want to continue trying to investigate this as a bug? I will leave it to someone else here to make that decision.
lvm: error while loading shared libraries: /lib64/libbz2.so.1: cannot read file data: Error 21 was clearly the culprit here. What is the output of: $ sudo lsinitrd <faulty_image> | grep libbz2 Please also run $ sudo dracut --debug -f --kver 3.18.3-201.fc21.x86_64 test.img &>debug.log $ sudo lsinitrd test.img | grep libbz2 &>> debug.log $ rm test.img and attach debug.log
I have been away and otherwise busy for the last couple of weeks and my plan was to do the steps asked by Harald Hoyer today - however I logged in to find a new kernel update waiting (3.18.6), applied it and rebooted - successful! The problem existed for 3.18.3 and 3.18.5, but is fixed in 3.18.6. Harald, do you want me to take your requested actions anyway? Or shall I close this bug?
Created attachment 991905 [details] debug.log of dracut --debug I've had the same problem on 3.18.3-201 and 3.18.6-200. Regenerating initramfs with dracut and --no-hostonly hasn't helped in both cases. Attached is the output of dracut --debug and lsinitrd for 3.18.6-200 This was a clean install of F21
Kernel 3.18.7 is also working fine for me - no regression.
(In reply to Bruce Wolfe from comment #14) > Kernel 3.18.7 is also working fine for me - no regression. Kernel 3.18.7 working here as well. Seems I had an additional issue where XFS and LVM had different ideas about partition size. For some reason this was fine for kernel < 3.17 but 3.18 would break. Resizing partitions and xfs_repair seems to have sorted it out.
(In reply to E Stalmans from comment #15) > (In reply to Bruce Wolfe from comment #14) > > Kernel 3.18.7 is also working fine for me - no regression. > > Kernel 3.18.7 working here as well. > > Seems I had an additional issue where XFS and LVM had different ideas about > partition size. For some reason this was fine for kernel < 3.17 but 3.18 > would break. Resizing partitions and xfs_repair seems to have sorted it out. well, nothing dracut can fix
This has reappeared in kernel 3.18.9-200.fc21.x86_64 I was able to use dracut --no-hostonly to resolve the issue again. Should this bug be reopened??
Sorry forgot to flag my last comment.
(In reply to Bruce Wolfe from comment #17) > This has reappeared in kernel 3.18.9-200.fc21.x86_64 > > I was able to use dracut --no-hostonly to resolve the issue again. > > Should this bug be reopened?? Well, the last two 3.8.x Fedora 20 updates haven't worked for me, so the real question is: if I update to the latest in the queue (3.18.9-100), and it replaces my 3.17.8 in Grub, will I be able to boot into my machine at all?