Bug 983287
| Summary: | Fedora 19 Rescue Option On GRUB Menu Returns Kernel Panic | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | KitchM <tech> | ||||||
| Component: | grubby | Assignee: | Peter Jones <pjones> | ||||||
| Status: | CLOSED EOL | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | 19 | CC: | bcl, bugzilla, gansalmon, itamar, jonathan, madhu.chinakonda, pjones, tech | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | --- | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2015-02-17 16:03:59 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
*********** MASS BUG UPDATE ************** We apologize for the inconvenience. There is a large number of bugs to go through and several of them have gone stale. Due to this, we are doing a mass bug update across all of the Fedora 19 kernel bugs. Fedora 19 has now been rebased to 3.11.1-200.fc19. Please test this kernel update and let us know if you issue has been resolved or if it is still present with the newer kernel. If you experience different issues, please open a new bug report for those. Does this mean that the problem is now fixed with the new kernel? (In reply to KitchM from comment #2) > Does this mean that the problem is now fixed with the new kernel? Did 3.11.1 boot? Either way, your original report has a panic that was produced by the lack of an initramfs. That isn't something the kernel can fix. Sure, it boots. Heaven help us if a kernel is downloaded to everyone and no one can boot any longer. Does that happen very often? The real issue has nothing to do with that. This bug has to do with the rescue kernel. And the question remains, is it fixed? Look, all the user knows is that they select a kernel at the boot up of their computer. At that point, if it fails, it is kernel issue. If somehow you believe it to be another issue, then it is the responsibility of the knowledgable person to give direction as to what should be done to solve the problem. No one has yet to tell us what to do. By the way, what is an initramfs? And why is it lacking if it is needed? Who would be the one who handles it? Does anyone know why this failure exists in Fedora 19 and who is responsible for it? (In reply to KitchM from comment #4) > Sure, it boots. Heaven help us if a kernel is downloaded to everyone and no > one can boot any longer. Does that happen very often? At times, yes. > The real issue has nothing to do with that. This bug has to do with the > rescue kernel. And the question remains, is it fixed? The kernel itself is no different from the other kernel you have installed. It should really be phrased as 'rescue initramfs'. It simply contains additional tools to help you fix a non-booting system. The irony here is not lost. > Look, all the user knows is that they select a kernel at the boot up of > their computer. At that point, if it fails, it is kernel issue. If somehow Sure, it looks that way. As I said, it isn't though. The kernel booted and it wasn't given enough information to find the root partition, so it can't do anything. > you believe it to be another issue, then it is the responsibility of the > knowledgable person to give direction as to what should be done to solve the > problem. No one has yet to tell us what to do. You need to regenerate the rescue initramfs. I actually don't know how to do that at the moment. > By the way, what is an initramfs? And why is it lacking if it is needed? An initramfs is the initial userspace code that is responsible for booting the system. It's needed in most cases. > Who would be the one who handles it? Does anyone know why this failure > exists in Fedora 19 and who is responsible for it? Grubby generates it when the kernel is installed by calling dracut. It not being created is an intermittent issue that nobody has figured out yet. Thank you for the explanation. However, what should be done with the problem? Who is responsible? I think it's anaconda's responsibility to create the initramfs for the rescue kernel. If so, bug 1013087 may be a dupe of this bug. I'm finding it 100% of the time with Fedora 20 installs. Thanks much, Chris. I guess I will have to create a new bug and/or add to yours. How many kernel options do you have in the grub menu other than the rescue option? Look in the /boot directory for the rescue kernel and matching initramfs. If you do: ls -l | grep rescue you should see two results: initramfs-0-rescue and vmlinuz-0-rescue. Do you? If not, which one is missing? If you have both, type the following: cat /boot/grub2/grub.cfg | grep rescue | grep initrd Is there a single line returned, and does that initramfs name match the earlier initramfs name? Yes/No. I believe there are five or six items in the grub boot list, with the last one being the rescue item. ls -l | grep rescue run from /boot returned: -rw------- 1 root root 26301600 Jul 1 18:06 initramfs-0-rescue-4b4c9af28f4d4ace83131c99832afaf6.img -rwxr-xr-x 1 root root 5534744 Jul 1 18:06 vmlinuz-0-rescue-4b4c9af28f4d4ace83131c99832afaf6 I tried cat /boot/grub2/grub.cfg | grep rescue | grep initrd and got: cat: /boot/grub2/grub.cfg: Permission denied So I tried it as root and got: initrd /initramfs-0-rescue-4b4c9af28f4d4ace83131c99832afaf6.img They look the same, but you decide. You have an initramfs for the rescue kernel, both have the same datetime stamp, so it was properly installed. And your grub.cfg has an initrd entry for the correct initramfs file. So your bug isn't related to my two bugs on this. If you're getting kernel panics with the rescue kernel that's really unexpected because that initramfs contains everything the original working initramfs has, and then some. So that's just going to take troubleshooting, there's no way for anyone to do that for you or speculate why it isn't working. What you can try is using the arrow key to highlight, but not select, the rescue kernel option in the grub menu. And then hit the e key to edit the entry. Scroll down to to find the linux line, and remove the items 'rhgb quiet'. And then press F10 to boot the edited entry. This will at least let you see boot text to take a photo of when the kernel panic happens, maybe there will be a clue there what's going on. So Josh's comment number 3 was wrong? Nope. Looks like maybe that initrd line is malformed after all. Also, I have kernel 3.9.5-300 not debug, and you have 3.9.8-300 debug. We haven't installed from the same source which may account for the difference in behaviors. Please attach your grub.cfg, as a plain text attachment. Created attachment 807885 [details]
grub.cfg as requested
You have /boot on its own partition, the grub.cfg is correct. It has entries for all of the kernels and their initramfs's. And you have an initramfs for the rescue kernel. So now I'm willing to bet that this initramfs is simply built wrong, or was moved from another computer. Try rebuilding it with dracut -f -N initramfs-0-rescueXXXXXXXX.img vmlinuz-0-rescueXXXXXX obviously you will need to put in the exact correct hash used for your rescue kernel and initramfs names. Make sure you cd to /boot first... Please specify the exact command line command to run. Still need response. Thanks. I gave you the exact command line to run. You have to fill in the XXXXX part for YOUR rescue kernel. I have no idea what that hash value is because it's different for each kernel version. Okay, here's what I did. I opened a terminal window and switched to the /boot directory. I ran dracut -f -N initramfs-0-rescue4b4c9af28f4d4ace83131c99832afaf6.img vmlinuz-0-rescue4b4c9af28f4d4ace83131c99832afaf6 but got an error. So I ran sudo dracut -f -N initramfs-0-rescue4b4c9af28f4d4ace83131c99832afaf6.img vmlinuz-0-rescue4b4c9af28f4d4ace83131c99832afaf6 and it returned without an error. I rebooted and chose the Recovery option on the grub menu, but got the same kernel panic. It does state that it is not tainted, if that helps any. Oh I know what's going on now. So the rescue kernel is simply a renamed kernel, and doesn't have its own stash of kernel modules on rootfs. And I bet dollars to donuts that kernel has been deleted. So the rescue kernel has no kernel modules to reference. That's why the dracut command fails *and* it's why you get a kernel panic *and* why when you get a kernel update, it doesn't fix the problem. On kernel updates some script determines if you already have a rescue kernel in /boot, sees the (broken) rescue kernel and therefore doesn't replace it. Somehow it wasn't cleaned up properly and that's the real bug, and that's going to be a hard one to track down - so good luck with that. So to fix your problem, delete the rescue kernel and its matching initramfs. Then grab a suitable older kernel that you know works on your hardware from koji.fedoraproject.org, and manually install that with rpm -ivh --force blahkernel.rpm and that will install that kernel, and a renamed version for rescue with an appropriate initramfs and a grubby entry. As a final icing on the cake, if you really want, you can run grub2-mkconfig -o /boot/grub2/grub.cfg to clean up the grub menu. Or you could run: /etc/kernel/postinst.d/51-dracut-rescue-postinst.sh after removing the rescue stuff from /boot/ you'll still need to clean up grub.cfg since it will set the rescue as the default entry. That seems like a lot of trouble for users who need to rescue their computer's OS. Doesn't that defeat the whole point of a rescue option? KitchM: You're suggesting the behavior is intentional, which it isn't. There's also no evidence it's widespread after a kernel update. So do what bcl suggests in c22, and follow-up if that works or not. Not at all. What I am saying is that this behavior is not useful to me or the average user. It is actually diametrically opposed to the intended function; that of repairing a problem with the OS. If this option must be repaired before it will work, how can it be useful? Further, there is no mention of intent, nor of blame; especially since we are not yet clear as to what is causing the unexpected behavior. Since I first mentioned this problem, it has gotten more complex and obtuse than first expected. As to the suggested step, I would want to know if you are asking me to do a test for you folks so that a fix can be created for all, or if you are just trying to make it work for me. If this will help others, I am willing to try next time I have a minute. If for me, I'd wait until 20 comes out. Please clarify for me. Let's distinguish the two cases first. One there is a bug and a fix described in bug 1013087 comment 31: which causes the rescue initramfs to not be created during the original install. There really isn't a way to fix this for F19. The other case, is the lack of a rescue initramfs not being subsequently created during a kernel update. Only you have reported such a case, so it's not clear at all that this is a widespread problem. And it's unclear how it happened. So I suggest you do as bcl says in c22 and see if that works. If it does it may have just been a one off event. If it doesn't, then we'll have to find out why it doesn't work. I've never reported such case. Right, sorry. Your case is that for some reason the supporting modules for your rescue kernel were deleted but the rescue kernel wasn't cleaned up and then replaced. Since I'm not familiar with the mechanism of its creation I can't speculate why it happened. But it's probably not wide spread because all of my rescue kernels work on F19 and F20 even through updates. Confirmed - Won't Fix No, that's incorrect. At this point the condition is unique to your configuration, and the cause isn't understood. If you can reproduce the situation with steps someone else can follow, then there's a chance it can be fixed. This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed. |
Created attachment 771870 [details] Photo of boot process and kernel panic Description of problem:Kernel Panic on booting from Rescue option Version-Release number of selected component (if applicable): Fedora 19 How reproducible: always Steps to Reproduce: 1.Select Fedora 19 Rescue from grub menu 2. 3. Actual results:kernel panic Expected results: Additional info: