Bug 1266898

Summary: Fedora doesn't boot with RAID /boot
Product: [Fedora] Fedora Reporter: Aniruddha <mailingdotlist>
Component: anacondaAssignee: Brian Lane <bcl>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 22CC: anaconda-maint-list, g.kaviyarasu, jonathan, mailingdotlist, vanmeeuwen+fedora
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: anaconda-23.19.6-1 anaconda-23.19.6-1.fc23 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-12 06:27:24 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
anaconda.log
none
dnf.log
none
dnf.rpm.log
none
journal.log
none
packaging.log
none
program.log
none
storage.log
none
syslog none

Description Aniruddha 2015-09-28 12:31:19 UTC
Created attachment 1077921 [details]
anaconda.log

I installed Fedora Server with  the following partitions

RAID 10 /boot 
RAID 10 swap
RAID 10 BTRFS /

After the install Fedora Server doesn't boot and GRUB gives the following error. 

error: disk not found
entering rescue mode...
grub rescue>

Comment 1 David Shea 2015-09-28 14:19:34 UTC
Please attach the other logs from the installation. They are available in /tmp in the installer environment and copied to /var/log/anaconda in the installed system.

Comment 2 Aniruddha 2015-09-28 15:26:33 UTC
Created attachment 1077955 [details]
dnf.log

Comment 3 Aniruddha 2015-09-28 15:27:11 UTC
Created attachment 1077956 [details]
dnf.rpm.log

Comment 4 Aniruddha 2015-09-28 15:27:36 UTC
Created attachment 1077959 [details]
journal.log

Comment 5 Aniruddha 2015-09-28 15:27:58 UTC
Created attachment 1077960 [details]
packaging.log

Comment 6 Aniruddha 2015-09-28 15:28:18 UTC
Created attachment 1077961 [details]
program.log

Comment 7 Aniruddha 2015-09-28 15:28:48 UTC
Created attachment 1077962 [details]
storage.log

Comment 8 Aniruddha 2015-09-28 15:29:09 UTC
Created attachment 1077963 [details]
syslog

Comment 9 Brian Lane 2015-09-28 15:51:22 UTC
I'm pretty sure grub2 cannot handle booting from RAID 10. Change it to RAID 1 and it should work correctly.

We should be catching this and not allowing it to be created though.

Comment 10 Aniruddha 2015-09-28 16:26:19 UTC
(In reply to Brian Lane from comment #9)
> I'm pretty sure grub2 cannot handle booting from RAID 10. Change it to RAID
> 1 and it should work correctly.
> 
> We should be catching this and not allowing it to be created though.

Isn't the problem that grub2-install --no-floppy only runs on /dev/sda? Shouldn't this run on sdb, sdc and sdd as well?

Comment 11 Aniruddha 2015-09-28 16:28:17 UTC
I tried a re-install with RAID 1 for /boot instead of RAID 10. The installation failed. Here is the relevant information from the anaconda log. Just let me know if you need more info or if I need to open another bug report. 

18:07:13,261 INFO anaconda: Creating ext4 on /dev/md/boot
18:07:13,916 INFO anaconda: Created ext4 on /dev/md/boot
18:16:23,062 INFO anaconda: Installing boot loader
18:16:23,062 INFO anaconda: boot loader stage1 target device is sda
18:16:23,063 INFO anaconda: boot loader stage2 target device is boot
18:16:23,093 DEBUG anaconda: new default image: <pyanaconda.bootloader.LinuxBootLoaderImage object at 0x7faf981d6cd0>
18:16:23,611 INFO anaconda: bootloader.py: mbr will be updated for grub2
18:16:47,519 INFO anaconda: bootloader.py: mbr will be updated for grub2
18:17:09,920 INFO anaconda: bootloader.py: used boot args: rd.md.uuid=81f27045:1b93f0dd:2af4981a:29191e32 rd.luks.uuid=luks-9356dfcb-1446-4d23-9969-db4293f3b542 rd.md.uuid=32e9303b:021b8104:49e691fb:e0e4ccc5 rd.luks.uuid=luks-e5448398-1cbb-4b63-ab8e-de1660b641c8 rd.luks.uuid=luks-b9d149a2-13c3-4ab6-bf1f-d5b150445633 rd.luks.uuid=luks-8d5c0ca9-db58-441f-8165-a00ce923a7bd rd.luks.uuid=luks-6b3cbb00-e7a6-402b-ac87-0d82713a2eaf rhgb quiet 
18:17:21,126 ERR anaconda: bootloader.write failed: boot loader install failed
18:17:37,517 DEBUG anaconda: running handleException
18:17:37,519 CRIT anaconda: Traceback (most recent call last):

  File "/usr/lib64/python2.7/site-packages/pyanaconda/threads.py", line 244, in run
    threading.Thread.run(self, *args, **kwargs)

  File "/usr/lib64/python2.7/threading.py", line 766, in run
    self.__target(*self.__args, **self.__kwargs)

  File "/usr/lib64/python2.7/site-packages/pyanaconda/install.py", line 266, in doInstall
    writeBootLoader(storage, payload, instClass, ksdata)

  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 2450, in writeBootLoader
    writeBootLoaderFinal(storage, payload, instClass, ksdata)

  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 2372, in writeBootLoaderFinal
    storage.bootloader.write()

  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1625, in write
    self.install()

  File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line 1610, in install
    raise BootLoaderError("boot loader install failed")

BootLoaderError: boot loader install failed

18:17:37,526 DEBUG anaconda: Gtk running, queuing exception handler to the main loop
18:17:37,527 INFO anaconda: Thread Done: AnaInstallThread (140392262526720)
18:18:14,425 INFO anaconda: Running kickstart %%traceback script(s)
18:18:14,426 INFO anaconda: All kickstart %%traceback script(s) have been run

Comment 12 Brian Lane 2015-09-28 23:35:06 UTC
(In reply to Aniruddha from comment #10)
> (In reply to Brian Lane from comment #9)
> > I'm pretty sure grub2 cannot handle booting from RAID 10. Change it to RAID
> > 1 and it should work correctly.
> > 
> > We should be catching this and not allowing it to be created though.
> 
> Isn't the problem that grub2-install --no-floppy only runs on /dev/sda?
> Shouldn't this run on sdb, sdc and sdd as well?

For RAID1 it does, but that's the only one it knows how to handle.

(In reply to Aniruddha from comment #11)
> I tried a re-install with RAID 1 for /boot instead of RAID 10. The
> installation failed. Here is the relevant information from the anaconda log.
> Just let me know if you need more info or if I need to open another bug
> report. 

>   File "/usr/lib64/python2.7/site-packages/pyanaconda/bootloader.py", line
> 1610, in install
>     raise BootLoaderError("boot loader install failed")
> 
> BootLoaderError: boot loader install failed

What do the logs from this run look like? I just did a RAID1 /boot and / install and it works for me.

Comment 13 Aniruddha 2015-09-29 06:48:44 UTC
I did another install with a RAID 10 /boot partition and the server boots. Probably because it boots from the correct hard drive which leads me to believe that if the installer issues a grub2-install on all drives, booting should work all the time.

Comment 14 Aniruddha 2015-09-29 06:51:09 UTC
# cat /proc/mdstat 

md127 : active raid10 sdb2[1] sdc2[2] sda3[0] sdd2[3]
      513024 blocks super 1.0 512K chunks 2 near-copies [4/4] [UUUU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

# df -h

/dev/md127      478M   92M  357M  21% /boot

Comment 15 Brian Lane 2015-09-29 23:43:30 UTC
Give this patch a try if you like. Ends up we weren't using the same list of allowed raid levels in bootloader.py vs. the GUI

Comment 16 Aniruddha 2015-09-30 06:30:11 UTC
Thanks, where can I find the patch? I've noticed another problem. The installer doesn't create a biosboot partition on all devices. Creating a biosboot partion on all devices would be expected behavior for a redundant setup. Instead only sda gets a biosboot partition.

Comment 17 Brian Lane 2015-09-30 14:11:56 UTC
(In reply to Aniruddha from comment #16)
> Thanks, where can I find the patch? I've noticed another problem. The
> installer doesn't create a biosboot partition on all devices. Creating a
> biosboot partion on all devices would be expected behavior for a redundant
> setup. Instead only sda gets a biosboot partition.

oops. Patch is here:

https://github.com/rhinstaller/anaconda/pull/378

I can make an updates.img is you need it.

Good point about biosboot, if you are using > 2T disks on BIOS we will select GPT for them. Could you open a new bug for that?

On UEFI we only support RAID1 for the ESP since the firmware has to be able to read the partition.

Comment 18 Aniruddha 2015-09-30 19:15:01 UTC
(In reply to Brian Lane from comment #17)

> oops. Patch is here:
> 
> https://github.com/rhinstaller/anaconda/pull/378
> 
> I can make an updates.img is you need it.

What is the best way to test this? Or can you point me to documentation on how to test a patch with anaconda? Thanks!

> Good point about biosboot, if you are using > 2T disks on BIOS we will
> select GPT for them. Could you open a new bug for that?

Done.

Comment 19 Brian Lane 2015-10-01 17:10:34 UTC
Boot a recent build of Fedora 23 and add this to the kernel cmdline:

inst.updates=https://bcl.fedorapeople.org/updates/1266898.img

Comment 20 Fedora Update System 2015-10-08 18:22:38 UTC
python-blivet-1.12.6-1.fc23 anaconda-23.19.6-1.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-6f7d535c92

Comment 21 Fedora Update System 2015-10-09 13:54:59 UTC
anaconda-23.19.6-1.fc23, python-blivet-1.12.6-1.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update anaconda python-blivet'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-6f7d535c92

Comment 22 Fedora Update System 2015-10-12 06:27:06 UTC
anaconda-23.19.6-1.fc23, python-blivet-1.12.6-1.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.