Bug 974742

Summary: can't install on md array
Product: [Fedora] Fedora Reporter: Mario <bug>
Component: python-blivetAssignee: David Lehman <dlehman>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 19CC: bcl, dlehman, dshea, g.kaviyarasu, jonathan, mkolman, paul-redhat, sbueno, vanmeeuwen+fedora
Target Milestone: ---Flags: bug: needinfo-
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-12-02 19:28:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
anaconda logs
none
kikstart installation logs
none
kikstart file used
none
anaconda log - interactive installation
none
anaconda program log - interactive installation
none
anaconda storage log - interactive installation
none
anaconda log - kickstart installation
none
anaconda program log - kickstart installation
none
anaconda storage log - kickstart installation
none
paulm_mdraid_problem__anaconda.log
none
paulm_mdraid_problem__anaconda-yum.conf
none
paulm_mdraid_problem__program.log
none
paulm_mdraid_problem__storage.log
none
paulm_mdraid_problem__storage.state
none
paulm_mdraid_problem__syslog none

Description Mario 2013-06-15 09:02:06 UTC
Description of problem:


Version-Release number of selected component (if applicable): 19 beta


How reproducible:


Steps to Reproduce:
1.Start Fedora 19 beta installation DVD media 
2. try to select a previously created md array in storage configuration


Actual results:
md0pX partition are not shown shown even if they are correctly detected. Only physical disks and and an "unknown" disk with the same size as the array size are shown. The brand new "specialized" disk menu does not help.

Expected results:
Just see in custom partition selection the existing md0pX partitions and select them as installation targets. It would be also desirable to just use pre-formatted (with all raid enhancements) partitions without having to reformat root target.

Additional info:
I am trying to install Fedora on a OCZ revodrive X3 SSD PCIe disk. In Linux this SSD is seen by mvsas kernel driver as 2 or 4 separate disks (dipending on size). According to community experiences, the only way to boot from Revodrive is to create a mdraid configuration as follows, the same BIOS seems to export:

mdadm --verbose --create /dev/md0 -e 1.0 --homehost=any --raid-devices=4 --chunk=64 --level=0 --name=0 /dev/sd[ghij]

I can create such an array inside a live distribution or from an existing fedora18 installation (I did so).
With the md configuration on the SSD I can manually install other distributions (i.e. Slackware) to md0pX targets. I currently run on a OCZ drive Fedora 18 installations built from partition images or file-by-file copy from an existing installation on a traditional disk. I can install grub on the OCZ md0 drive and correctly boot from it. So there is no hardware or software issue preventing fedora installation on this drive.
I attach anaconda logs

Comment 1 Mario 2013-06-15 09:03:27 UTC
Created attachment 761540 [details]
anaconda logs

Comment 2 Mario 2013-06-15 15:50:38 UTC
Created attachment 761615 [details]
kikstart installation logs

Comment 3 Mario 2013-06-15 15:51:40 UTC
Created attachment 761616 [details]
kikstart file used

Comment 4 Mario 2013-06-15 15:52:37 UTC
No luck feeding a kickstart file via http. See logs

Comment 5 Mario 2013-06-16 08:40:19 UTC
BTW, I can select the (4 in 240GB size) single SSD disks from the OCZ card in anaconda storage configurator, and choose to make a raid0 array, but the result is a set of striped partitions from all disks. Total size is the same, but such array will not boot at all.

Furthermore I think that anaconda installer (both in Fedora and REHL) should allow, when choosing custom disk configuration, the maximum flexibility, i.e. choosing any existing partition and not formatting any of them (included /) if not requested to do.

Thanks all.
Mario

Comment 6 Brian Lane 2013-06-19 00:36:33 UTC
Please attach the logs as individual text/plain files.

Comment 7 Mario 2013-06-19 11:26:27 UTC
Created attachment 762874 [details]
anaconda log - interactive installation

Comment 8 Mario 2013-06-19 11:31:34 UTC
Created attachment 762875 [details]
anaconda program log - interactive installation

Comment 9 Mario 2013-06-19 11:32:14 UTC
Created attachment 762876 [details]
anaconda storage log - interactive installation

Comment 10 Mario 2013-06-19 11:33:03 UTC
Created attachment 762877 [details]
anaconda  log - kickstart installation

Comment 11 Mario 2013-06-19 11:33:35 UTC
Created attachment 762878 [details]
anaconda program log - kickstart installation

Comment 12 Mario 2013-06-19 11:34:28 UTC
Created attachment 762881 [details]
anaconda  storage log - kickstart installation

let me know if other logs are needed
Thanks

Comment 13 David Lehman 2013-06-19 12:48:38 UTC
Installation to partitioned md devices other than firmware-RAID arrays is not supported. There are no plans to add support at this time. You might have some luck if you create a DDF container using mdadm so that it looks like fwraid to the installer.

Comment 14 Mario 2013-06-19 14:25:18 UTC
(In reply to David Lehman from comment #13)
> You might
> have some luck if you create a DDF container using mdadm so that it looks
> like fwraid to the installer.

I'll try and report here the result

Comment 15 Paul Mansfield 2013-12-09 10:12:39 UTC
this problem still exists in Fedora 20 beta, if you have an existing mdraid setup. 

On the main installer screen, it tells you to do something with the disk configurations.

Select the disks. Choose custom partioning. The disk partitioning tool will find the mdraid devices having triggered an mdadm assemble/scan, allow you to choose the /dev/md devices and format them and choose mount points, but on returning to the main install action screen says there's a problem with the disks and it won't proceed.

I have tried flipping to the command console to do an mdadm --assemble --scan and the /dev/md devices get created and it all looks fine.

So at the moment I am unable to progress past the main installation screen!

Comment 16 Brian Lane 2013-12-09 19:55:27 UTC
Paul, please attach the logs from /tmp/*log as individual text/plain attachments.

Also, if there is an error you should be able to view it by going to the installation destination spoke and clicking on the orange bar at the bottom.

Comment 17 Paul Mansfield 2013-12-10 11:12:53 UTC
Created attachment 834707 [details]
paulm_mdraid_problem__anaconda.log

Comment 18 Paul Mansfield 2013-12-10 11:13:31 UTC
Created attachment 834708 [details]
paulm_mdraid_problem__anaconda-yum.conf

Comment 19 Paul Mansfield 2013-12-10 11:16:10 UTC
Created attachment 834709 [details]
paulm_mdraid_problem__program.log

Comment 20 Paul Mansfield 2013-12-10 11:16:41 UTC
Created attachment 834710 [details]
paulm_mdraid_problem__storage.log

Comment 21 Paul Mansfield 2013-12-10 11:17:27 UTC
Created attachment 834711 [details]
paulm_mdraid_problem__storage.state

Comment 22 Paul Mansfield 2013-12-10 11:17:53 UTC
Created attachment 834712 [details]
paulm_mdraid_problem__syslog

Comment 23 Paul Mansfield 2013-12-10 11:26:52 UTC
here's the "story".. first frame is when first booted and you have to make some installation choices, through to clicking through to disk partitioner.

I have tried this with and without doing the mdadm-assemble-scan bit in the hope of getting it working.


I've also found that fedora19 deliberately won't install when you have a degraded mirror set. Sometimes I want to upgrade a PC which has mdraid sets by breaking the mirrors, replacing the system partitions then adding on the 2nd drive sub mirrors. This should be a warning, not a failure.

Comment 24 Paul Mansfield 2013-12-10 11:27:11 UTC
here's the "story".. first frame is when first booted and you have to make some installation choices, through to clicking through to disk partitioner.
https://plus.google.com/photos/116367041269325477816/albums/5955721279092111249?authkey=CNbRi-nilNW68QE

I have tried this with and without doing the mdadm-assemble-scan bit in the hope of getting it working.


I've also found that fedora19 deliberately won't install when you have a degraded mirror set. Sometimes I want to upgrade a PC which has mdraid sets by breaking the mirrors, replacing the system partitions then adding on the 2nd drive sub mirrors. This should be a warning, not a failure.

Comment 25 Paul Mansfield 2013-12-11 10:53:17 UTC
(In reply to Brian C. Lane from comment #16)
> Also, if there is an error you should be able to view it by going to the
> installation destination spoke and clicking on the orange bar at the bottom.

the orange bar has nothing to offer in the way of explanatory text.

it's easy to reproduce this problem.

create a two-disk computer. boot the installer. switch to a command shell. create the following partitions on /dev/sda

p1 - 512M - /boot - type xFD
p2 - 1024 - swap - type x82
p3 - whole disk - extended
l5 - everything - / - type xFD

copy the partition table to /dev/sdb using "sfdisk -id /dev/sda | sfdisk /dev/sdb". 

create the mirrors, something like "mdadm create --raid-devices=2 --level=1 --metadata 0.90 /dev/md1 /dev/sda1 /dev/sdb1", similarly for /dev/md5 but without the metadata bit.

then "mkswap -L SWAP1 /dev/sda2 ; mkswap -L SWAP2 /dev/sdb2"

and then go through the disk partitioner tool.

Comment 26 Brian Lane 2013-12-11 16:59:22 UTC
Did you tell custom to rescan the disks after you made your changes? It has zero chance of working if you didn't do that. Also note that installing to degraded array is not supported. Clean it up before running the installer.

Comment 27 David Lehman 2013-12-11 18:37:19 UTC
(In reply to Paul Mansfield from comment #25)
> (In reply to Brian C. Lane from comment #16)
> > Also, if there is an error you should be able to view it by going to the
> > installation destination spoke and clicking on the orange bar at the bottom.
> 
> the orange bar has nothing to offer in the way of explanatory text.
> 
> it's easy to reproduce this problem.
> 
> create a two-disk computer. boot the installer. switch to a command shell.
> create the following partitions on /dev/sda
> 
> p1 - 512M - /boot - type xFD
> p2 - 1024 - swap - type x82
> p3 - whole disk - extended
> l5 - everything - / - type xFD
> 
> copy the partition table to /dev/sdb using "sfdisk -id /dev/sda | sfdisk
> /dev/sdb". 

Anaconda's partitioning interface is capable of creating this layout with a great deal less work, unless you insist on specific partition ordering and/or start/end sectors, for which there is very little use.

> 
> create the mirrors, something like "mdadm create --raid-devices=2 --level=1
> --metadata 0.90 /dev/md1 /dev/sda1 /dev/sdb1", similarly for /dev/md5 but
> without the metadata bit.

Fedora 20 uses GRUB2, which can boot from newer metadata version md arrays, including 1.0, 1.1, and 1.2. There is no need to continue using the old 0.90 metadata. This also allows you to name your md arrays instead of trying to pin down a minor number. I have observed behavior in mdadm which I consider to be a bug when creating a v0.90 array from the shell on tty2: it does not create a /dev/md/<minor> symlink, although this would be present under any other circumstances as far as I know.

So, in summary, I suggest any combination of the following:

1. Use anaconda to create your storage layout.
2. stop using v0.90 metadata for /boot
3. give your arrays a name
4. Reboot after creating and v0.90 metadata arrays before trying to use them as installation targets.

Comment 28 Paul Mansfield 2013-12-12 14:40:28 UTC
I tried rebooting the box and the fedora installer still didn't want to install onto the existing mirrors. I don't recall seeing a feature which would allow destroying the mirrors either.

I just started an F19 install and was able to create a mirrored set on blank drives without a problem, however, it wasn't all that intuitive to be honest.

One of the best partitioning tools for installation is that in openSuse. They went through some pain a while back to make it work well.

Comment 29 Paul Mansfield 2014-05-29 16:13:31 UTC
I was trying to reinstall a fedora desktop computer with existing home partition, but wanted to wipe the boot and root partitions and keep home.

all of them are mdraid, and the fedora20 release dvd installer failed to detect the mdraid devices, even after I manually formatted them - I had hoped I could pick the existing mdraid devices and either reformat them or simply specify a mount point.

Incidentally, I was building a new PC with brand new "blank" Seagate 1TB drives and the installer failed in trying to detect the disks until I zeroed out the start of the disk. I'll see if I can get a screenshot with the errors. This happened a previous time, also with brand new shrink-wrapped disks from Seagate.


Overall then the experience is very poor, it used to be adequate if you were starting from scratch, but now installing F20 at all is a lottery.

Comment 30 Brian Lane 2014-05-29 18:19:22 UTC
The storage.log and output from 'parted -s /dev/sdX u s p' on the undetected disks would be more helpful than a screenshot.

Comment 31 David Lehman 2014-05-30 20:45:46 UTC
(In reply to Paul Mansfield from comment #25)
> (In reply to Brian C. Lane from comment #16)
> create the mirrors, something like "mdadm create --raid-devices=2 --level=1
> --metadata 0.90 /dev/md1 /dev/sda1 /dev/sdb1", similarly for /dev/md5 but
> without the metadata bit.

There is a bug in mdadm whereby it does not create a symlink in /dev/md/ when you create arrays this way. If you do either of the following things I suspect your arrays will be detected correctly:

 1. specify /dev/md/1 (or a name with actual meaning, like '/dev/md/swap')
    instead of /dev/md1
 2. reboot after creating the arrays


Your first problem (comment 15) was far simpler. The error in your configuration was that the start sector of the first partition on sdb was too low for grub2 to be able to reliably work. The way to see such errors is to click on the button/selector that is displaying the error, then click on the banner at the bottom to see the details.

It would still be useful to see the logs from the installer failing to detect branch new disks.