Bug 1219264 - Intel firmware RAID set does not appear in INSTALLATION DESTINATION in live installer [NEEDINFO]
Summary: Intel firmware RAID set does not appear in INSTALLATION DESTINATION in live i...
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: anaconda
Version: 22
Hardware: All
OS: All
unspecified
high
Target Milestone: ---
Assignee: Anaconda Maintenance Team
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard: RejectedBlocker https://fedoraproject...
: 1212036 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-05-06 21:43 UTC by Adam Williamson
Modified: 2016-07-19 19:07 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-07-19 19:07:52 UTC
dlehman: needinfo? (extras-qa)


Attachments (Terms of Use)
storage.log (180.77 KB, text/plain)
2015-05-06 21:44 UTC, Adam Williamson
no flags Details
program.log (33.60 KB, text/plain)
2015-05-06 21:47 UTC, Adam Williamson
no flags Details
journal from Final TC3 Server netinst (165.37 KB, text/plain)
2015-05-14 20:28 UTC, Adam Williamson
no flags Details
storage.log from Final TC3 Server netinst (139.95 KB, text/plain)
2015-05-14 20:29 UTC, Adam Williamson
no flags Details
program.log from Final TC3 Server netinst (33.26 KB, text/plain)
2015-05-14 20:29 UTC, Adam Williamson
no flags Details
journal from Final TC3 Workstation live (388.45 KB, text/plain)
2015-05-14 20:38 UTC, Adam Williamson
no flags Details
storage.log from Final TC3 Workstation live (181.31 KB, text/plain)
2015-05-14 20:38 UTC, Adam Williamson
no flags Details
program.log from Final TC3 Workstation live (33.60 KB, text/plain)
2015-05-14 20:39 UTC, Adam Williamson
no flags Details
storage.log file from anaconda install F24 Beta (71.24 KB, text/plain)
2016-05-11 01:20 UTC, bpk678
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Bugzilla 1172426 None None None Never
Red Hat Bugzilla 1212036 None None None Never

Internal Links: 1172426 1212036

Description Adam Williamson 2015-05-06 21:43:39 UTC
Another Intel firmware RAID bug! Isn't this fun? Don't worry, I brought gin.

Seems the firmware RAID set doesn't show up when running the installer from the Workstation live image (in Final TC2, but I never tested it from a live image at Beta, so this probably hasn't ever worked in 22). Non-live is (still) OK, but in the live installer, the only device shown in INSTALLATION DESTINATION is the USB stick I booted from.

Attaching program.log and storage.log.

I'm proposing this as a blocker as a conditional violation of "The installer must be able to detect and install to hardware or firmware RAID storage devices." - https://fedoraproject.org/wiki/Fedora_22_Beta_Release_Criteria#Hardware_and_firmware_RAID - in the live install case.

Comment 1 Adam Williamson 2015-05-06 21:44:29 UTC
Created attachment 1022820 [details]
storage.log

Comment 2 Adam Williamson 2015-05-06 21:47:52 UTC
Created attachment 1022821 [details]
program.log

Comment 3 Dan Mossor [danofsatx] 2015-05-11 19:25:55 UTC
Discussed at the 2015-05-11 blocker review meeting[0]. Voted as AcceptedBlocker.

#agreed 1219264 - AcceptedBlocker - violates "The installer must be able to detect and install to hardware or firmware RAID storage devices" for live installs

[0] http://meetbot.fedoraproject.org/fedora-blocker-review/2015-05-11/

Comment 4 Vratislav Podzimek 2015-05-14 11:43:35 UTC
Adam, could you please test it with libblockdev-0.13-1 (available in updates-testing)? Thanks.

Comment 5 Adam Williamson 2015-05-14 15:57:15 UTC
Tested - sorry, but still no good :(

Comment 6 Adam Williamson 2015-05-14 20:28:46 UTC
Created attachment 1025602 [details]
journal from Final TC3 Server netinst

dlehman said he'd like to see a log comparison between live and non-live, so I'll attach journal, storage and program from the Server netinst, and add journal output from a live boot.

Comment 7 Adam Williamson 2015-05-14 20:29:13 UTC
Created attachment 1025603 [details]
storage.log from Final TC3 Server netinst

Comment 8 Adam Williamson 2015-05-14 20:29:35 UTC
Created attachment 1025604 [details]
program.log from Final TC3 Server netinst

Comment 9 Adam Williamson 2015-05-14 20:38:11 UTC
Created attachment 1025605 [details]
journal from Final TC3 Workstation live

Comment 10 Adam Williamson 2015-05-14 20:38:38 UTC
Created attachment 1025606 [details]
storage.log from Final TC3 Workstation live

Comment 11 Adam Williamson 2015-05-14 20:39:04 UTC
Created attachment 1025607 [details]
program.log from Final TC3 Workstation live

Comment 12 Adam Williamson 2015-05-14 20:40:01 UTC
Note - I had to filter audit messages out of the live journalctl output to make it fit under the fpaste limit.

Comment 13 Vratislav Podzimek 2015-05-15 07:18:25 UTC
From the netinst storage.log:
> 20:24:41,638 INFO blivet: devices to scan: [u'sdc', u'sdc1', u'sdc2', u'sdc3', u'sda', u'sdb', u'sr0', u'loop0', u'loop1', u'loop2', u'live-rw', u'live-base', u'fedora-swap00', u'fedora-home00', u'fedora-root00', u'md126', u'md126p1', u'md126p2', u'md127']

from the live storage.log:
> 16:34:19,396 INFO blivet: devices to scan: [u'sdc', u'sdc1', u'sdc2', u'sdc3', u'sda', u'sdb', u'sdb1', u'sdb2', u'sdb3', u'sr0', u'loop0', u'loop1', u'loop2', u'loop3', u'loop4', u'fedora_adam-swap', u'fedora_adam-home', u'fedora_adam-root', u'live-rw', u'live-base', u'live-osimg-min', u'md127']

And of course the md126* devices are not processed by blivet in the live installation.

Comment 14 Jaroslav Reznik 2015-05-15 14:26:49 UTC
(In reply to Vratislav Podzimek from comment #13)
> And of course the md126* devices are not processed by blivet in the live
> installation.

Is it intentional (I doubt it is but you never know) or bug? Sorry, it's just not clear from "and of course md... not processed" :).

Comment 15 Adam Williamson 2015-05-15 15:15:38 UTC
I think by 'of course' he simply means 'they're not present, so of course they're not processed' - i.e. the problem is the actual RAID-0 set is not getting activated (just its container, md127).

The question is what ought to be activating it and why it isn't, I believe. To that end I'll check if this actually worked on F21 shortly, and add logs from that case.

Comment 16 Adam Williamson 2015-05-15 18:52:27 UTC
So, it doesn't work on F21 either. Same symptom, no RAID set visible on INSTALLATION DESTINATION.

IIRC when we talked about this on IRC the other day, someone mentioned that RAID discovery had been disabled for liveinst on the assumption that the OS would do it, but it doesn't look like it does. Live images actually have 'rd.md=0 rd.dm=0 rd.luks=0' on the cmdline in order to explicitly tell dracut *not* to do it...but note that removing those parameters isn't enough to make the array show up (IIRC, dracut doesn't just automatically activate any RAID sets it finds).

Comment 17 Adam Williamson 2015-05-15 18:55:36 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1172426 looks like an earlier report of this.

Comment 18 Adam Williamson 2015-05-15 19:13:43 UTC
Hum - as that reporter suggested, I tried recreating the RAID set and now both F21 and F22 lives see it. One of the member disks had its own GPT partition table, so perhaps that was screwing up the activation? I guess the RAID metadata got written over the main GPT, but the backup was still present (this is what parted said after I destroyed the original RAID set). I destroyed the set, booted live, used parted and fdisk to wipe the partitions from the disk and create a new MBR disk label, then rebooted back to the RAID interface, created the RAID set again, and then both F21 and F22 lives could see it.

I'm gonna move this back to proposed blocker - it might be good for a couple of other testers to try and see if they see problems or not, but it does look like the bug isn't as simple as 'fwraid sets never show up on live'.

Comment 19 Stephen Gallagher 2015-05-18 14:05:38 UTC
Given that it's not happening in all cases and there is a simple workaround (use the netinstall), I'm inclined to vote -1 blocker on this and send it to Common Bugs.

Comment 20 Jaroslav Reznik 2015-05-18 15:29:42 UTC
Same here, -1 blocker.

Comment 21 Petr Schindler 2015-05-18 16:15:07 UTC
Discussed at today's blocker review meeting [1]. 

This bug was rejected as a blocker but it was accepted as Freeze Exception: We haven't found clear reproduction steps, there's an easy workaround and more reports haven't shown up, so rejecting as a blocker. Accepted as an FE if the devs can make a sane fix for this.

[1] http://meetbot.fedoraproject.org/fedora-blocker-review/2015-05-18

Comment 22 Kamil Páral 2015-05-25 10:32:20 UTC
I believe Petr Schindler saw something similar - a RAID5 array was not visible neither in F22 nor in F21 installer. After we completely wiped the disks, it started to be visible (in F22, didn't check F21 I think). There had to be some remnant of previous contents which prevented recognizing the array.

Comment 23 Eric Work 2015-05-25 18:30:10 UTC
Kamil,
I believe this might be due to the problem I mentioned in BZ 1212036.  Fedora 21 (or maybe it was 20) created MD RAID arrays without a homehost.  This seems to cause problems for the Fedora 22 installer.  I had to manually set the homehost on my existing arrays (in addition to manually assembling them a few times) to finally get the installer to recognize my existing RAID arrays and not crash during the formatting stage.

Comment 24 Kevin Fenzi 2015-05-31 20:13:52 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=1225184 may be another case of this same issue...

Comment 25 Adam Williamson 2015-06-10 22:48:31 UTC
Clearing F22 accepted / nominated freeze exception status as F22 has shipped, per https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers . You may nominate as an Alpha, Beta or Final freeze exception for F23 if desired using the web application - https://qa.fedoraproject.org/blockerbugs/propose_bug (though it is not currently set up for F23) - or by marking the bug as blocking AlphaFreezeException, BetaFreezeException, or FinalFreezeException.

Eric: I don't think that matches my case. The RAID sets in my tests are always firmware RAID sets created in the firmware interface, not created by anaconda.

Comment 26 bpk678 2015-07-28 19:36:04 UTC
this needs to be fixed in an updated anaconda installer that can be used to install F22.  i have tried using pxeboot with both the server iso and the netinst iso, along with the server iso on usb and the netinst iso on usb.  no option has worked.  this is an outright failure of the install and should have been a release stopper.  i have hardware that i cant build.

Comment 27 David Shea 2016-02-02 20:58:16 UTC
*** Bug 1212036 has been marked as a duplicate of this bug. ***

Comment 28 David Shea 2016-02-29 16:05:39 UTC
*** Bug 1273684 has been marked as a duplicate of this bug. ***

Comment 29 David Lehman 2016-03-01 16:24:45 UTC
Where does this bug stand now? Was it re-tested for F23?

Comment 30 Adam Williamson 2016-03-01 16:33:51 UTC
Well I personally don't have a reliable reproducer - if you follow my comments it turned out this was happening with just one specific configuration of my RAID test box, and when I blew the disks away and re-created the RAID set the bug disappeared. I don't know exactly what it was about that specific configuration that triggered the bug, hence I can't easily re-test it.

I'll be doing RAID tests for F24 Alpha once all the current breakage calms down and I'm done working on the robots, but if you want to close this for lack of reproducibility in the mean time I wouldn't scream. One of the other reporters may have a better reproducer, though.

Comment 31 bpk678 2016-03-06 21:55:56 UTC
It seems I am able to reproduce the issue reliably.  I have a sacrificial box I can test on.  What testing is necessary?

Comment 32 bpk678 2016-05-10 20:28:00 UTC
i have downloaded F24Beta and tested with it.  the bug is still present, but Anaconda does not crash, it just does not identify any local disks to install to.  i have tar/gzipped the contents of /tmp, during the install and have the logs from the attempt to install.  how do i upload them for review?

Comment 33 bpk678 2016-05-11 01:20:26 UTC
Created attachment 1155944 [details]
storage.log file from anaconda install F24 Beta

Comment 34 David Lehman 2016-07-01 22:56:22 UTC
(In reply to bpk678 from comment #33)
> Created attachment 1155944 [details]
> storage.log file from anaconda install F24 Beta

Yours is not intel firmware RAID -- it's a promise fastrack array which the system is failing to assemble for some reason. This is not an installer problem, but rather a problem with either your array or the system tools for managing those arrays (dmraid being the main one).

Comment 35 bpk678 2016-07-02 15:15:26 UTC
(In reply to David Lehman from comment #34)
> (In reply to bpk678 from comment #33)
> > Created attachment 1155944 [details]
> > storage.log file from anaconda install F24 Beta
> 
> Yours is not intel firmware RAID -- it's a promise fastrack array which the
> system is failing to assemble for some reason. This is not an installer
> problem, but rather a problem with either your array or the system tools for
> managing those arrays (dmraid being the main one).

complete and utter rubbish.  Fedora 20 installs fine and runs with it.  i have tried F22, F23, F24Beta abd F24, all of which fail to install because of issues finding or using the very same array controller and disks.  literally the same exact devices.  i have 6 HP microservers and every single one of them exhibit this same exact failure.

when will someone fix this clear regression and stop passing the buck or trying to blame someone or something else?  this is pathetic that in 2 years or more, you have not even acknowledged the bug, never mind actually fixing it.

Comment 36 Adam Williamson 2016-07-03 03:46:47 UTC
David's trying to help you. Throwing insults around is not going to help you. He looked at your logs and told you what they told him; he's not lying:

20:47:06,308 INFO blivet: type detected on 'sda' is 'promise_fasttrack_raid_member'

20:47:06,639 WARN blivet: dmraid member sda does not appear to belong to any array

20:47:07,888 INFO blivet: type detected on 'sdb' is 'promise_fasttrack_raid_member'

20:47:08,054 WARN blivet: dmraid member sdb does not appear to belong to any array

anaconda and blivet rely on the underlying kernel and dmraid to pick up such arrays; if they don't, there really isn't anything anaconda can do. If you're sure the array is OK, then the appropriate component to move the bug to would be dmraid.

Comment 37 bpk678 2016-07-03 14:11:17 UTC
the user experience here is beyond frustrating and i have every right to voice that.  at least one other bug was opened for this and i provided info on it.  See Bug 1273684.  it was bounced around and marked as a dup of this issue.  that bug was not marked as a dup of this by me.  so someone needs to own why i keep getting jerked around and fix the bug already.

both this bug and the other one have languished in some kind of void, and have prevented me from upgrading to newer, better and more secure versions of software.  having waited just about 2 years for a bug/regression to NOT be fixed is not what i consider appropriate.

if this is in all actuality a separate bug, then you really need to evaluate the quality of changes being committed.  you likely have a more global issue on your hands affecting multiple types of software controllers or too many changes to too many components were not properly vetted and you have multiple bugs to address.

split these into two bugs or keep them as a single bug, that is for you to decide.  i had a separate bug open.  you open a new case or reopen the previous bug.  just tell me what info you need and where to post it.

Comment 38 Adam Williamson 2016-07-04 14:51:45 UTC
No-one 'needs to own' anything. This is a bug tracker, for a product you've paid nothing at all for. I think you may need to re-adjust your expectations a little. You don't get to issue instructions on what people working a product do just because you used it and had a problem. This is a bug tracker, it is not a customer service forum; it's provided for the convenience *of the developers* to help them track and fix bugs, not as a place for you to yell your frustrations at people who work very hard and could do without them. Now if you go buy some service from Red Hat and then you report an issue and no-one does anything about it, you can yell the damn house down. But let's be clear, in this particular relationship, no-one *owes* you squat, so being a jerk about it is simply going to make your bug much less likely to get fixed.

Again, I'm telling you this to help you: what I'm telling you is that if you want a bug to be fixed, this is not the way to go about it. If you just want to yell at the internet, you're doing fine.

My take is that your bug doesn't actually seem much like this one at all (this one was, I think, likely caused by one of the disks that was part of the RAID array also having stale LVM metadata from when it was used as a simple disk for an install); dshea probably just closed it as a dupe because you identified it as "the same problem as https://bugzilla.redhat.com/show_bug.cgi?id=1219264" in the initial submission. So let's re-open that one and re-assign it to dmraid.

Comment 39 Erik J 2016-07-13 19:44:43 UTC
Hi Adam, 

I am encountering a similar problem to what bpk678@yahoo.com reports above, that Anaconda does not correctly detect a Promise FastTrack RAID. Since the current ticket is about Intel Frimware RAID and not really applicable to my issue, I have posted my comments to the following ticket:

https://bugzilla.redhat.com/show_bug.cgi?id=1225184

Thanks for any guidance or help you might be able to provide me.

Thanks, Erik J.

Comment 40 Fedora End Of Life 2016-07-19 19:07:52 UTC
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.