Description of problem: installing Fedora 28 on hardware raid mirrored drives. LVM2 failed PV scan, due to duplicate IDs Version-Release number of selected component: anaconda-28.22.10 The following was filed automatically by anaconda: anaconda 28.22.10 exception report Traceback (most recent call first): File "/usr/lib/python3.6/site-packages/blivet/devices/partition.py", line 185, in __init__ self._parted_partition = self.disk.format.parted_disk.getPartitionByPath(self.path) File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock return m(*args, **kwargs) File "/usr/lib/python3.6/site-packages/blivet/populator/helpers/partition.py", line 106, in run exists=True, parents=[disk]) File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 263, in handle_device device = helper_class(self, info).run() File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock return m(*args, **kwargs) File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 462, in _populate self.handle_device(dev) File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock return m(*args, **kwargs) File "/usr/lib/python3.6/site-packages/blivet/populator/populator.py", line 412, in populate self._populate() File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock return m(*args, **kwargs) File "/usr/lib/python3.6/site-packages/blivet/blivet.py", line 161, in reset self.devicetree.populate(cleanup_only=cleanup_only) File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock return m(*args, **kwargs) File "/usr/lib64/python3.6/site-packages/pyanaconda/storage/osinstall.py", line 1670, in reset super().reset(cleanup_only=cleanup_only) File "/usr/lib/python3.6/site-packages/blivet/threads.py", line 53, in run_with_lock return m(*args, **kwargs) File "/usr/lib64/python3.6/site-packages/pyanaconda/storage/osinstall.py", line 2193, in storage_initialize storage.reset() File "/usr/lib64/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/usr/lib64/python3.6/site-packages/pyanaconda/threading.py", line 291, in run threading.Thread.run(self) AttributeError: 'NoneType' object has no attribute 'getPartitionByPath' Additional info: addons: com_redhat_kdump, com_redhat_docker cmdline: /usr/bin/python3 /sbin/anaconda cmdline_file: BOOT_IMAGE=vmlinuz initrd=initrd.img inst.stage2=hd:LABEL=Fedora-S-dvd-x86_64-28 quiet executable: /sbin/anaconda hashmarkername: anaconda kernel: 4.16.3-301.fc28.x86_64 product: Fedora release: Cannot get release name. type: anaconda version: 28
Created attachment 1443909 [details] File: anaconda-tb
Created attachment 1443910 [details] File: anaconda.log
Created attachment 1443911 [details] File: dbus.log
Created attachment 1443912 [details] File: dnf.librepo.log
Created attachment 1443913 [details] File: environ
Created attachment 1443914 [details] File: hawkey.log
Created attachment 1443915 [details] File: lorax-packages.log
Created attachment 1443916 [details] File: lsblk_output
Created attachment 1443917 [details] File: nmcli_dev_list
Created attachment 1443918 [details] File: os_info
Created attachment 1443919 [details] File: program.log
Created attachment 1443920 [details] File: storage.log
Created attachment 1443921 [details] File: ifcfg.log
Created attachment 1443922 [details] File: packaging.log
I've been attempting to load Fedora 28 through upgrades as well .. https://ask.fedoraproject.org/en/question/121758/fedora-28-anaconda-install-hardware-raid-mirror/?answer=121794#post-id-121794
Created attachment 1443923 [details] lsinitrd of Fedora 27 initramfs ... image file
Created attachment 1443924 [details] lsinitrd of Fedora 28 initramfs ... image file
Bug reports I have been referencing are: https://bugzilla.redhat.com/show_bug.cgi?id=1225184 https://bugzilla.redhat.com/show_bug.cgi?id=1201962 https://bugzilla.redhat.com/show_bug.cgi?id=1575762 As they discuss Anaconda having install issues with RAID sets, possibly due to lvm2 issues. I tried downgrading lvm2 .. but the system can't find it. <code> > dnf history list ID | Command line | Date and time | Action(s) | Altered ------------------------------------------------------------------------------- 6 | system-upgrade upgrade | 2018-05-28 00:55 | D, E, I, O, U | 1400 EE 5 | install dnf-utils | 2018-05-28 00:40 | I, O, U | 180 EE 4 | --releasever=27 system-u | 2018-05-27 21:02 | E, I, O, U | 1392 E< 3 | install dnf-plugin-syste | 2018-05-27 19:47 | Install | 3 >< 2 | install fedora-upgrade | 2018-05-27 15:59 | Install | 8 > 1 | upgrade | 2018-05-27 15:17 | I, O, U | 559 EE > dnf downgrade lvm2 Last metadata expiration check: 1:26:25 ago on Mon 28 May 2018 01:31:58 PM PDT. Package lvm2 of lowest version already installed, cannot downgrade it. Error: No packages marked for downgrade. </code>
Noted something strange in the initrmfs-4.11.16-300.fc28.x86_64.img, the blkid method returns UUID's that are garbage, or are the formatting string for the UUID. I can't quite tell. blkid on the LiveCD for Fedora 28 works properly.
Created attachment 1445105 [details] lsblk -f
Created attachment 1445107 [details] blkid with hard drive UUID
So I had a thought, using the LiveCD I booted into a command shell through the trouble shooting option. Here I am able to mount the boot partition of the RAID array ... > mount /dev/mapper/ddf1_BOOT01T1 /mnt/sysimage This is the path that Anaconda is expecting the image appear on. Thought 1: .. using the LiveCD .. try to create a new ramdisk using dracut. However, dracut is not part of the LiveCD image =(. And, since I can't trust the copy on the RAID array .. I stopped here. Thought 2: .. since the drive is mounted, execute ananconda and see if it will run in command line mode. Unfortunately, 2 sets of errors appeared. The original one that started this thread, and a new one, similar to the original one. I can go get a pic of it if there is interest. > /sbin/anaconda ... python failure ...
It looks to be an issue in the storage library. Reassigning.
In general it is not supported to run anaconda from a command line. It needs its environment set-up properly. However, from the live instalation there is a script which will set-up the environment. So if you want to start the installation you can do that by running 'liveinst' instead of 'anaconda'. About the rest. We are using python-blivet storage library so they may have better knowledge in this field so they could help you better then us. Also based on the traceback this exception is raised from their code. Changing components, could blivet developers please look on this issue?
Attempted to run liveinst > liveinst /sbin/liveinst: line 87: rpm : command not found Unable to init server: Could no connect: Connection refused (zenity:2214): Gtk-WARNING **: 18:23:45.379: cannot open display: > rpm bash: rpm: command not found
opened a bug concerning the upgrade path, will try to keep notes separate for both bugs from this point forward. https://bugzilla.redhat.com/show_bug.cgi?id=1583813
Is there anyone watching this bug?
If there is a solution to loading Fedora 28 onto mirrored hard drives, it probably lies in manually setting up the mirrors via mdadm, and running parted mklabel msdos, before attempting to run the install process. Below is the process I used to setup the mirrors after installing the OS on a single drive, and then mirror 2 data drives. > lsblk -f > mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sda /dev/sdb > cat /proc/mdstat > mdadm --detail --scan | tee -a /etc/mdadm/mdadm.conf > parted /mnt/md0 mklabel msdos ext4 0% 100% > parted /mnt/md0 mkpart primary > mkfs.ext4 -l [volume name] /dev/md0 > mkdir -p /mnt/md0 > mount /dev/md0 /mnt/md0 > update-initramfs -u > vi /etc/fstab ... UUID=[/dev/md0 UUID] /mnt/md0 ext4 default 0 2 ===== How To Partition and Format Storage Devices in Linux https://www.digitalocean.com/community/tutorials/how-to-partition-and-format-storage-devices-in-linux ow to Mount and Unmount Filesystem / Partition in Linux (Mount/Umount Command Examples) https://www.thegeekstuff.com/2013/01/mount-umount-examples/?utm_source=tuicool Hardware vs. Software RAID in the Real World https://backdrift.org/hardware-vs-software-raid-in-the-real-world-2 How To Create RAID Arrays with mdadm on Ubuntu 16.04 https://www.digitalocean.com/community/tutorials/how-to-create-raid-arrays-with-mdadm-on-ubuntu-16-04
What I found while loading Fedora 28 onto a single drive and then creating the mirrored partitions, as mentioned in comment 28, was that Linux ignored the hardware raid capabilities of the SuperMicro H8SGL and the StarTek RAID controllers. Instead, it continued to treat the drives as individual drives, till mdadm was configured. While the hardware reports that the RAIDS are optimal, I'm no longer sure which is performing the work .. Linux or the Controller. Or if both are doing the work.
An idea I hadn't considered while running into this problem was loading the OS on one of the drives, creating a mirrored raid set on a single drive, then copying the OS to the mirrored raid set, and then adding the drive into the mirrored raid set. It's not a time optimal solution, but could probably be done. The following URLS provide details on how to do this: 5.9.9.2. Rebuilding a RAID array https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/4/html/Introduction_To_System_Administration/s3-storage-raid-day2day-add.html Mdadm Cheat Sheet http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/ Mdadm Cheat Sheet https://gist.github.com/plepe/52ecc9f18efb32c68d18
additional URL for Comment 30 Setting Up RAID using mdadm on Existing Drive https://www.guyrutenberg.com/2013/12/01/setting-up-raid-using-mdadm-on-existing-drive/
Well there's obviously at least one bug here because anaconda shouldn't crash. The RAID metadata format is clearly DDF. I thought a long time ago DDF RAID was supposed to be managed by mdadm, and all the dmraid stuff was deprecated, so I'm not really sure why I'm seeing a bunch of dmraid and dmsetup references in the various logs, and not a single mdadm reference. Also, lsblk appears to show sda and sdb have two conventional partitions, and a third partition which is DDF. This is so confusing... I'd expect the whole drive to be DDF and only show up as a RAID member, no partitions; and then if the RAID successfully runs to then show the partitions on the array. But what we're seeing is a mix of all of this. Maybe dlehman can make heads or tails out of this and a way forward. If it were me, I'd update the backups, and obliterate everything (since blivet is apparently confused I'm not sure it's any help in doing a proper tear down: wipefs in reverse order to remove all signatures, or use hdparm security erase to make sure all these stale signatures are gone); and make sure the computer firmware is up to date. Next, since this RAID method is firmware initiated, but still DDF metadata, that means it's really software RAID. The only advantage to using firmware+DDF RAID over native mdadm or LVM managed RAID, is if you're also using Windows and need it to see and use the RAID too. If you're not using Windows, I would not use firmware RAID, and install just let the installer create arrays, which if you create LVM LV's and mark them as e.g. raid1, it will make them as LVM raid1, not as mdadm raid1 with LVM on top. So...there you go.
OK based on https://bugzilla.redhat.com/show_bug.cgi?id=1583813 it looks like there's a bug outside the installer that likely ends up confusing the installer.
I'm trapped in the same situation, and I found that the path of ddf1_BOOT01T is wrong: it should be /dev/mapper instead of /dev/ infomation in this anaconda-tb: Local variables in innermost frame: disk_tags: None weight: None primary: False part_type: None exists: True parents: [DiskDevice instance (0x7fcb85f6eeb8) -- name = ddf1_BOOT01T status = False id = 90 children = ['existing 0 B partition ddf1_BOOT01T1 (97)'] parents = [] uuid = None size = 0 B format = existing disklabel major = 253 minor = 2 exists = True protected = False sysfs path = target size = 0 B path = /dev/ddf1_BOOT01T format args = [] original_format = disklabel removable = wwn = None] sysfs_path: /sys/devices/virtual/block/dm-3 bootable: None minor: 3 major: 253 end: None start: None maxsize: None grow: False size: None uuid: 00060011-01 fmt: None name: ddf1_BOOT01T1 self: existing 0 B partition ddf1_BOOT01T1 (97) and erros in storage.log:[Errno 2] No such file or directory: '/dev/ddf1_BOOT01T1' 21:23:44,827 INF blivet: scanning ddf1_BOOT01T1 (/sys/devices/virtual/block/dm-3)... 21:23:44,830 DBG blivet: DeviceTree.get_device_by_name: name: ddf1_BOOT01T1 ; incomplete: False ; hidden: False ; 21:23:44,834 DBG blivet: DeviceTree.get_device_by_name returned None 21:23:44,912 DBG blivet: PartitionDevicePopulator.run: name: ddf1_BOOT01T1 ; 21:23:45,063 DBG blivet: DeviceTree.get_device_by_name: name: ddf1_BOOT01T ; incomplete: False ; hidden: False ; 21:23:45,066 DBG blivet: DeviceTree.get_device_by_name returned existing 0 B disk ddf1_BOOT01T (90) 21:23:45,068 INF blivet: ignoring 'None' format on disk that contains 'ddf1_BOOT01T1' 21:23:45,070 DBG blivet: DiskLabel.__init__: exists: True ; device: /dev/ddf1_BOOT01T ; 21:23:45,071 INF blivet: DiskLabel.parted_device: /dev/ddf1_BOOT01T does not exist 21:23:45,071 INF blivet: DiskLabel.parted_device returning None 21:23:45,072 DBG blivet: get_format('disklabel') returning DiskLabel instance with object id 95 21:23:45,074 DBG blivet: DiskDevice._set_format: ddf1_BOOT01T ; type: disklabel ; current: None ; 21:23:45,075 DBG blivet: get_format('None') returning DeviceFormat instance with object id 98 21:23:45,079 DBG blivet: DiskDevice.add_child: name: ddf1_BOOT01T ; child: ddf1_BOOT01T1 ; kids: 0 ; 21:23:45,080 DBG blivet: get_format('None') returning DeviceFormat instance with object id 99 21:23:45,083 DBG blivet: PartitionDevice._set_format: ddf1_BOOT01T1 ; type: None ; current: None ; 21:23:45,086 DBG blivet: PartitionDevice.update_sysfs_path: ddf1_BOOT01T1 ; status: False ; 21:23:45,086 ERR blivet: failed to update sysfs path for ddf1_BOOT01T1: [Errno 2] No such file or directory: '/dev/ddf1_BOOT01T1' 21:23:45,087 DBG blivet: looking up parted Partition: /dev/ddf1_BOOT01T1 21:23:45,087 INF blivet: DiskLabel.parted_device: /dev/ddf1_BOOT01T does not exist 21:23:45,088 INF blivet: DiskLabel.parted_device returning None 21:23:45,100 INF blivet: DiskLabel.parted_device: /dev/ddf1_BOOT01T does not exist 21:23:45,101 INF blivet: DiskLabel.parted_device returning None 21:23:45,103 DBG blivet: /dev/sdc is mounted on /run/install/repo 21:23:45,105 DBG blivet: /dev/sdc is mounted on /run/install/repo 21:23:45,114 DBG blivet: IGNORED: Caught exception, continuing. 21:23:45,115 DBG blivet: IGNORED: Begin exception details. 21:23:45,116 DBG blivet: IGNORED: Traceback (most recent call last): 21:23:45,119 DBG blivet: IGNORED: File "/usr/lib/python3.6/site-packages/blivet/formats/disklabel.py", line 287, in label_type 21:23:45,120 DBG blivet: IGNORED: lt = self.parted_disk.type 21:23:45,120 DBG blivet: IGNORED: AttributeError: 'NoneType' object has no attribute 'type' 21:23:45,121 DBG blivet: IGNORED: End exception details. 21:23:45,164 DBG blivet: DeviceTree.get_device_by_path: path: /dev/sdc ; incomplete: False ; hidden: False ; 21:23:45,166 DBG blivet: DeviceTree.get_device_by_path returned existing 14.59 GiB disk sdc (23) with existing iso9660 filesystem 21:23:45,174 INF blivet: DiskLabel.parted_device: /dev/ddf1_BOOT01T does not exist 21:23:45,175 INF blivet: DiskLabel.parted_device returning None 21:23:45,225 DBG blivet: IGNORED: Caught exception, continuing. 21:23:45,226 DBG blivet: IGNORED: Begin exception details. 21:23:45,226 DBG blivet: IGNORED: Traceback (most recent call last): 21:23:45,227 DBG blivet: IGNORED: File "/usr/lib/python3.6/site-packages/blivet/formats/disklabel.py", line 287, in label_type 21:23:45,227 DBG blivet: IGNORED: lt = self.parted_disk.type 21:23:45,228 DBG blivet: IGNORED: AttributeError: 'NoneType' object has no attribute 'type' 21:23:45,228 DBG blivet: IGNORED: End exception details. 21:23:45,299 INF blivet: DiskLabel.parted_device: /dev/ddf1_BOOT01T does not exist 21:23:45,301 INF blivet: DiskLabel.parted_device returning None 21:23:45,305 DBG blivet: IGNORED: Caught exception, continuing. 21:23:45,305 DBG blivet: IGNORED: Begin exception details. 21:23:45,306 DBG blivet: IGNORED: Traceback (most recent call last): 21:23:45,306 DBG blivet: IGNORED: File "/usr/lib/python3.6/site-packages/blivet/formats/disklabel.py", line 287, in label_type 21:23:45,306 DBG blivet: IGNORED: lt = self.parted_disk.type 21:23:45,307 DBG blivet: IGNORED: AttributeError: 'NoneType' object has no attribute 'type' 21:23:45,307 DBG blivet: IGNORED: End exception details. Does the wrong path of ddf device make any difference ?
This message is a reminder that Fedora 28 is nearing its end of life. On 2019-May-28 Fedora will stop maintaining and issuing updates for Fedora 28. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '28'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 28 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
I've been told that upstream mdadm now has working/stable support for DDF. I am reopening this to track a change to python-blivet to use mdadm to manage it and we'll see how far we get.
This bug appears to have been reported against 'rawhide' during the Fedora 32 development cycle. Changing version to 32.
I am still getting 16:48:49,994 CRT exception: Traceback (most recent call last): File "/usr/lib64/python3.8/site-packages/pyanaconda/threading.py", line 280, in run threading.Thread.run(self) File "/usr/lib64/python3.8/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/usr/lib64/python3.8/site-packages/pyanaconda/ui/lib/storage.py", line 92, in reset_storage sync_run_task(task_proxy) File "/usr/lib64/python3.8/site-packages/pyanaconda/modules/common/task/__init__.py", line 46, in sync_run_task task_proxy.Finish() File "/usr/lib/python3.8/site-packages/dasbus/client/handler.py", line 442, in _call_method return self._get_method_reply( File "/usr/lib/python3.8/site-packages/dasbus/client/handler.py", line 474, in _get_method_reply return self._handle_method_error(error) File "/usr/lib/python3.8/site-packages/dasbus/client/handler.py", line 496, in _handle_method_error raise exception from None dasbus.error.DBusError: 'NoneType' object has no attribute 'getPartitionByPath' when trying to install f32 on a machine with dmraid. I am not even trying to install on to the raid device. I get that error on the language selection screen. I think this is the same bug.
Created attachment 1680369 [details] anaconda.log
Created attachment 1680370 [details] dbus.log
Created attachment 1680371 [details] lvm.log
Created attachment 1680372 [details] program.log
Created attachment 1680373 [details] storage.log
it does look like the same bug to me, messages in your storage.log look very similar to the ones in #c34, and I see ddf paths in there.
I came here trying to install F32. I have several disks installed with a ZFS storage pool on them. However the drive I want to boot from is just plain. I am going to unplug the drives and see if I can get the install to proceed, then reconnect them later.
I did unplug all the storage except for the install drive and it was successful.
*** Bug 1844719 has been marked as a duplicate of this bug. ***
*** Bug 1866158 has been marked as a duplicate of this bug. ***
upstream PR: https://github.com/storaged-project/libblockdev/pull/560 updates image (for Fedora 32): https://vtrefny.fedorapeople.org/img/rhbz1583351.img I don't have a system with DDF RAID but I was able to reproduce the crash and test the fix with a DDF RAID created by mdadm so it should also work for the "real" DDF firmware RAID. Fixed version of libblockdev should be available in Fedora 33. Sorry it took us so long to triage and fix this.