Bug 1259745 - Can't start installation in Rawhide or F23 recent development images
Can't start installation in Rawhide or F23 recent development images
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: util-linux (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Karel Zak
Fedora Extras Quality Assurance
: Reopened
: 1283314 (view as bug list)
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-03 09:18 EDT by Mustafa Muhammad
Modified: 2016-02-01 21:24 EST (History)
21 users (show)

See Also:
Fixed In Version: util-linux-2.27.1-2.fc23
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-11-21 21:22:47 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
storage.log (44.86 KB, text/plain)
2015-09-03 17:23 EDT, Mustafa Muhammad
no flags Details
anaconda.log (11.54 KB, text/plain)
2015-09-03 17:23 EDT, Mustafa Muhammad
no flags Details
Hard-coded workaround to problem. (70.93 KB, text/plain)
2015-11-15 12:25 EST, Alex Markley
no flags Details

  None (edit)
Description Mustafa Muhammad 2015-09-03 09:18:27 EDT
Description of problem:
When I start the installer, I get this message and can't proceed:
failed to scan disk sda
For some reason we were unable to locate a disklabel on a disk that the kernel is reporting partitions on. It is unclear what the exact problem is. Please file a bug at http://bugzilla.redhat.com

Version-Release number of selected component (if applicable):
F23 recent development images (28-8-2015)
Rawhide images (28-8-2015)

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

I can provide logs upon request
Comment 1 David Shea 2015-09-03 10:00:29 EDT
(In reply to Mustafa Muhammad from comment #0)
> I can provide logs upon request

Yes, please.
Comment 2 Mustafa Muhammad 2015-09-03 17:22:11 EDT
This is /tmp/anaconda.log and /tmp/storage.log
If you need anything more please tell me.
Comment 3 Mustafa Muhammad 2015-09-03 17:23:19 EDT
Created attachment 1070123 [details]
storage.log
Comment 4 Mustafa Muhammad 2015-09-03 17:23:42 EDT
Created attachment 1070124 [details]
anaconda.log
Comment 5 Brian Lane 2015-09-08 14:11:23 EDT
Looks like you're trying to use a zfs disk, which isn't supported. If you want to reuse this disk, please wipe it first.
Comment 6 Mustafa Muhammad 2015-09-08 15:55:15 EDT
(In reply to Brian Lane from comment #5)
> Looks like you're trying to use a zfs disk, which isn't supported. If you
> want to reuse this disk, please wipe it first.

I am not using zfs disk, I installed from an earlier image and it all went fine.
This disk has multiple distributions installed, including F23 alpha.
Comment 7 Brian Lane 2015-09-08 18:22:32 EDT
Populator.addUdevDevice: info: {'DEVLINKS': '/dev/disk/by-id/wwn-0xbaacfbeba97e5e83 '
             '/dev/disk/by-id/ata-OCZ-VERTEX3_OCZ-T76TX6B7HIK69M2A',
 'DEVNAME': '/dev/sda',
 'DEVPATH': '/devices/pci0000:00/0000:00:1f.2/ata3/host2/target2:0:0/2:0:0:0/block/sda',
 'DEVTYPE': 'disk',
 'ID_ATA': '1',
 'ID_ATA_DOWNLOAD_MICROCODE': '1',
 'ID_ATA_FEATURE_SET_APM': '1',
 'ID_ATA_FEATURE_SET_APM_CURRENT_VALUE': '254',
 'ID_ATA_FEATURE_SET_APM_ENABLED': '1',
 'ID_ATA_FEATURE_SET_HPA': '1',
 'ID_ATA_FEATURE_SET_HPA_ENABLED': '1',
 'ID_ATA_FEATURE_SET_PM': '1',
 'ID_ATA_FEATURE_SET_PM_ENABLED': '1',
 'ID_ATA_FEATURE_SET_PUIS': '1',
 'ID_ATA_FEATURE_SET_PUIS_ENABLED': '0',
 'ID_ATA_FEATURE_SET_SECURITY': '1',
 'ID_ATA_FEATURE_SET_SECURITY_ENABLED': '0',
 'ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN': '2',
 'ID_ATA_FEATURE_SET_SECURITY_FROZEN': '1',
 'ID_ATA_FEATURE_SET_SMART': '1',
 'ID_ATA_FEATURE_SET_SMART_ENABLED': '1',
 'ID_ATA_ROTATION_RATE_RPM': '0',
 'ID_ATA_SATA': '1',
 'ID_ATA_SATA_SIGNAL_RATE_GEN1': '1',
 'ID_ATA_SATA_SIGNAL_RATE_GEN2': '1',
 'ID_ATA_WRITE_CACHE': '1',
 'ID_ATA_WRITE_CACHE_ENABLED': '1',
 'ID_BUS': 'ata',
 'ID_FS_TYPE': 'zfs_member',
 'ID_FS_USAGE': 'raid',
 'ID_FS_VERSION': '5000',
 'ID_MODEL': 'OCZ-VERTEX3',
 'ID_MODEL_ENC': 'OCZ-VERTEX3\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20\\x20',
 'ID_REVISION': '2.25',
 'ID_SERIAL': 'OCZ-VERTEX3_OCZ-T76TX6B7HIK69M2A',
 'ID_SERIAL_SHORT': 'OCZ-T76TX6B7HIK69M2A',
 'ID_TYPE': 'disk',
 'ID_WWN': '0xbaacfbeba97e5e83',
 'ID_WWN_WITH_EXTENSION': '0xbaacfbeba97e5e83',
 'MAJOR': '8',
 'MINOR': '0',
 'MPATH_SBIN_PATH': '/sbin',
 'SUBSYSTEM': 'block',
 'TAGS': ':systemd:',
 'USEC_INITIALIZED': '49686575'} ; name: sda ;

Note the ID_FS_TYPE says zfs_member.

What does the output of parted -s /dev/sda u s p free' show?
Comment 8 Mustafa Muhammad 2015-09-09 08:20:55 EDT
[root@localhost mustafa]# parted -s /dev/sda u s p free
Model: ATA OCZ-VERTEX3 (scsi)
Disk /dev/sda: 234441648s
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start       End         Size       File system  Name                  Flags
        34s         20479s      20446s     Free Space
 1      20480s      225279s     204800s    fat32        EFI System Partition  boot, esp
 2      225280s     31682559s   31457280s  ext4
 3      31682560s   63139839s   31457280s  btrfs
 4      63139840s   94597119s   31457280s
 5      94597120s   126054399s  31457280s  btrfs
 6      126054400s  147025919s  20971520s  btrfs
 7      147025920s  234420223s  87394304s  ext4         Data
        234420224s  234441614s  21391s     Free Space

It was working well until about 20-8-2015 (not sure exactly) rawhide images and after that in the F23 test images.
Comment 9 Mustafa Muhammad 2015-09-17 12:52:28 EDT
I've downloaded multiple versions of the iso to find when the problem happened.

Fedora-Live-KDE-x86_64-23-20150825.iso  works fine
Fedora-Live-KDE-x86_64-23-20150826.iso  has another problem (crash)
Fedora-Live-KDE-x86_64-23-20150827.iso  has this problem.

Hope this will help identify what change triggered this bug.
Comment 10 David Lehman 2015-09-17 13:36:40 EDT
blkid is responsible for identifying the formatting on sda
Comment 11 Philip Guyton 2015-10-28 12:16:38 EDT
Thanks all.

Also just hit this one on my sda and then sdb:-
Fedora Server 23 Beta iso

blkid -o value -s TYPE /dev/sda

zfs_member

And sure enough this drive was in the past a member of a ZFS RaidZ1 I think.
However parted shows it as msdos partition table with a 512MB ext2 and the rest as an LVM PV and it successfully boots via Grub 2.02 an older version of Ubuntu Studio.

From within this older version of Ubuntu Studio

udevadm info /dev/sda | grep PART
E: ID_PART_TABLE_TYPE=dos

There were no entries containing "FS"

older wipefs /dev/sda "... appears to contain 'dos' partition table."

And using the newer udev version in F23 there are no entries containing "PART" but we have entries containing "FS":-

udevadm info /dev/sda | grep FS
E: ID_FS_TYPE=zfs_member
E: ID_FS_USAGE=raid
E: ID_FS_VERSION=5000

so that looks about right ie version 5000 of ZFS.

wipefs (in F23 Beta installer)

wipefs /dev/sda
offset           type
-------------------------
0x1fe            dos [partition table]
0x12a1ebf000     zfs_member  [raid]

So it looks like to me that F23 Beta is picking up the second copy of the ZFS partition table at the end of the drive. (80GB drive)


Just noting here in case it helps, ie remnants of ZFS even after grub 2.02 install and repartitioning.

In case it helps anyone dropping in on this bug and be very careful and know that you will delete the entire contents of the drive:-

wipefs -n -t zfs_member /dev/sda

the above command with the -n removed and replaced with -a -f should delete all copies (the second in my case) of the zfs_member signature but it ended up deleting a whole series of zfs_member signatures and also the "dos" signature (and so removing the originally working partition table):-

On another drive (of the same original raid set) I tried the following:-
(Note this drive also had a working "msdos" partition table and a SteamOS linux install.)

wipefs -o <offset_of_zfs_member_sig> /dev/sdb

This succeeded in removing 8 bytes of a signature but wipefs devname returned another zfs_member signature and removing that by offset returned another so I haven't found a way to remove all zfs_member signatures without also loosing the dos partition table. 

If one is happy to loose all on the drive, as I was, then OK but it is suspicious that the -f zfs_member also ended up removing the "dos" signature and partition table.

Hope this helps narrow down the problem or give people a way to wipe their drives to work around this issue.
Comment 12 Alex Markley 2015-11-15 11:06:43 EST
This bug is affecting me also. I cannot start Anaconda because the error message appears: "For some reason we were unable to locate a disklabel on a disk that the kernel is reporting partitions on. It is unclear what the exact problem is. Please file a bug at http://bugzilla.redhat.com"

Sure enough, blkid incorrectly returns zfs_member as the type of the disk:

$ blkid -o value -s TYPE /dev/sda
zfs_member

Here's my partition table:

$ fdisk -l /dev/sda
Disk /dev/sda: 931.9 GiB, 1000555581440 bytes, 1954210120 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 036C1DDD-EFAC-4592-AA29-996D897A6627

Device         Start        End    Sectors   Size Type
/dev/sda1         40     409639     409600   200M EFI System
/dev/sda2     409640  186902247  186492608  88.9G Apple HFS/HFS+
/dev/sda3  186902248  188171783    1269536 619.9M Apple boot
/dev/sda4  188172288  188581887     409600   200M Apple HFS/HFS+
/dev/sda5  188581888  189605887    1024000   500M Linux filesystem
/dev/sda6  189605888  340598783  150992896    72G Linux LVM
/dev/sda7  340598784 1954210086 1613611303 769.4G Solaris /usr & Apple ZFS

NOTE: This system is a mac. The ZFS partition is very intentionally in place so that I can store my /home directory in a real (non HFS) volume and share it equally between my OS X and Linux systems.

Anaconda needs to ignore partitions that it doesn't understand and not flame out.

How do I work around this?
Comment 13 Alex Markley 2015-11-15 12:23:39 EST
For anybody following along, I have modified my live boot system's copy of /usr/lib/python3.4/site-packages/blivet/populator.py to create a crazy workaround.

This CRAZY and UGLY workaround is working for me and Anaconda is starting up now. I'll post my populator.py for edutainment purposes. If you want to adapt it for your own use, feel free, but be warned that you are playing with fire.

I will post again when the installer is completed and we shall see if my disk is ruined afterward. ;-P
Comment 14 Alex Markley 2015-11-15 12:25 EST
Created attachment 1094520 [details]
Hard-coded workaround to problem.

This hard-coded workaround will notice if sda is being detected as a zfs_member and force it to be detected as a gpt disklabel. This is probably a bad idea.
Comment 15 Alex Markley 2015-11-15 12:47:33 EST
I am happy to report that my egregious workaround did not prompt Anaconda to trash my disk, my partition table, or any of my partitions.

I was able to use the installer to select the partitions I wanted, assign mount points to them, and even selectively reformat the system partitions.

After installation I am able to confirm that the newly-installed system boots up fine, and I can still boot into the OS X system, confirming that the other partitions on the system were not affected by the adventure.


Of course the newly-installed F23 system does not support the ZFS partition out of the box, but that is as expected.

I should reiterate my position that the mere presence of a single foreign partition on a completely standard partition table should not screw up the installer this badly.
Comment 16 David Lehman 2015-11-16 13:37:28 EST
(In reply to Alex Markley from comment #12) 
> Sure enough, blkid incorrectly returns zfs_member as the type of the disk:
> 
> $ blkid -o value -s TYPE /dev/sda
> zfs_member
> 
...
> Anaconda needs to ignore partitions that it doesn't understand and not flame
> out.

You just showed that blkid has a bug (or that your disk layout isn't as standard as you think/claim). The way to resolve this issue is to get blkid or your disk layout fixed.

To be as clear as possible, the issue here is not an installer failure handling a foreign partition on a standard disklabel. You demonstrated that above. I understand this type of situation can be frustrating, but being accusatory will not help you.
Comment 17 Mustafa Muhammad 2015-11-17 12:31:21 EST
I already pointed when this change entered F23 test iso, hope this clarifies what package caused this bug (what is related and was updated in the two days)

Fedora-Live-KDE-x86_64-23-20150825.iso  works fine
Fedora-Live-KDE-x86_64-23-20150826.iso  has another problem (crash)
Fedora-Live-KDE-x86_64-23-20150827.iso  has this problem.
Comment 18 Fedora Update System 2015-11-18 06:51:16 EST
util-linux-2.27.1-2.fc23 has been submitted as an update to Fedora 23. https://bodhi.fedoraproject.org/updates/FEDORA-2015-a3e431eaa3
Comment 19 Karel Zak 2015-11-18 06:54:24 EST
(In reply to Alex Markley from comment #12)
> Sure enough, blkid incorrectly returns zfs_member as the type of the disk:
> 
> $ blkid -o value -s TYPE /dev/sda
> zfs_member

ZFS detection is complex and requires four matches (in four independent blocks) on the device... so I have doubts that any positive ZFS detection is  coincidence.

For libblkid it's pretty difficult make a decision that on your device is the valid partition table rather than ZFS if these two things do not overlap.

The solution is wipe unwanted superblocks from the device, it's also mkfs/fdisk-like utils responsibility to inform users about possible collisions  if the device already contains another superblock/PT.


Anyway, I found a bug that ZFS has been marked as RAID in the libblkid -- in this case libblkid does not probe for partition tables. This bug should be fixed in v2.27.1-2 (f23) and v2.27.1-3 (rawhide).
Comment 20 Fedora Update System 2015-11-19 10:26:51 EST
util-linux-2.27.1-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update util-linux'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-a3e431eaa3
Comment 21 Fedora Update System 2015-11-19 11:55:26 EST
util-linux-2.27.1-2.fc23 has been pushed to the Fedora 23 testing repository. If problems still persist, please make note of it in this bug report.
If you want to test the update, you can install it with
$ su -c 'dnf --enablerepo=updates-testing update util-linux'
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2015-a3e431eaa3
Comment 22 David Lehman 2015-11-19 15:39:52 EST
*** Bug 1283314 has been marked as a duplicate of this bug. ***
Comment 23 goranj 2015-11-19 15:51:50 EST
I have the same problem, but I had one ext4 partition and Fedora 22 on it. For my install I used the official 23 release. Not a beta. I formatted my disk again to ext4 and I still got the same error message
Comment 24 goranj 2015-11-20 03:08:41 EST
The same problem apears when I try to install Korora 23 Beta. Obliviously since its based on Fedora 23, but wanted to let everyone know.
Comment 25 Fedora Update System 2015-11-21 21:22:38 EST
util-linux-2.27.1-2.fc23 has been pushed to the Fedora 23 stable repository. If problems still persist, please make note of it in this bug report.
Comment 26 goranj 2015-11-23 11:47:24 EST
Sorry, I'm new to this bug submitting. Does this mean that I have to download a new ISO and the fixed util-linux-2.27.1-2.fc23 package will be included?

Thanks.

Note You need to log in before you can comment on or make changes to this bug.