Bug 2053593 - mdadm is unable to see Alder Lake IMSM array
Summary: mdadm is unable to see Alder Lake IMSM array
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: mdadm
Version: 35
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Jes Sorensen
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-02-11 15:06 UTC by Kris Knigga
Modified: 2022-12-13 16:37 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-12-13 16:37:59 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Kris Knigga 2022-02-11 15:06:32 UTC
Description of problem:

mdadm doesn't seem to know what to do with Alder Lake's version of RST.

When asked to assemble the array an error is reported on the member disks about not having OROM/EFI properties or RAID superblock.

$ sudo mdadm --verbose --assemble --scan
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/zram0
mdadm: no recogniseable superblock on /dev/dm-0
mdadm: no recogniseable superblock on /dev/sdd3
mdadm: Cannot assemble mbr metadata on /dev/sdd2
mdadm: Cannot assemble mbr metadata on /dev/sdd1
mdadm: Cannot assemble mbr metadata on /dev/sdd
mdadm: no recogniseable superblock on /dev/sdc3
mdadm: no recogniseable superblock on /dev/sdc2
mdadm: Cannot assemble mbr metadata on /dev/sdc1
mdadm: No OROM/EFI properties for /dev/sdc
mdadm: no RAID superblock on /dev/sdc
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: Cannot assemble mbr metadata on /dev/sdb1
mdadm: No OROM/EFI properties for /dev/sdb
mdadm: no RAID superblock on /dev/sdb
mdadm: no recogniseable superblock on /dev/sda3
mdadm: no recogniseable superblock on /dev/sda2
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: No arrays found in config file or automatically


However, when asked to examine the array members mdadm seems to see the RAID information.

$ sudo mdadm --examine /dev/sdc
/dev/sdc:
          Magic : Intel Raid ISM Cfg Sig.
        Version : 1.3.00
    Orig Family : efb1dfab
         Family : efb1dfab
     Generation : 0000003b
  Creation Time : Unknown
     Attributes : All supported
           UUID : 0d6d6f84:7dc5c9bf:c813c382:07d52193
       Checksum : 0ce5c877 correct
    MPB Sectors : 2
          Disks : 2
   RAID Devices : 1

  Disk01 Serial : Z302PZ9Q
          State : active
             Id : 00000002
    Usable Size : 7814031360 (3.64 TiB 4.00 TB)

[Volume1]:
       Subarray : 0
           UUID : 095000c5:c6a0218c:307e4085:704eafdd
     RAID Level : 1 <-- 1
        Members : 2 <-- 2
          Slots : [UU] <-- [UU]
    Failed disk : none
      This Slot : 1
    Sector Size : 512
     Array Size : 7814031360 (3.64 TiB 4.00 TB)
   Per Dev Size : 7814031624 (3.64 TiB 4.00 TB)
  Sector Offset : 0
    Num Stripes : 30523560
     Chunk Size : 64 KiB <-- 64 KiB
       Reserved : 0
  Migrate State : initialize
      Map State : normal <-- uninitialized
     Checkpoint : 2406092 (512)
    Dirty State : clean
     RWH Policy : off
      Volume ID : 1

  Disk00 Serial : Z302Q06H
          State : active
             Id : 00000001
    Usable Size : 7814031360 (3.64 TiB 4.00 TB)


IMSM capabilities aren't found by mdadm.

$ sudo mdadm --detail-platform
mdadm: imsm capabilities not found for controller: /sys/devices/pci0000:00/0000:00:0e.0/pci10000:e0/10000:e0:17.0 (type SATA)


Running the 'assemble' command with IMSM_NO_PLATFORM set does work as a workaround.


Version-Release number of selected component (if applicable):

* Fedora 35 fully updated (mdadm-4.2-rc2.fc35)

$ cat /etc/fedora-release 
Fedora release 35 (Thirty Five)

$ rpm -q kernel mdadm
kernel-5.14.10-300.fc35.x86_64
kernel-5.16.7-200.fc35.x86_64
mdadm-4.2-rc2.fc35.x86_64

$ uname -r
5.16.7-200.fc35.x86_64

* Also tried building the freshest version of mdadm, 4.2


How reproducible:  Very, on the limited hardware I have available.


Steps to Reproduce:
1.  mdadm --assemble --scan


Actual results:
Array isn't assembled.

Expected results:
Array is assembled.

Additional info:

The motherboard is an Asus Prime H670-PLUS D4, obviously with the H670 chipset.  The CPU is a i5-12600K.

$ sudo lspci | grep -i intel
0000:00:00.0 Host bridge: Intel Corporation Device 4648 (rev 02)
0000:00:02.0 VGA compatible controller: Intel Corporation AlderLake-S GT1 (rev 0c)
0000:00:0a.0 Signal processing controller: Intel Corporation Platform Monitoring Technology (rev 01)
0000:00:0e.0 RAID bus controller: Intel Corporation Volume Management Device NVMe RAID Controller
0000:00:14.0 USB controller: Intel Corporation Alder Lake-S PCH USB 3.2 Gen 2x2 XHCI Controller (rev 11)
0000:00:14.2 RAM memory: Intel Corporation Alder Lake-S PCH Shared SRAM (rev 11)
0000:00:15.0 Serial bus controller: Intel Corporation Alder Lake-S PCH I2C Controller #0 (rev 11)
0000:00:15.2 Serial bus controller: Intel Corporation Device 7ace (rev 11)
0000:00:16.0 Communication controller: Intel Corporation Alder Lake-S PCH HECI Controller #1 (rev 11)
0000:00:17.0 System peripheral: Intel Corporation Device 09ab
0000:00:1a.0 PCI bridge: Intel Corporation Device 7ac8 (rev 11)
0000:00:1b.0 PCI bridge: Intel Corporation Device 7ac0 (rev 11)
0000:00:1c.0 PCI bridge: Intel Corporation Device 7ab8 (rev 11)
0000:00:1c.1 PCI bridge: Intel Corporation Device 7ab9 (rev 11)
0000:00:1c.2 PCI bridge: Intel Corporation Device 7aba (rev 11)
0000:00:1c.4 PCI bridge: Intel Corporation Device 7abc (rev 11)
0000:00:1f.0 ISA bridge: Intel Corporation Device 7a85 (rev 11)
0000:00:1f.3 Audio device: Intel Corporation Alder Lake-S HD Audio Controller (rev 11)
0000:00:1f.4 SMBus: Intel Corporation Alder Lake-S PCH SMBus Controller (rev 11)
0000:00:1f.5 Serial bus controller: Intel Corporation Alder Lake-S PCH SPI Controller (rev 11)
0000:04:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
10000:e0:17.0 SATA controller: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode] (rev 11)


$ sudo lspci -vs 0000:00:0e.0
0000:00:0e.0 RAID bus controller: Intel Corporation Volume Management Device NVMe RAID Controller
	DeviceName: Onboard - Other
	Subsystem: ASUSTeK Computer Inc. Device 8694
	Flags: bus master, fast devsel, latency 0
	Memory at 6000000000 (64-bit, non-prefetchable) [size=32M]
	Memory at 82000000 (32-bit, non-prefetchable) [size=32M]
	Memory at 6003100000 (64-bit, non-prefetchable) [size=1M]
	Capabilities: [80] MSI-X: Enable+ Count=19 Masked-
	Capabilities: [90] Express Root Complex Integrated Endpoint, MSI 00
	Capabilities: [e0] Power Management version 3
	Kernel driver in use: vmd
	Kernel modules: vmd

$ head /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 151
model name	: 12th Gen Intel(R) Core(TM) i5-12600K
stepping	: 2
microcode	: 0x18
cpu MHz		: 800.073
cache size	: 20480 KB
physical id	: 0


I've read that this problem can come from being unable to read EFI vars from /sys/firmware/efi/efivars, but that doesn't seem to be the case here.

$ ll /sys/firmware/efi/efivars | egrep -i 'rst|vmd'
-rw-r--r--. 1 root root     5 Feb 11 08:40 FirstBootFlag-ec87d643-eba4-4bb5-a1e5-3f3e36b20da9
-rw-r--r--. 1 root root    12 Feb 11 08:40 IntelRstFeatures-ca2fc9c8-71e7-4f72-b433-c284456ff72b
-rw-r--r--. 1 root root    68 Feb 11 08:40 IntelVmdOsVariable-61a14fe8-4dab-4a19-b1e3-97fb23d09212
-rw-r--r--. 1 root root    42 Feb 11 08:40 RstVmdV-193dfefa-a445-4302-99d8-ef3aad1a04c6
-rw-r--r--. 1 root root    16 Feb 11 08:40 VARSTORE_OCMR_SETTINGS_AP_DATA_NAME-a3a3b874-7636-4182-ba1e-f52c584494e3
-rw-r--r--. 1 root root 19135 Feb 11 08:40 VARSTORE_OCMR_SETTINGS_NAME-a3a3b874-7636-4182-ba1e-f52c584494e3


Thanks for taking a look!

Comment 1 Kris Knigga 2022-02-11 15:11:43 UTC
As noted above, trying with IMSM_NO_PLATFORM does work.

$ sudo IMSM_NO_PLATFORM=1 mdadm --verbose --assemble --scan
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/zram0
mdadm: no recogniseable superblock on /dev/dm-0
mdadm: no recogniseable superblock on /dev/sdd3
mdadm: Cannot assemble mbr metadata on /dev/sdd2
mdadm: Cannot assemble mbr metadata on /dev/sdd1
mdadm: Cannot assemble mbr metadata on /dev/sdd
mdadm: no recogniseable superblock on /dev/sdc3
mdadm: no recogniseable superblock on /dev/sdc2
mdadm: Cannot assemble mbr metadata on /dev/sdc1
mdadm: no RAID superblock on /dev/sdb3
mdadm: no RAID superblock on /dev/sdb2
mdadm: no RAID superblock on /dev/sdb1
mdadm: no RAID superblock on /dev/sda3
mdadm: no RAID superblock on /dev/sda2
mdadm: no RAID superblock on /dev/sda1
mdadm: no IMSM anchor on /dev/sda
mdadm: Failed to load all information sections on /dev/sda
mdadm: no RAID superblock on /dev/sda
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: /dev/sdc is identified as a member of /dev/md/imsm0, slot 1.
mdadm: /dev/sdb is identified as a member of /dev/md/imsm0, slot 0.
mdadm: added /dev/sdc to /dev/md/imsm0 as 1
mdadm: added /dev/sdb to /dev/md/imsm0 as 0
mdadm: Container /dev/md/imsm0 has been assembled with 2 drives
mdadm: looking for devices for further assembly
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/zram0
mdadm: no recogniseable superblock on /dev/dm-0
mdadm: no recogniseable superblock on /dev/sdd3
mdadm: Cannot assemble mbr metadata on /dev/sdd2
mdadm: Cannot assemble mbr metadata on /dev/sdd1
mdadm: Cannot assemble mbr metadata on /dev/sdd
mdadm: no recogniseable superblock on /dev/sdc3
mdadm: no recogniseable superblock on /dev/sdc2
mdadm: Cannot assemble mbr metadata on /dev/sdc1
mdadm: /dev/sdc is busy - skipping
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: Cannot assemble mbr metadata on /dev/sdb1
mdadm: /dev/sdb is busy - skipping
mdadm: no recogniseable superblock on /dev/sda3
mdadm: no recogniseable superblock on /dev/sda2
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: looking in container /dev/md/imsm0
mdadm: found match on member /md127/0 in /dev/md/imsm0
mdadm: /dev/md/Volume1_0 has been assembled with 2 devices and started.
mdadm: looking for devices for further assembly
mdadm: /dev/sdc is busy - skipping
mdadm: /dev/sdb is busy - skipping
mdadm: looking in container /dev/md/imsm0
mdadm: member /md127/0 in /dev/md/imsm0 is already assembled
mdadm: looking for devices for further assembly
mdadm: no recogniseable superblock on /dev/md126p3
mdadm: no recogniseable superblock on /dev/md126p2
mdadm: Cannot assemble mbr metadata on /dev/md126p1
mdadm: Cannot assemble mbr metadata on /dev/md/Volume1_0
mdadm: no recogniseable superblock on /dev/zram0
mdadm: no recogniseable superblock on /dev/dm-0
mdadm: no recogniseable superblock on /dev/sdd3
mdadm: Cannot assemble mbr metadata on /dev/sdd2
mdadm: Cannot assemble mbr metadata on /dev/sdd1
mdadm: Cannot assemble mbr metadata on /dev/sdd
mdadm: no recogniseable superblock on /dev/sdc3
mdadm: no recogniseable superblock on /dev/sdc2
mdadm: Cannot assemble mbr metadata on /dev/sdc1
mdadm: /dev/sdc is busy - skipping
mdadm: no recogniseable superblock on /dev/sdb3
mdadm: no recogniseable superblock on /dev/sdb2
mdadm: Cannot assemble mbr metadata on /dev/sdb1
mdadm: /dev/sdb is busy - skipping
mdadm: no recogniseable superblock on /dev/sda3
mdadm: no recogniseable superblock on /dev/sda2
mdadm: Cannot assemble mbr metadata on /dev/sda1
mdadm: Cannot assemble mbr metadata on /dev/sda
mdadm: cannot open device /dev/sr1: No medium found
mdadm: cannot open device /dev/sr0: No medium found
mdadm: looking in container /dev/md/imsm0
mdadm: member /md127/0 in /dev/md/imsm0 is already assembled


$ cat /proc/mdstat 
Personalities : [raid1] 
md126 : active raid1 sdb[1] sdc[0]
      3907015680 blocks super external:/md127/0 [2/2] [UU]
      [===>.................]  resync = 15.8% (618772736/3907015680) finish=331.1min speed=165481K/sec
      
md127 : inactive sdb[1](S) sdc[0](S)
      5808 blocks super external:imsm
       
unused devices: <none>

Comment 2 Kris Knigga 2022-02-28 19:50:28 UTC
I've rebuilt 4.2 with DEBUG turned on and have learned slightly more.

```
$ sudo mdadm --detail-platform
mdadm: scan: cannot find pciExpDataStruct 
mdadm: detail_platform_imsm: imsm capabilities not found for controller: /sys/devices/pci0000:00/0000:00:0e.0/pci10000:e0/10000:e0:17.0 (type SATA)
```

Comment 3 XiaoNi 2022-03-07 03:01:39 UTC
Hi Kris

It's better to report your problem to upstream(linux-raid.org). Intel people are experts about IMSM raid.
From your comments, it looks like you don't enable imsm in bios. Can you create raid device in bios? Intel provides 
a tool to create imsm raid device when machine boots.

Thanks
Xiao

Comment 4 Kris Knigga 2022-03-09 15:13:40 UTC
Xiao,


IMSM is enabled in the BIOS according to the manufacturer's instructions and a RAID array has been created.

Thanks for the referral to upstream.  I'll contact that mailing list.

Kris

Comment 5 Ben Cotton 2022-11-29 17:52:16 UTC
This message is a reminder that Fedora Linux 35 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
'version' of '35'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, change the 'version' 
to a later Fedora Linux version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora Linux 35 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora Linux, you are encouraged to change the 'version' to a later version
prior to this bug being closed.

Comment 6 Ben Cotton 2022-12-13 16:37:59 UTC
Fedora Linux 35 entered end-of-life (EOL) status on 2022-12-13.

Fedora Linux 35 is no longer maintained, which means that it
will not receive any further security or bug fix updates. As a result we
are closing this bug.

If you can reproduce this bug against a currently maintained version of Fedora Linux
please feel free to reopen this bug against that version. Note that the version
field may be hidden. Click the "Show advanced fields" button if you do not see
the version field.

If you are unable to reopen this bug, please file a new report against an
active release.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.