Bug 2031810
Summary: | Symlinks to NVMe drives are missing in /dev/disk/by-path | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | Red Hat Enterprise Linux 9 | Reporter: | Lukas Musil <lmusil> | ||||||||||
Component: | systemd | Assignee: | David Tardon <dtardon> | ||||||||||
Status: | CLOSED ERRATA | QA Contact: | Frantisek Sumsal <fsumsal> | ||||||||||
Severity: | medium | Docs Contact: | |||||||||||
Priority: | unspecified | ||||||||||||
Version: | 9.0 | CC: | bugproxy, david.chang, dtardon, edemeule, jamacku, jhladky, jjurca, msekleta, rudnev.vv, smahanga, spetrovi, systemd-maint-list, trinh.dao, yizhan, yuwatana | ||||||||||
Target Milestone: | rc | Keywords: | FutureFeature, TestOnly, Triaged | ||||||||||
Target Release: | --- | ||||||||||||
Hardware: | x86_64 | ||||||||||||
OS: | Linux | ||||||||||||
Whiteboard: | |||||||||||||
Fixed In Version: | systemd-252-3.el9 | Doc Type: | If docs needed, set a value | ||||||||||
Doc Text: | Story Points: | --- | |||||||||||
Clone Of: | |||||||||||||
: | 2217131 (view as bug list) | Environment: | |||||||||||
Last Closed: | 2023-05-09 08:21:58 UTC | Type: | Bug | ||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||
Documentation: | --- | CRM: | |||||||||||
Verified Versions: | Category: | --- | |||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
Embargoed: | |||||||||||||
Bug Depends On: | 2138081 | ||||||||||||
Bug Blocks: | 2099868, 2124964, 2217131 | ||||||||||||
Attachments: |
|
Description
Lukas Musil
2021-12-13 13:27:43 UTC
Hello. We have the same problem with Dell Ent NVMe AGN MU U.2 1.6TB (Samsung PM1733). Our setup has: - 4x NVMe Dell Express Flash NVMe P4610 3.2TB SFF (Intel P4610 3.2TB) - 7x NVMe Dell Ent NVMe AGN MU U.2 1.6TB (Samsung PM1733) - SATA 480 SSD # nvme list Node SN Model Namespace Usage Format FW Rev --------------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme0n1 S61ENA0R100937 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 /dev/nvme1n1 S61ENA0R100933 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 /dev/nvme10n1 PHLN040100CD3P2BGN Dell Express Flash NVMe P4610 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 /dev/nvme2n1 S61ENA0R100766 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 /dev/nvme3n1 S61ENA0R100935 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 /dev/nvme4n1 S61ENA0R100938 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 /dev/nvme5n1 S61ENE0NB02363 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.0.2 /dev/nvme6n1 S61ENA0R100931 Dell Ent NVMe AGN MU U.2 1.6TB 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 /dev/nvme7n1 PHLN040100X13P2BGN Dell Express Flash NVMe P4610 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 /dev/nvme8n1 PHLN0401019B3P2BGN Dell Express Flash NVMe P4610 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 /dev/nvme9n1 PHLN0401011L3P2BGN Dell Express Flash NVMe P4610 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 /dev/disk/by-path/ doesn't contain symlinks for "7x NVMe Dell Ent NVMe AGN MU U.2 1.6TB": # ls -al /dev/disk/by-path/ total 0 drwxr-xr-x 2 root root 200 Jul 26 13:01 . drwxr-xr-x 7 root root 140 Jul 26 13:01 .. lrwxrwxrwx 1 root root 9 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0 -> ../../sda lrwxrwxrwx 1 root root 10 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0-part1 -> ../../sda1 lrwxrwxrwx 1 root root 10 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0-part2 -> ../../sda2 lrwxrwxrwx 1 root root 10 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0-part3 -> ../../sda3 lrwxrwxrwx 1 root root 13 Jul 26 13:01 pci-0000:da:00.0-nvme-1 -> ../../nvme7n1 lrwxrwxrwx 1 root root 13 Jul 26 13:01 pci-0000:db:00.0-nvme-1 -> ../../nvme8n1 lrwxrwxrwx 1 root root 13 Jul 26 13:01 pci-0000:dc:00.0-nvme-1 -> ../../nvme9n1 lrwxrwxrwx 1 root root 14 Jul 26 13:01 pci-0000:dd:00.0-nvme-1 -> ../../nvme10n1 As I can see, the problem repeats in all distros which use kernel v4.19 and newer. I noticed that a kernel creates DEVPATH `/sys/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0/nvme0c0n1` for the problem drives, which should creates in multipath mode, according to kernel sources (https://github.com/torvalds/linux/blob/v4.19/drivers/nvme/host/multipath.c#L41). So I disabled NVMe multipath with "nvme-core.multipath=N" kernel cmdline parameter and it looks like a workaround: # ls -al /dev/disk/by-path/ total 0 drwxr-xr-x 2 root root 340 Jul 26 10:59 . drwxr-xr-x 7 root root 140 Jul 26 10:59 .. lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:3d:00.0-nvme-1 -> ../../nvme0n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:3f:00.0-nvme-1 -> ../../nvme1n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:40:00.0-nvme-1 -> ../../nvme2n1 lrwxrwxrwx 1 root root 9 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0 -> ../../sda lrwxrwxrwx 1 root root 10 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0-part1 -> ../../sda1 lrwxrwxrwx 1 root root 10 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0-part2 -> ../../sda2 lrwxrwxrwx 1 root root 10 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0-part3 -> ../../sda3 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b1:00.0-nvme-1 -> ../../nvme3n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b2:00.0-nvme-1 -> ../../nvme4n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b3:00.0-nvme-1 -> ../../nvme5n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b4:00.0-nvme-1 -> ../../nvme6n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:da:00.0-nvme-1 -> ../../nvme7n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:db:00.0-nvme-1 -> ../../nvme8n1 lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:dc:00.0-nvme-1 -> ../../nvme9n1 lrwxrwxrwx 1 root root 14 Jul 26 10:59 pci-0000:dd:00.0-nvme-1 -> ../../nvme10n1 Additional info: # multipath -e -ll Segmentation fault (core dumped) # nvme list-subsys nvme-subsys0 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100937 \ +- nvme0 pcie 0000:3d:00.0 live nvme-subsys1 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100933 \ +- nvme1 pcie 0000:3f:00.0 live nvme-subsys10 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN040100CD3P2BGN Dell Express Flash NVMe P4610 3.2TB SFF \ +- nvme10 pcie 0000:dd:00.0 live nvme-subsys2 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100766 \ +- nvme2 pcie 0000:40:00.0 live nvme-subsys3 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100935 \ +- nvme3 pcie 0000:b1:00.0 live nvme-subsys4 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100938 \ +- nvme4 pcie 0000:b2:00.0 live nvme-subsys5 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENE0NB02363 \ +- nvme5 pcie 0000:b3:00.0 live nvme-subsys6 - NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100931 \ +- nvme6 pcie 0000:b4:00.0 live nvme-subsys7 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN040100X13P2BGN Dell Express Flash NVMe P4610 3.2TB SFF \ +- nvme7 pcie 0000:da:00.0 live nvme-subsys8 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN0401019B3P2BGN Dell Express Flash NVMe P4610 3.2TB SFF \ +- nvme8 pcie 0000:db:00.0 live nvme-subsys9 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN0401011L3P2BGN Dell Express Flash NVMe P4610 3.2TB SFF \ +- nvme9 pcie 0000:dc:00.0 live # nvme list-ctrl /dev/nvme0 num of ctrls present: 2 [ 0]:0x41 [ 1]:0x42 # udevadm info --query=all --name=/dev/nvme0n1 P: /devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1 N: nvme0n1 L: 0 S: disk/by-id/nvme-Dell_Ent_NVMe_AGN_MU_U.2_1.6TB_S61ENA0R100937 S: disk/by-id/nvme-eui.36314530521009370025384100000004 E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1 E: DEVNAME=/dev/nvme0n1 E: DEVTYPE=disk E: DISKSEQ=10 E: MAJOR=259 E: MINOR=1 E: SUBSYSTEM=block E: USEC_INITIALIZED=4344756 E: MPATH_SBIN_PATH=/sbin E: DM_MULTIPATH_DEVICE_PATH=0 E: ID_SERIAL_SHORT=S61ENA0R100937 E: ID_WWN=eui.36314530521009370025384100000004 E: ID_MODEL=Dell Ent NVMe AGN MU U.2 1.6TB E: ID_REVISION=2.4.0 E: ID_SERIAL=Dell_Ent_NVMe_AGN_MU_U.2_1.6TB_S61ENA0R100937 E: DEVLINKS=/dev/disk/by-id/nvme-Dell_Ent_NVMe_AGN_MU_U.2_1.6TB_S61ENA0R100937 /dev/disk/by-id/nvme-eui.36314530521009370025384100000004 E: TAGS=:systemd: E: CURRENT_TAGS=:systems: (In reply to Volodymyr Rudniev from comment #18) > Hello. > > We have the same problem with Dell Ent NVMe AGN MU U.2 1.6TB (Samsung > PM1733). > Our setup has: > - 4x NVMe Dell Express Flash NVMe P4610 3.2TB SFF (Intel P4610 3.2TB) > - 7x NVMe Dell Ent NVMe AGN MU U.2 1.6TB (Samsung PM1733) > - SATA 480 SSD > > # nvme list > Node SN Model > Namespace Usage Format FW Rev > --------------------- -------------------- > ---------------------------------------- --------- > -------------------------- ---------------- -------- > /dev/nvme0n1 S61ENA0R100937 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 > /dev/nvme1n1 S61ENA0R100933 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 > /dev/nvme10n1 PHLN040100CD3P2BGN Dell Express Flash NVMe P4610 > 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 > /dev/nvme2n1 S61ENA0R100766 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 > /dev/nvme3n1 S61ENA0R100935 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 > /dev/nvme4n1 S61ENA0R100938 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 > /dev/nvme5n1 S61ENE0NB02363 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.0.2 > /dev/nvme6n1 S61ENA0R100931 Dell Ent NVMe AGN MU U.2 1.6TB > 1 1.60 TB / 1.60 TB 512 B + 0 B 2.4.0 > /dev/nvme7n1 PHLN040100X13P2BGN Dell Express Flash NVMe P4610 > 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 > /dev/nvme8n1 PHLN0401019B3P2BGN Dell Express Flash NVMe P4610 > 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 > /dev/nvme9n1 PHLN0401011L3P2BGN Dell Express Flash NVMe P4610 > 3.2TB SFF 1 3.20 TB / 3.20 TB 512 B + 0 B VDV1DP25 > > > /dev/disk/by-path/ doesn't contain symlinks for "7x NVMe Dell Ent NVMe AGN > MU U.2 1.6TB": > > # ls -al /dev/disk/by-path/ > total 0 > drwxr-xr-x 2 root root 200 Jul 26 13:01 . > drwxr-xr-x 7 root root 140 Jul 26 13:01 .. > lrwxrwxrwx 1 root root 9 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0 -> > ../../sda > lrwxrwxrwx 1 root root 10 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0-part1 > -> ../../sda1 > lrwxrwxrwx 1 root root 10 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0-part2 > -> ../../sda2 > lrwxrwxrwx 1 root root 10 Jul 26 13:01 pci-0000:86:00.0-scsi-0:2:0:0-part3 > -> ../../sda3 > lrwxrwxrwx 1 root root 13 Jul 26 13:01 pci-0000:da:00.0-nvme-1 -> > ../../nvme7n1 > lrwxrwxrwx 1 root root 13 Jul 26 13:01 pci-0000:db:00.0-nvme-1 -> > ../../nvme8n1 > lrwxrwxrwx 1 root root 13 Jul 26 13:01 pci-0000:dc:00.0-nvme-1 -> > ../../nvme9n1 > lrwxrwxrwx 1 root root 14 Jul 26 13:01 pci-0000:dd:00.0-nvme-1 -> > ../../nvme10n1 > > > As I can see, the problem repeats in all distros which use kernel v4.19 and > newer. > > I noticed that a kernel creates DEVPATH > `/sys/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0/nvme0c0n1` for the > problem drives, which should creates in multipath mode, according to kernel > sources > (https://github.com/torvalds/linux/blob/v4.19/drivers/nvme/host/multipath. > c#L41). So I disabled NVMe multipath with "nvme-core.multipath=N" kernel > cmdline parameter and it looks like a workaround: > > # ls -al /dev/disk/by-path/ > total 0 > drwxr-xr-x 2 root root 340 Jul 26 10:59 . > drwxr-xr-x 7 root root 140 Jul 26 10:59 .. > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:3d:00.0-nvme-1 -> > ../../nvme0n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:3f:00.0-nvme-1 -> > ../../nvme1n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:40:00.0-nvme-1 -> > ../../nvme2n1 > lrwxrwxrwx 1 root root 9 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0 -> > ../../sda > lrwxrwxrwx 1 root root 10 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0-part1 > -> ../../sda1 > lrwxrwxrwx 1 root root 10 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0-part2 > -> ../../sda2 > lrwxrwxrwx 1 root root 10 Jul 26 10:59 pci-0000:86:00.0-scsi-0:2:0:0-part3 > -> ../../sda3 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b1:00.0-nvme-1 -> > ../../nvme3n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b2:00.0-nvme-1 -> > ../../nvme4n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b3:00.0-nvme-1 -> > ../../nvme5n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:b4:00.0-nvme-1 -> > ../../nvme6n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:da:00.0-nvme-1 -> > ../../nvme7n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:db:00.0-nvme-1 -> > ../../nvme8n1 > lrwxrwxrwx 1 root root 13 Jul 26 10:59 pci-0000:dc:00.0-nvme-1 -> > ../../nvme9n1 > lrwxrwxrwx 1 root root 14 Jul 26 10:59 pci-0000:dd:00.0-nvme-1 -> > ../../nvme10n1 > > > Additional info: > # multipath -e -ll > Segmentation fault (core dumped) > > # nvme list-subsys > nvme-subsys0 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100937 > \ > +- nvme0 pcie 0000:3d:00.0 live > nvme-subsys1 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100933 > \ > +- nvme1 pcie 0000:3f:00.0 live > nvme-subsys10 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN040100CD3P2BGN > Dell Express Flash NVMe P4610 3.2TB SFF > \ > +- nvme10 pcie 0000:dd:00.0 live > nvme-subsys2 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100766 > \ > +- nvme2 pcie 0000:40:00.0 live > nvme-subsys3 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100935 > \ > +- nvme3 pcie 0000:b1:00.0 live > nvme-subsys4 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100938 > \ > +- nvme4 pcie 0000:b2:00.0 live > nvme-subsys5 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENE0NB02363 > \ > +- nvme5 pcie 0000:b3:00.0 live > nvme-subsys6 - > NQN=nqn.1994-11.com.samsung:nvme:PM1733:2.5-inch:S61ENA0R100931 > \ > +- nvme6 pcie 0000:b4:00.0 live > nvme-subsys7 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN040100X13P2BGN > Dell Express Flash NVMe P4610 3.2TB SFF > \ > +- nvme7 pcie 0000:da:00.0 live > nvme-subsys8 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN0401019B3P2BGN > Dell Express Flash NVMe P4610 3.2TB SFF > \ > +- nvme8 pcie 0000:db:00.0 live > nvme-subsys9 - NQN=nqn.2014.08.org.nvmexpress:80861028PHLN0401011L3P2BGN > Dell Express Flash NVMe P4610 3.2TB SFF > \ > +- nvme9 pcie 0000:dc:00.0 live > > # nvme list-ctrl /dev/nvme0 > num of ctrls present: 2 > [ 0]:0x41 > [ 1]:0x42 > > > # udevadm info --query=all --name=/dev/nvme0n1 > P: /devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1 > N: nvme0n1 > L: 0 > S: disk/by-id/nvme-Dell_Ent_NVMe_AGN_MU_U.2_1.6TB_S61ENA0R100937 > S: disk/by-id/nvme-eui.36314530521009370025384100000004 > E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys0/nvme0n1 > E: DEVNAME=/dev/nvme0n1 > E: DEVTYPE=disk > E: DISKSEQ=10 > E: MAJOR=259 > E: MINOR=1 > E: SUBSYSTEM=block > E: USEC_INITIALIZED=4344756 > E: MPATH_SBIN_PATH=/sbin > E: DM_MULTIPATH_DEVICE_PATH=0 > E: ID_SERIAL_SHORT=S61ENA0R100937 > E: ID_WWN=eui.36314530521009370025384100000004 > E: ID_MODEL=Dell Ent NVMe AGN MU U.2 1.6TB > E: ID_REVISION=2.4.0 > E: ID_SERIAL=Dell_Ent_NVMe_AGN_MU_U.2_1.6TB_S61ENA0R100937 > E: > DEVLINKS=/dev/disk/by-id/nvme-Dell_Ent_NVMe_AGN_MU_U.2_1.6TB_S61ENA0R100937 > /dev/disk/by-id/nvme-eui.36314530521009370025384100000004 > E: TAGS=:systemd: > E: CURRENT_TAGS=:systems: Wondering if this is how it is designed. Here is the explanation from upstream ~~~ If you're using nvme native multipathing, and your namespace reports that it is multipath capable (ID_NS.NMIC), then the driver will set up the virtual device for the visible block device. If your namespace isn't multipath capable, you will only get the physical device. That's just for pci, though; fabrics targets always link to a virtual nvme subsystem device. In the case you have a multipath namespace, the driver will also create "hidden" block devices for each controller path that it found. As an exampe, if you have multipath nvme /sys/block/nvme0n1, there should be a /sys/block/nvme0c0n1, which should link to a physical device in sysfs for pci. ~~~ Thanks Gopal Hi, could anyone test the upstream PR? https://github.com/systemd/systemd/pull/24766 I merely check RHBZ. Please feedback on the PR page, instead of here. Thank you. Sorry, the upstream PR has a bug. Follow-up is here https://github.com/systemd/systemd/pull/24946 *** Bug 2124964 has been marked as a duplicate of this bug. *** ------- Comment From wenxiong.com 2022-09-08 11:55 EDT------- I didn't submit/upstream the patch yet. Can systemd team in Redhat review the patch or do the pull request for systemd github? This is general issue in rhel87 and rhel91. Thanks, Wendy ------- Comment From wenxiong.com 2022-09-19 13:31 EDT------- Pull Request to github/systemd. https://github.com/systemd/systemd/pull/24748 ------- Comment From wenxiong.com 2022-09-22 19:56 EDT------- >>Possible replacement: https://github.com/systemd/systemd/pull/24766 [root@ltcrain119-lp3 by-path]# pwd /dev/disk/by-path [root@ltcrain119-lp3 by-path]# ls -l|more lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-1 -> ../../nvme0n1 lrwxrwxrwx. 1 root root 14 Sep 22 18:45 pci-0581:10:00.0-nvme-10 -> ../../nvme0n10 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-2 -> ../../nvme0n2 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-3 -> ../../nvme0n3 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-4 -> ../../nvme0n4 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-5 -> ../../nvme0n5 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-6 -> ../../nvme0n6 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-7 -> ../../nvme0n7 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-8 -> ../../nvme0n8 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0581:10:00.0-nvme-9 -> ../../nvme0n9 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0582:18:00.0-nvme-1 -> ../../nvme1n1 lrwxrwxrwx. 1 root root 14 Sep 22 18:45 pci-0582:18:00.0-nvme-10 -> ../../nvme1n10 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0582:18:00.0-nvme-2 -> ../../nvme1n2 lrwxrwxrwx. 1 root root 13 Sep 22 18:45 pci-0582:18:00.0-nvme-3 -> ../../nvme1n3 It works! Thanks! ------- Comment From wenxiong.com 2022-10-03 11:16 EDT------- >>Possible replacement: https://github.com/systemd/systemd/pull/24766 Please pull the patch into rhel91. Thanks, Wendy ------- Comment From wenxiong.com 2022-10-10 14:49 EDT------- Tested https://github.com/systemd/systemd/pull/24766 + 24946 on powerpc, works fine. [root@ltcrain119-lp3 by-path]# pwd /dev/disk/by-path lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-1 -> ../../nvme0n1 lrwxrwxrwx. 1 root root 14 Oct 10 13:44 pci-0581:10:00.0-nvme-10 -> ../../nvme0n 10 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-2 -> ../../nvme0n2 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-3 -> ../../nvme0n3 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-4 -> ../../nvme0n4 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-5 -> ../../nvme0n5 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-6 -> ../../nvme0n6 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-7 -> ../../nvme0n7 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-8 -> ../../nvme0n8 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0581:10:00.0-nvme-9 -> ../../nvme0n9 lrwxrwxrwx. 1 root root 13 Oct 10 13:44 pci-0582:18:00.0-nvme-1 -> ../../nvme1n1 lrwxrwxrwx. 1 root root 14 Oct 10 13:44 pci-0582:18:00.0-nvme-10 -> ../../nvme1n ------- Comment From wenxiong.com 2022-10-13 14:07 EDT------- Can you include these PR in rhel91? https://github.com/systemd/systemd/pull/24946 Thanks, Wendy ------- Comment From wenxiong.com 2022-10-24 10:10 EDT-------
>>I found there is a "virtual" in DEVPATH for Samsung NVMe SSD on RHEL9; is this correct?
yes. Because nvme_core.multipath=y is set in rhel9 by default.
The output of udevadm looks good.
Created attachment 1919991 [details]
patched systemd-udev
Created attachment 1919992 [details]
patched systemd
Created attachment 1919994 [details]
patch for systemd/udev
*** Bug 2133681 has been marked as a duplicate of this bug. *** ------- Comment From wenxiong.com 2023-01-10 13:09 EDT------- [root@ltcd41-lp11 by-path]# ls -l total 0 lrwxrwxrwx. 1 root root 13 Jan 9 17:33 nvme-236 -> ../../nvme2n1 lrwxrwxrwx. 1 root root 13 Jan 9 17:33 nvme-237 -> ../../nvme2n2 lrwxrwxrwx. 1 root root 13 Jan 9 17:33 nvme-619 -> ../../nvme2n3 lrwxrwxrwx. 1 root root 13 Jan 9 17:33 nvme-620 -> ../../nvme2n4 lrwxrwxrwx. 1 root root 15 Jan 9 17:33 nvme-620-part1 -> ../../nvme2n4p1 lrwxrwxrwx. 1 root root 15 Jan 9 17:33 nvme-620-part2 -> ../../nvme2n4p2 lrwxrwxrwx. 1 root root 15 Jan 9 17:33 nvme-620-part3 -> ../../nvme2n4p3 lrwxrwxrwx. 1 root root 15 Jan 9 17:33 nvme-620-part4 -> ../../nvme2n4p4 lrwxrwxrwx. 1 root root 15 Jan 9 17:33 nvme-620-part5 -> ../../nvme2n4p5 lrwxrwxrwx. 1 root root 15 Jan 9 17:33 nvme-620-part6 -> ../../nvme2n4p6 [root@ltcd41-lp11 by-path]# uname -r 5.14.0-226.el9.ppc64le [root@ltcd41-lp11 by-path]# pwd /dev/disk/by-path This issue should be fixed via rebase to systemd v252. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (systemd bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2023:2531 |