Bug 2136906

I am encountering the issue in two cases:

DASDs in PowerVM LPARs (they are presented as SCSI disks, but without WWIDs)
DASDs in VMs on S/390x (those are not SCSI)

The first case was originally mentioned in bz2119473.

/sys/class/block/sda is a symlink to the same target as /sys/dev/block/8:0 (both point to ../../devices/vio/30000003/host0/target0:0:1/0:0:1:0/block/sda ). So adding /sys/class/block/sda when /sys/dev/block/major:minor is already used does not seem to have an advantage. The proposed /sys/class/block/sda/serial does not exist there (nor does serial exist under the device subdirectory). /sys/class/block/sda/device/vpd_pg0 exists, but does not seem to contain an ID:
# od -a /sys/class/block/sda/device/vpd_pg0
0000000 nul nul nul eot nul nul   1   z
0000010
(it apparently contains the list of supported VPD pages).
/sys/class/block/sda/device/vpd_pg80 would do the trick, it contains a serial identifier that seems to be reasonably unique and it matches what is reported by lsblk:

# lsblk --list -o +serial
NAME     MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS SERIAL
sda        8:0    0  40G  0 disk             00fac60100004c000000017bead7ca41.8
sdb        8:16   0   5G  0 disk             00fac60100004c000000017bead7ca41.39
sdc        8:32   0   5G  0 disk             00fac60100004c000000017bead7ca41.40
sdd        8:48   0   5G  0 disk             00fac60100004c000000017bead7ca41.41
sde        8:64   0   5G  0 disk             00fac60100004c000000017bead7ca41.42


Not sure what to do in the second case . /etc/lvm/devices/system.devices has this:

# LVM uses devices listed in this file.
# Created by LVM command lvmdevices pid 2746 at Mon Oct 24 04:35:52 2022
VERSION=1.1.2
IDTYPE=devname IDNAME=/dev/dasda2 DEVNAME=/dev/dasda2 PVID=sc9k1vMNcQpyWTeCdC8zQIKLApSCQQtN PART=2

and SCSI commands to retrieve an ID can't be used, as it is not a SCSI disk. /dev/disk/by-id has this:

lrwxrwxrwx. 1 root root 11 Oct 24 00:38 ccw-0X0120 -> ../../dasda
lrwxrwxrwx. 1 root root 11 Oct 24 00:38 ccw-IBM.750000000CWR71.71b6.13 -> ../../dasda
lrwxrwxrwx. 1 root root 11 Oct 24 00:38 ccw-IBM.750000000CWR71.71b6.13.000000000000d9610000000000000000 -> ../../dasda

so there are some identifiers that could potentially be used, but I don't know how to retrieve them.

"udevadm info /dev/dasda" says:

P: /devices/css0/0.0.0004/0.0.0120/block/dasda
N: dasda
L: 0
S: disk/by-path/ccw-0.0.0120
E: DEVPATH=/devices/css0/0.0.0004/0.0.0120/block/dasda
E: DEVNAME=/dev/dasda
E: DEVTYPE=disk
E: DISKSEQ=1
E: MAJOR=94
E: MINOR=0
E: SUBSYSTEM=block
E: USEC_INITIALIZED=3621300
E: ID_PATH=ccw-0.0.0120
E: ID_PATH_TAG=ccw-0_0_0120
E: DEVLINKS=/dev/disk/by-path/ccw-0.0.0120
E: TAGS=:systemd:
E: CURRENT_TAGS=:systemd:

and lsblk --list -o +serial does not report any serial number. There is no "serial" under  /sys/block/dasda/ nor under  /sys/block/dasda/device.

Update: I see where are the IDs on z/VM DASDs (s390x) coming from

# cat /sys/block/dasda/device/uid
IBM.750000000CWR71.71b6.13.000000000000d9610000000000000000
# cat /sys/block/dasda/device/vendor
IBM

Concerning SCSI disks and page 0x80, as lsblk is already capable of showing the serial number properly, would it be possible to take the value from there, instead of reimplementing it? I suspect the value is coming from udev:
# udevadm info /dev/sda
P: /devices/vio/30000003/host0/target0:0:1/0:0:1:0/block/sda
N: sda
L: 0
S: disk/by-id/scsi-SAIX_VDASD_00fac60100004c000000017bead7ca41.8
E: DEVPATH=/devices/vio/30000003/host0/target0:0:1/0:0:1:0/block/sda
E: DEVNAME=/dev/sda
E: DEVTYPE=disk
E: DISKSEQ=1
E: MAJOR=8
E: MINOR=0
E: SUBSYSTEM=block
E: USEC_INITIALIZED=15929949
E: ID_SCSI=1
E: ID_VENDOR=AIX
E: ID_VENDOR_ENC=AIX\x20\x20\x20\x20\x20
E: ID_MODEL=VDASD
E: ID_MODEL_ENC=VDASD\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20
E: ID_REVISION=0001
E: ID_TYPE=disk
E: ID_SERIAL=SAIX_VDASD_00fac60100004c000000017bead7ca41.8
E: ID_SERIAL_SHORT=00fac60100004c000000017bead7ca41.8
E: ID_BUS=scsi
E: ID_PART_TABLE_UUID=3f511404
E: ID_PART_TABLE_TYPE=dos
E: SCSI_TPGS=0
E: SCSI_TYPE=disk
E: SCSI_VENDOR=AIX
E: SCSI_VENDOR_ENC=AIX\x20\x20\x20\x20\x20
E: SCSI_MODEL=VDASD
E: SCSI_MODEL_ENC=VDASD\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20
E: SCSI_REVISION=0001
E: ID_SCSI_INQUIRY=1
E: SCSI_IDENT_SERIAL=00fac60100004c000000017bead7ca41.8
E: ID_SCSI_SERIAL=00fac60100004c000000017bead7ca41.8
E: DEVLINKS=/dev/disk/by-id/scsi-SAIX_VDASD_00fac60100004c000000017bead7ca41.8
E: TAGS=:systemd:
E: CURRENT_TAGS=:systemd:

(In reply to Pavel Cahyna from comment #2)
> The proposed /sys/class/block/sda/serial does not exist there 

virtio devices use /sys/class/block/vda/serial, so I'm only checking this for /dev/vd*.

> /sys/class/block/sda/device/vpd_pg0 exists, but does not seem to contain an
> (it apparently contains the list of supported VPD pages).

right, pg0 is not useful

> /sys/class/block/sda/device/vpd_pg80 would do the trick, it contains a
> serial identifier that seems to be reasonably unique and it matches what is
> reported by lsblk:

This will be the most useful addition.  The "reasonable" caveat can be a problem.  I have some iscsi disks that all report the same serial number (although they do have unique wwids which are always used first, so it's not an actual problem.)  I'm currently working on the ability to find the right device when the serial numbers are not unique (which is somewhat like handling the devname idtype.)

> Not sure what to do in the second case . /etc/lvm/devices/system.devices has
> this:
> IDTYPE=devname IDNAME=/dev/dasda2 DEVNAME=/dev/dasda2
> PVID=sc9k1vMNcQpyWTeCdC8zQIKLApSCQQtN PART=2

The devname idtype is ok, although it's not optimal when the names change.  I assume that these dasd names change like sd device names?  If so, we'll benefit from using the /sys/block/dasda/device/uid value.  I'll add a new device_id_type for these, e.g. IDTYPE=dasd_uid, which reads that sysfs uid path.  I have anticipated doing this for various unique device types that I didn't have access to during the original development.

> as lsblk is already capable of showing the serial number properly, would it be possible to take the value from there

device ids are always read directly from the sysfs source (except devname when there's no proper id.)

(In reply to David Teigland from comment #4)
> (In reply to Pavel Cahyna from comment #2)
> > The proposed /sys/class/block/sda/serial does not exist there 
> 
> virtio devices use /sys/class/block/vda/serial, so I'm only checking this
> for /dev/vd*.
> 
> > /sys/class/block/sda/device/vpd_pg0 exists, but does not seem to contain an
> > (it apparently contains the list of supported VPD pages).
> 
> right, pg0 is not useful
> 
> > /sys/class/block/sda/device/vpd_pg80 would do the trick, it contains a
> > serial identifier that seems to be reasonably unique and it matches what is
> > reported by lsblk:
> 
> This will be the most useful addition.  The "reasonable" caveat can be a
> problem.  I have some iscsi disks that all report the same serial number
> (although they do have unique wwids which are always used first, so it's not
> an actual problem.)  I'm currently working on the ability to find the right
> device when the serial numbers are not unique (which is somewhat like
> handling the devname idtype.)

This problem has been encountered in other context as well: https://sourceforge.net/p/iscsitarget/mailman/message/12621616/

The scsi_id helper of udev uses page 0x83 by default and if not available then page 0x80: https://manpages.org/scsi_id/8

> > Not sure what to do in the second case . /etc/lvm/devices/system.devices has
> > this:
> > IDTYPE=devname IDNAME=/dev/dasda2 DEVNAME=/dev/dasda2
> > PVID=sc9k1vMNcQpyWTeCdC8zQIKLApSCQQtN PART=2
> 
> The devname idtype is ok, although it's not optimal when the names change. 
> I assume that these dasd names change like sd device names?  If so, we'll

The dasd names are apparently not as unpredictable as SCSI names, but their order depends on the order of activation of devices, because on s390x, the disks are offline by default and one has to activate them (using the chccwdev -e command). For disks that do not get activated by udev during the boot sequence but you enable them later, this can result in a changed order. (This is not so different from hotplugging of SCSI devices, except that it concerns also devices that are continually attached.) Devices that are activated during the boot sequence seem to have a stable order (but I am not so familiar with RHEL 9 on s390x, more with RHEL 8, so I am not 100% sure).

> benefit from using the /sys/block/dasda/device/uid value.  I'll add a new
> device_id_type for these, e.g. IDTYPE=dasd_uid, which reads that sysfs uid
> path.  I have anticipated doing this for various unique device types that I
> didn't have access to during the original development.

This makes sense, although I am not sure if it would not be better to avoid introducing another id_type instead of using the same type as used for the serial number of SCSI/SATA/virtio disks.

> > as lsblk is already capable of showing the serial number properly, would it be possible to take the value from there
> 
> device ids are always read directly from the sysfs source (except devname
> when there's no proper id.)

yeah, I was wondering whether it would be possible to reuse what lsblk is doing (and add the necessary support for z/VM DASDs to lsblk or udev instead).

pushed to main
https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=bdab36cf3f059e597371bb504646f4dfb7a89f50
https://sourceware.org/git/?p=lvm2.git;a=commitdiff;h=761b922178c8522fd9abb207ba31bd27f9fa9dc4

commit bdab36cf3f059e597371bb504646f4dfb7a89f50
Author: David Teigland <teigland>
Date:   Mon Oct 24 16:23:36 2022 -0500

    device_id: look for serial number in other locations
    
    Only /sys/dev/block/major:minor/device/serial was read to find
    a disk serial number, but a serial number seems to be reported
    more often in other locations, so check these also:
    /sys/dev/block/major:minor/device/vpd_pg80
    /sys/class/block/vda/serial (for virtio disks only)

commit 761b922178c8522fd9abb207ba31bd27f9fa9dc4
Author: David Teigland <teigland>
Date:   Mon Oct 31 16:14:01 2022 -0500

    device_id: handle duplicate serial numbers
    
    Handle multiple devices using the same serial number as
    their device id.  After matching devices to devices file
    entries, if there is a discrepency between the ondisk PVID
    and the devices file PVID, then rematch devices to
    devices file entries using PVID, looking at all disks
    on the system with the same serial number.


The first commit is the main purpose of this bz, to read the serial number from other locations.  Testing this requires finding disks that report a value from a sysfs path listed in the commit message, then adding them to the devices file.  If the disks also report wwid's then you need to specify that lvm should use the serial number (since the wwid is preferred), i.e. lvmdevices --adddev /dev/sda --deviceidtype sys_serial.  (Without disks with serial numbers, a fake sysfs dir can be set up and populated with generated serial numbers for testing.)

The first commit is likely to result in serial numbers being used more often as device ids.  There is some doubt about whether disks may report the same serial number without any unique wwid.  A misconfiguration of virtual disk parameters may be a more likely case.  Since we don't know how likely this is, the second commit has been added to cover this scenario by detecting it and falling back to using the PVID instead of serial number.  A fake sysfs device setup can be used to test this (lvm tests do this), or possibly a libvirt config that duplicates the <serial> setting for multiple disks.

bug 2140728 is for adding dasd device ids.

Marking this Verfied:Tested with the latest rpms, with the caveat that this currently can lead to bug 2150348.

kernel-5.14.0-205.el9    BUILT: Fri Dec  2 07:14:37 AM CST 2022
lvm2-2.03.17-2.el9    BUILT: Wed Nov 30 02:56:30 AM CST 2022
lvm2-libs-2.03.17-2.el9    BUILT: Wed Nov 30 02:56:30 AM CST 2022



SCENARIO - attempt_all_valid_deviceidtypes:  Test basic devicesfile addition using all known deviceidtypes (1974337|2136906)
Available idtypes to attempt: ['sys_wwid', 'devname', 'sys_serial']

[...]

adding entry to the devices file for /dev/sdg1
removing entry from the devices file for /dev/sdg1
Add devices file entry with overriden type **sys_serial** for /dev/sdg1
lvmdevices --deviceidtype sys_serial --adddev /dev/sdg1
Grabbing serial name from /sys/dev/block/.../evice/vpd_pg80
major=8 minor=96
sysfs idname: 0083c91d2209183c2300e1d883604609
devicesfile idname: 0083c91d2209183c2300e1d883604609
adding entry to the devices file for /dev/sdg1
creating PV on hayes-03 using device /dev/sdg1
pvcreate --yes -ff   /dev/sdg1
  Physical volume "/dev/sdg1" successfully created.
Device naa.6d09466083d8e100233c1809221dc983 has idtype sys_wwid
1a. Check for valid devicesfile entry for newly created sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ
IDTYPE=sys_wwid IDNAME=naa.6d09466083d8e100233c1809221dc983 DEVNAME=/dev/sdg1 PVID=sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ PART=1
1b. Remove device file entry based on PV name /dev/sdg1
removing entry from the devices file for /dev/sdg1
2a. Add device file entry based on PVID sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ
  Found PVID sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ on /dev/sdg1.
IDTYPE=sys_serial IDNAME=0083c91d2209183c2300e1d883604609 DEVNAME=/dev/sdg1 PVID=sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ PART=1
2b. Remove device file entry based on PVID sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ
3. Add device file entry based on PV name
adding entry to the devices file for /dev/sdg1
IDTYPE=sys_wwid IDNAME=naa.6d09466083d8e100233c1809221dc983 DEVNAME=/dev/sdg1 PVID=sXrYb3nQXNx2tiGJXT2pfGFpVXXyLjBJ PART=1
removing pv /dev/sdg1 on hayes-03
  Labels on physical volume "/dev/sdg1" successfully wiped.
removing entry from the devices file for /dev/sdg1

Verified in the latest rpms as well.

kernel-5.14.0-205.el9    BUILT: Fri Dec  2 07:14:37 AM CST 2022
lvm2-2.03.17-3.el9    BUILT: Wed Dec  7 10:41:40 AM CST 2022
lvm2-libs-2.03.17-3.el9    BUILT: Wed Dec  7 10:41:40 AM CST 2022

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (lvm2 bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2544