Bug 1322110

There is definitely NO grammar for  DM-UUID - except its it's just  128bytes +'\0' string and should be unique.

There are certain  undocumented  INTERNAL rules in lvm2 for  UUID generation - but lvm2 is FREE to change them any time it needs to.

The rule for lvm2 user accessible devices is:

LVM-VGuuidLVuuid

Every other lvm2 device uuid is  lvm2  PRIVATE uuid which noone else should evere try to understand.

Generic plan we aim for to simplify  'blkid' detection of lvm2 private device is to always add some -suffix  -  once there is -suffix  noone else is supposed to read such device.

So whoever gets an idea to detect  i.e. raid/thin-pool images is doing just useless work.

Just wondering  where you get the idea to write grammar for it??
Is that some new  project ??

What information is missing from the udev database that is leading to this attempt to parse the private content of the uuid?

If people can identify the missing information and explain why it's needed, then we can try to update the standard udev rules for the component that owns the private part of the uuid in question to provide this extra information directly in the database.

As far as grammar is concerned, each component that uses dm devices is encouraged to register a unique prefix, such as "LVM-" and then the recommended format is that prefix followed by a private part under the complete specification and control of the component itself.  This is not fixed, has changed already, and will change further over time, so don't try to write any code that depends on this please - use the supported interfaces instead.

Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.

Your example:

DM_UUID=part1-mpath-WDC_WD10EFRX-68PJCN0_WD-WCC4JPFAUX6A

part1- is a prefix owned by kpartx.
So this means you ask kpartx to interpret the rest of the uuid.

You'll find that kpartx gives you the answer that you should treat the rest as a standalone uuid in its own right.

So you then see the mpath prefix.
Strip it off and ask multipath to interpret it.  It'll say it's the private uuid it's chosen to use to refer to the underlying device.

None of this is for end users.

Developers see libdevmapper.h:

 * Configure default UUID prefix string.
 * Conventionally this is a short capitalised prefix indicating the subsystem
 * that is managing the devices, e.g. "LVM-" or "MPATH-".
 * To support stacks of devices from different subsystems, recursive functions
 * stop recursing if they reach a device with a different prefix.

Thanks for the comments. I'm seeking further clarification.

Aside: The comment from libdevmapper.h is slightly misleading,
"mpath" is actually _not_ capitalized :)
DM_UUID=mpath-WDC_WD10EFRX-68PJCN0_WD-WCC4J5VFRFV3

Comment #2: I'm not familiar with every storage/udev related utility or library. But I know that at least blivet and lsblk parse out DM_UUID (or dm/uuid) as the case may be. In both cases, they are looking for a subsystem value, e.g., "LVM", "mpath". blivet seems to be looking for the last or primary subsystem value, lsblk for the first. My own enthusiasm for parsing values in udev is a good deal less than 0. However, sometimes it seems there is no other option, and I'ld rather be doing this parsing separately from the application or library that needs this information. Hence a separate and privately used library that I run separate tests on. Library is probably too grand a word for what it is, but test _is_ a good description of what I do with it.

Comment #3: In the cases that I have seen, it seems to be _some_ device-mapper subsystem which is being extracted from the DM_UUID value. Evidently that is not available any other way, or at least that's what the developers of blivet and lsblk think.

AFAICT, from comments #4 and following, the grammar is really something like:
DM_UUID -> G
G -> SUBSYSTEM_STR | SUBSYSTEM_STR "-" G
SUBSYSTEM_STR -> SUBSYSTEM_PREFIX "-" SUBSYSTEM_SPECIFIC_STR
SUBSYSTEM_PREFIX -> "part" PARTITIONID | "LVM" | "mpath" | ...
SUBSYSTEM_SPECIFIC_STR -> <just about anything>

I _think_ that it is this SUBSYSTEM_SPECIFIC_STR which is considered to be private in the above comments.

This grammar is unambiguous and context-free only if SUBSYSTEM_PREFIX can be unambiguously identified.
That would be possible if the SUBSYSTEM_PREFIX production could not intersect with the SUBSYSTEM_STR production. e.g., if all the prefixes were known and unused in SUBSYSTEM_SPECIFIC_STR.

If my understanding of the grammar is correct, then the way blivet at least does the parsing is wrong according to the grammar, but "works" because kpartx, which uses the "part..." prefix, currently has a value for SUBSYSTEM_SPECIFIC_STR of "". See example like:

def device_dm_subsystem_match(info, subsystem):
    """ Return True if the device matches a given device-mapper subsystem. """
    uuid = info.get("DM_UUID", "")
    uuid_fields = uuid.split("-")
    _subsystem = uuid_fields[0]
    if _subsystem.lower().startswith("part") and len(uuid_fields) > 1:
        # kpartx uses partN- as a subsystem prefix, which we ignore because
        # we only care about the subsystem of the partitions' parent device.
        _subsystem = uuid_fields[1]

    if _subsystem == uuid or not _subsystem:
        return False

    return _subsystem.lower() == subsystem.lower()

Is my understanding correct?

I would be happy with either:
1. An umambigous grammar for DM_UUID where I did not dig into the SUBSYSTEM_SPECIFIC_STR part at all that is robust to future changes.
2. A robust alternative for identifying the subsystem(s) that manage a particular device.

But the fact of the matter is that currently, and since 2011 at least in the case of lsblk, parsing DM_UUID has been the established way of doing it and, if my understanding is correct, parsing it without understanding of the intended grammar is pretty well-established, too.

lsblk parses _only_ prefix (subsystem), it never expects something from the rest of dm-uuid, IOW the first substring before "-". 

(And it is just for convenience to display name of subsystem to user - if this fails, it will simply display device type as "dm", nothing should depend on it.)

Comment #9: Thanks for confirming...lsblk parses the _first_ subsystem only.

So, it does not distinguish between a partition on a multipath device and one that is not, as it looks no deeper. By contrast, blivet does.

lsblk traverses through holders in sysfs, it doesn't need to guess storage stack from private part of DM-UUID. That was the whole idea behind lsblk.
(And you can parse output from lsblk in scripts.)

(BTW LUKS uses the same device-mapper UUID convention, internally it parses own created info from DM-UUID but this is not interface for anyone except libcryptsetup itself. Private DM_UUID part can change anytime, despite it has fixed structure now.)

(In reply to mulhern from comment #8)
> Aside: The comment from libdevmapper.h is slightly misleading,
> "mpath" is actually _not_ capitalized :)
> DM_UUID=mpath-WDC_WD10EFRX-68PJCN0_WD-WCC4J5VFRFV3

I know: I hope one day that will change.  I can try hard to get other components to meet the standard here, but it's not an enforced standard and I was content there just to have achieved the use of a prefix!

> Comment #2: I'm not familiar with every storage/udev related utility or
> library. But I know that at least blivet and lsblk parse out DM_UUID (or
> dm/uuid) as the case may be. In both cases, they are looking for a subsystem
> value, e.g., "LVM", "mpath". blivet seems to be looking for the last or
> primary subsystem value, lsblk for the first.

They do that at their own risk.  'lsblk' behaves reasonably and understands the issues as Milan explained in the previous comments.

If blivet is looking more deeply using an unsupported procedure, that sounds fragile to me.
 
> AFAICT, from comments #4 and following, the grammar is really something like:
> DM_UUID -> G
> G -> SUBSYSTEM_STR | SUBSYSTEM_STR "-" G
> SUBSYSTEM_STR -> SUBSYSTEM_PREFIX "-" SUBSYSTEM_SPECIFIC_STR
> SUBSYSTEM_PREFIX -> "part" PARTITIONID | "LVM" | "mpath" | ...
> SUBSYSTEM_SPECIFIC_STR -> <just about anything>

Nope - it is simply PREFIX PRIVATE_CONTENT. Each subsystem has chosen its own PREFIX (not used by any other subsystem).  PRIVATE_CONTENT is defined by the subsystem that owns the prefix and its format is not tied down but is free to change in any future release.  (The hyphen is technically considered part of the prefix - not all subsystem prefixes end with a hyphen.)

> I would be happy with either:
> 1. An umambigous grammar for DM_UUID where I did not dig into the
> SUBSYSTEM_SPECIFIC_STR part at all that is robust to future changes.
> 2. A robust alternative for identifying the subsystem(s) that manage a
> particular device.

The PREFIX defines the subsystem.  That's all you can rely upon.  Once you know the subsystem, you should obtain information about what is underneath it from the subsystem itself, either by asking it or by looking in the udev database or sysfs or wherever else the required information is (or could be) stored.

There is nothing for fixed in  'lvm2' neither in 'dm' package - closing....

For further discussion I'd suggest to use dm-devel list
(i.e. for impact on  mpath...)

Comment #11: This is really an aside, but I think that storage is complex enough that holders/slaves relationship is not really adequate to describe some of the relationships between devices, see: https://bugzilla.redhat.com/show_bug.cgi?id=1195374.

But the real reason to reopen is that the DM_UUID grammar is still unclear to me.

According to agk, the grammar is really very simple:

DM_UUID -> G
G -> PREFIX PRIVATE_CONTENT
PREFIX -> <anything>
PRIVATE_CONTENT -> <anything>

In particular, PREFIX may or may not have a '-' at the end.

But, in that case, the grammar is ambiguous and lsblk is wrong, as it explicitly assumes a '-' as a delimiter between the PREFIX and the rest, i.e., its assumed grammar is more like:

DM_UUID -> G
G -> PREFIX '-' PRIVATE_CONTENT
PREFIX -> <anything except a '-'>
PRIVATE_CONTENT -> <anything>
PREFIX -> "part" PARTITIONID | NONPARTPREFIX
NONPARTPREFIX -> <anything>
PARTITIONID -> [1-9][0-9]*

See excerpted code below:

static char *get_type(struct blkdev_cxt *cxt)
{
    char *res = NULL, *p;

    if (is_dm(cxt->name)) {
        char *dm_uuid = sysfs_strdup(&cxt->sysfs, "dm/uuid");

        /* The DM_UUID prefix should be set to subsystem owning
         * the device - LVM, CRYPT, DMRAID, MPATH, PART */
        if (dm_uuid) {
            char *tmp = dm_uuid;
            char *dm_uuid_prefix = strsep(&tmp, "-");

            if (dm_uuid_prefix) {
                /* kpartx hack to remove partition number */
                if (strncasecmp(dm_uuid_prefix, "part", 4) == 0)
                    dm_uuid_prefix[4] = '\0';

                res = xstrdup(dm_uuid_prefix);
            }
        }
...

The only way to make the grammar unambiguous is to add a complete list of productions for PREFIX, like:

PREFIX -> "mpath-" | "LVM-" | "part" PARTITIONID "-" | ... <must be complete>
PARTITIONID -> [1-9][0-9]*

=================================================

The only other way for this to make sense is for each dm subsystem to
have its own separate grammar, like:

DM_UUID => G
G => MYPREFIX MYSTUFF NOTMYSTUFF | NOTMYSTUFF
MYPREFIX => [subsystem appropriate prefix]
MYSTUFF => [subsystem appropriate]
NOTMYSTUFF => "" | <anything>

so that, for mpath, the MYPREFIX production is:

MYPREFIX => "mpath-"
MYSTUFF => <I'm not sure, kinda complicated> 

for lvm,

MYPREFIX => "LVM-"
MYSTUFF => <I'm not quite sure, kinda complicated>

for kpartx,

MYPREFIX => "part" PARTITIONID '-'
PARTITIONID => [1-9][0-9]*
MYSTUFF => ""

and so on.

Maybe this is the case? If it is, then _no_ external entity should be parsing DM_UUID because no external entity has its own grammar. Which would solve my immediate problem nicely, but not lsblk's or blivet's ongoing ones.

(In reply to mulhern from comment #14)
> _no_ external entity should be
> parsing DM_UUID 

Correct.  Anything that attempts to parse it is very likely doing something wrong because all the information it should need should already be available elsewhere through supported mechanisms.

This is *developer* information.  I doubt we need to write anything about this in the user guides aimed at sysadmins.