Hi Colin,

Just wondering which version of fedora you are referring to ? I'll check and see if I can update to that or can pull in specific patch from. As there some of unsupported feature we have in latest nvme-cli package which we can't update to that right away. 

Thanks 
Gopal

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

This issue is still valid and we are still working around it in rpm-ostree. This will impact us again when we move to RHEL 9.

To solve this issue, you should add a check for /run/ostree-booted in your %post script to skip all uuid or fdqn generation when running under rpm-ostree.
Then, you need to create a systemd unit that will be run on boot and will generate those files if they don't exists.

Thanks

I'm moving this to RHEL 9 given that we don't strictly "need" the fix in RHEL 8 even though the issue is there too.

(In reply to Colin Walters from comment #0)
> However, this value duplicates `/etc/machine-id` that is managed by systemd.
> Do you really need a separate one at all?  If you do, see
> https://www.freedesktop.org/software/systemd/man/machine-id.html

FYI, libnvme (where the code has moved recently) needs to stay init system-independent upstream. While it can be made to use systemd API when available, both the /etc/nvme/hostnqn and /etc/nvme/hostid are currently used by several other packages and a single location is preferred. Besides, the hostid and machine-id are theoretically independent, you may supply any arbitrary hostid to libnvme calls. Also, the hostid may be freely broadcasted over the network, that breaks the confidentiality requirement of the machine-id.

While a systemd unit may seem like an obvious solution, similar to the iscsi-init.service from iscsi-initiator-utils rpm, it's more complicated in libnvme and nvme-cli case. It will bring a requirement to wire the systemd unit startup to all components that are about to use it (blivet, anaconda). The difference from iscsi-initiator-utils is that iscsid needs to be started to make things work and adding another systemd unit dependency is easy. There's no daemon or service involved when using libnvme.

I'm wondering if there's a possibility to ship any ostree-specific hooks that would serve as a replacement for the nvme-cli rpm post-scripts. It might be certainly easier than modify everything else.

> FYI, libnvme (where the code has moved recently) needs to stay init system-independent upstream.

/etc/machine-id is a generic concept that any other init system could easily implement, and I believe others do.

> While it can be made to use systemd API when available, both the /etc/nvme/hostnqn and /etc/nvme/hostid are currently used by several other packages and a single location is preferred. 

Sure, but those could be symlinks to the machine-id, or the code could check for them if present, and otherwise fall back to deriving from machine-id.

> Besides, the hostid and machine-id are theoretically independent, you may supply any arbitrary hostid to libnvme calls. Also, the hostid may be freely broadcasted over the network, that breaks the confidentiality requirement of the machine-id.

Yes, this is a great point, but it's covered by
https://www.freedesktop.org/software/systemd/man/machine-id.html
which says:

> This ID uniquely identifies the host. It should be considered "confidential", and must not be exposed in untrusted environments, in particular on the network. If a stable unique identifier that is tied to the machine is needed for some application, the machine ID or any part of it must not be used directly. Instead the machine ID should be hashed with a cryptographic, keyed hash function, using a fixed, application-specific key. That way the ID will be properly unique, and derived in a constant way from the machine ID but there will be no way to retrieve the original machine ID from the application-specific one. The sd_id128_get_machine_app_specific(3) API provides an implementation of such an algorithm.

> I'm wondering if there's a possibility to ship any ostree-specific hooks

See my first comment: https://bugzilla.redhat.com/show_bug.cgi?id=1900691#c0

> is wrong for rpm-ostree (or really, any image update system). 

I have to emphasize that this isn't an ostree-specific problem, it's a problem that *any* technology for "image based updates" would hit.
For example, some operating systems use dual-partition dm-verity type setups, etc.

> It will bring a requirement to wire the systemd
> unit startup to all components that are about to use it (blivet, anaconda).
> The difference from iscsi-initiator-utils is that iscsid needs to be started
> to make things work and adding another systemd unit dependency is easy.
> There's no daemon or service involved when using libnvme.

We should be able to use a generic systemd target to order against it. When are applications using those files started?

If you take the approach of by default deriving a unique ID from /etc/machine-id, then no custom systemd unit logic is required at all.  /etc/machine-id is created *very* early on on the first boot of the system, and that's it.  This seems the most preferable option to me.

(In reply to Colin Walters from comment #10)
> If you take the approach of by default deriving a unique ID from
> /etc/machine-id, then no custom systemd unit logic is required at all. 
> /etc/machine-id is created *very* early on on the first boot of the system,
> and that's it.  This seems the most preferable option to me.

Unless supplied via /etc/nvme/hostnqn, libnvme derives its default hostnqn string from system DMI data, to ensure stable ID. This is somewhat similar to the machine-id concept (and I vaguely remember nvme-cli used to use machine-id, it's gone in the current codebase). The hostnqn string should be in form of "nqn.2014-08.org.nvmexpress:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" and the hostid should be in uuid form of "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx". This is different from what /etc/machine-id contains and can't be symlinked directly.

Most of the nvme userspace stack can cope with missing /etc/nvme/hostnqn and /etc/nvme/hostid files just fine. It's the nvme-stas that requires the files to be present, also it's a good practice to have the IDs generated upon installation to ensure persistency.

(In reply to Timothée Ravier from comment #9)
> We should be able to use a generic systemd target to order against it. When
> are applications using those files started?

In the extreme use case these files might be needed for nvme-cli and/or nvme-stas running within initramfs to discover and connect NVMeoF rootfs to boot the system from. I guess a simple systemd unit can do this and though its overhead may be negligible for most systems, the number of installations containing it will likely be large.

> Unless supplied via /etc/nvme/hostnqn, libnvme derives its default hostnqn string from system DMI data,

Yep, that also must be discovered and executed on the target host, which for image based updates cannot be in %post.

>  to ensure stable ID. This is somewhat similar to the machine-id concept (and I vaguely remember nvme-cli used to use machine-id, it's gone in the current codebase). The hostnqn string should be in form of "nqn.2014-08.org.nvmexpress:uuid:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" and the hostid should be in uuid form of "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx". This is different from what /etc/machine-id contains and can't be symlinked directly.

We've already covered that we're not talking about symlinking, but in computing derived data.

> It's the nvme-stas that requires the files to be present, also it's a good practice to have the IDs generated upon installation to ensure persistency.

Computing derived uuids from /etc/machine-id as suggested *is* persistent - or to say it another way, it's equally persistent as other files in /etc as are generated in the %post today.

It sounds like nvme-stas should change to detect if the files in /etc/nvme are present, and if so - use them.  If not, then compute identifiers from /etc/machine-id in the desired format.

I've filed https://github.com/linux-nvme/nvme-stas/issues/130 requesting to use auto-generated (stable) hostnqn as a fallback.

To move forward, let's try to make the whole userspace nvme stack ready for missing /etc/nvme machine-specific files. With RHEL 9.1 rebasing nvme-cli to the 2.x series I believe there's still chance to slightly change how things work. We may hit a wall somewhere but let's try that at least. It's not only about nvme-stas, there might be foreign scripts reading /etc/nvme/hostnqn that we're not aware of.

Anaconda will likely generate hostnqn and hostid files after installation anyway.

On Fedora, I've just built nvme-stas-1.1.3-2.fc36 that includes stas-config.target and stas-config@.service systemd units that generate required hostnqn and hostid files when missing, prior to stacd.service and stafd.service startup. This potentially allows to drop the nvme-cli %post scriptlets.

@Maurizio: are we okay going this way forward?

I want to point out that there is a new NVMe Technical Proposal 4126 which recommends that he SMBIOS system UUID, found in the System Information (Type 1) structure at offset 8h, should be used for generate the HostNQN. 

I've attached TP-4126 to this BZ.

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

Looks like this has been fixed with https://github.com/linux-nvme/nvme-stas/pull/154 which is (at least) in RHEL 9.2 so this should be fixed.

Re-opening based on previous (private) comment.

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.