I'm not positive which component this falls under. It's probably not anaconda as this issue probably needs to be resolved before anaconda starts.

This bug is a request for Red Hat to support NVMe Boot From SAN by implementing the "NVME Host Parameters for Fibre Channel Adapters" documented posted here:
https://standards.incits.org/apps/group_public/download.php/130241/T11-2021-00123-v000.pdf

Fortunately, some of the required work is already available upstream in dracut which is included in RHEL 9.0. dracut now includes a module called "nvmf" that implements nvme autoconnect to discover root when it resides on a nvme namespace.

Each vendor now has to make changes to their installer to initiate a NVMe autoconnect before their installer discovers installation devices. The RHEL installer should do the following:

1. Use nvme gen-hostnqn to create a hostnqn file using the server's system-uuid:

# nvme gen-hostnqn > /etc/nvme/hostnqn


2. Also use dmidecode to create the hostid file.

# dmidecode -s system-uuid > /etc/nvme/hostid


3. Initiate a NVMe discovery:

# echo add > /sys/class/fc/fc_udev_device/nvme_discovery


The installer can now proceed to discover installation devices. The NVMe namespace should automatically be displayed. The OS installation can now proceed.

Sometime during the OS post installation (probably when post install scripts are being run), the following actions must be done.

1. Copy the hostnqn and hostid files into /mnt/sysimage.

# cp /etc/nvme/hostnqn /mnt/sysimage/etc/nvme/hostnqn
# cp /etc/nvme/hostid /mnt/sysimage/etc/nvme/hostnqn

2. Rebuild the initramfs using dracut.

----------------------------------------------------------------------------

Broadcom ECD has a kit that does these steps to implement NVMe Boot-From-SAN on RHEL 9.0. This is a request for Red Hat to implement these changes into their installer to hopefully provide a seamless experience when using NVMe Boot-From-SAN.

I need to update the instructions after Step 3.

The installer can now proceed to discover installation devices. The NVMe namespace should automatically be displayed. The OS installation can now proceed. The OS installer should install nvme-cli. I did not see a way to manually select nvme-cli for installation.

Sometime during the OS post installation (probably when post install scripts are being run), the following actions must be done.

1. Copy the hostnqn and hostid files into /mnt/sysimage.

# cp /etc/nvme/hostnqn /mnt/sysimage/etc/nvme/hostnqn
# cp /etc/nvme/hostid /mnt/sysimage/etc/nvme/hostnqn

2. Rebuild the initramfs using dracut.

Sirius,

Can you please help us get movement from the Anaconda team.
We need help and this bug is not making progress within RH.
Perhaps that's not the correct component or it's lying on 
some forgotten pile. 

Please help.

thank you
Laurie

(In reply to Laurie Barry from comment #8)
> Sirius,
> 
> Can you please help us get movement from the Anaconda team.
> We need help and this bug is not making progress within RH.
> Perhaps that's not the correct component or it's lying on 
> some forgotten pile. 
> 
> Please help.
> 
> thank you
> Laurie

Hi there Laurie,

It had not been forgotten, there were internal comments discussing the problem. I have put it back to the Anaconda team as assignee (or it would have fallen through cracks). We are waiting for the feedback from one of the engineers.

Kind regards,

/S

Support for NVMeoF boot from SAN is not planned for 9.0. We are in the process of actively planning for this support to appear in a future release.

Maurizio, the hostnqn should at least have been generated from the system UUID, right?

Even if the rest of the boot from SAN mechanism is not yet implemented.

Broadcom has an out-of-box solution that presumably depends upon the nvme-cli changes
to generate the hostnqn from the system UUID so that it does not change.

(In reply to Ewan D. Milne from comment #14)
> Maurizio, the hostnqn should at least have been generated from the system
> UUID, right?
> 

Yes, nvme-cli generates an hostnqn based on the system UUID, if the DMI tables are available.

(In reply to Ewan D. Milne from comment #14)
> Maurizio, the hostnqn should at least have been generated from the system
> UUID, right?
> 
> Even if the rest of the boot from SAN mechanism is not yet implemented.
> 
> Broadcom has an out-of-box solution that presumably depends upon the
> nvme-cli changes
> to generate the hostnqn from the system UUID so that it does not change.

FYI the changes proposed in bug 1900691 may change the moment at which point hostnqn and hostid files are generated. The requirement is to avoid generating the machine-dependent files in nvme-cli rpm %post scripts. Instead, nvme-stas will provide a systemd service that will populate required files when not present.

(In reply to Tomáš Bžatek from comment #21)
> (In reply to Ewan D. Milne from comment #14)
> > Maurizio, the hostnqn should at least have been generated from the system
> > UUID, right?
> > 
> > Even if the rest of the boot from SAN mechanism is not yet implemented.
> > 
> > Broadcom has an out-of-box solution that presumably depends upon the
> > nvme-cli changes
> > to generate the hostnqn from the system UUID so that it does not change.
> 
> FYI the changes proposed in bug 1900691 may change the moment at which point
> hostnqn and hostid files are generated. The requirement is to avoid
> generating the machine-dependent files in nvme-cli rpm %post scripts.
> Instead, nvme-stas will provide a systemd service that will populate
> required files when not present.

This would make generating the HostNQN dependent upon nvme-stas. I don't see how
this would be any better then being dependent on nvme-cli. And all off these solutions
must be brought into line with the new NVMe Boot specification. There is a new Technical
Proposal (TP-4126) which recommends that the SMBIOS system UUID, found in the System Information 
(Type 1) structure at offset 8h, should be used to generate the HostNQN.  This TP will be
released as a part of the new NVMe Boot specification, and it should apply to all transports.

See BZ1900691 for more information.

(In reply to John Meneghini from comment #22)
> This would make generating the HostNQN dependent upon nvme-stas. I don't see
> how
> this would be any better then being dependent on nvme-cli. And all off these
> solutions
> must be brought into line with the new NVMe Boot specification. There is a
> new Technical
> Proposal (TP-4126) which recommends that the SMBIOS system UUID, found in
> the System Information 
> (Type 1) structure at offset 8h, should be used to generate the HostNQN. 
> This TP will be
> released as a part of the new NVMe Boot specification, and it should apply
> to all transports.

Currently libnvme and nvme-cli generates a stable HostNQN on the fly in case the /etc/nvme/hostnqn file is missing. This is based on /sys/class/dmi/id/product_uuid (and I expect the TP4126 will land in libnvme in a similar manner if not the same already).

Only nvme-stas requires /etc/nvme/hostnqn and /etc/nvme/hostid files to be present. The systemd stas-config.target is to ensure those files exist before starting nvme-stas services.

The point is that the HostNQN string should be consistent in both cases. Only HostID will be generated fresh by stas-config@.service.

And we will want to have both identifiers generated as part of the installation so the above fallback way applies only to pre-generated images (ostree). That's only a proposal though, open for suggestions.

According to everything discussed in this BZ, Anaconda should set the HostNQN based on the system UUID reported in the DMI tables.  The Host-id, according the the latest nvme-boot specification, can be set to 0.

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.

(In reply to John Meneghini from comment #28)
> According to everything discussed in this BZ, Anaconda should set the
> HostNQN based on the system UUID reported in the DMI tables.  The Host-id,
> according the the latest nvme-boot specification, can be set to 0.

This is incorrect.  The latest boot specification does not say anything about setting the HostID. TP-4110 does this. However, we should treat TP-4110 as a separate issue.  The requirements for for booting with nvme-of are defined in TP-4126.

See https://issues.redhat.com/browse/RHEL-1147 for more information about TP-4126

(In reply to Michael from comment #0)
> Description of problem:
> When booting the installer, a 'hostnqn' file is created. The hostnqn should
> be generated using the UUID of the host system, however it currently is not.

> Steps to Reproduce:
> 1. Boot nightly ISO to graphical installer
> 2. Switch to a console
> 3. Read the hostnqn ($ cat /etc/nvme/hostnqn) and compare it to the hostnqn
> that is expected ($ nvme gen-hostnqn)
> 
> Actual results:
> The hostnqn is not the same as what the nvme utility generates
> 
> Expected results:
> The hostnqn should be the same as what is generated by the nvme utility

As of RHEL 9.3, this problem should be resolved with the changes for https://issues.redhat.com/browse/RHEL-1147

The hostnqn & hostid are created by the nvme-cli.rpm install procedure.  So as long as the rpm is installed and configured in initramfs, the hostnqn & hostid should be correct. 
 
> Additional info:
> Before the graphical installer begins, a couple things should happen in
> order to NVME Boot From SAN devices to be discovered automatically:
> 
> 1. the correct NQN should be generated using 'nvme' utility:
> nvme gen-hostnqn > /etc/nvme/hostnqn

This looks correct.

> 2. NVME Discovery should be initiated:
> echo add > /sys/class/fc/fc_udev_device/nvme_discovery
> 
> 3. The NQN and Host ID files should be copied to sysimage so that the files
> get included by dracut
> cp /etc/nvme/hostnqn /mnt/sysimage/etc/nvme/hostnqn
> cp /etc/nvme/hostid /mnt/sysimage/etc/nvme/hostnqn

We may want to change the above procedure. See BZ 1900691

> Following these three steps will enable automatic discovery of NVMe devices

(In reply to John Meneghini from comment #32)
> See https://issues.redhat.com/browse/RHEL-1147 for more information about TP-4126

Filed https://issues.redhat.com/browse/RHEL-1254 for python-blivet changes. The blivet part was originally created as a clone of this bugzilla and included in RHEL 9.1.0: bug 2073008. That code is not currently called by Anaconda AFAIK (which was a task of this bug 2049991).

Still, Anaconda needs to cover a couple more scenarios. Here's my proposal, feel free to propose any change:

* Presently the installer image should not contain any /etc/nvme/{hostnqn,hostid} files - this has been done in bug 2071074 for RHEL 9.1.0.

* The dracut 95nvmf module included in RHEL 9.3.0 (bug 1933787) will recognize rd.nvmf.hostnqn and rd.nvmf.hostid kernel commandline arguments and will write them to /etc/nvme/{hostnqn,hostid} files in the initramfs environment. Anaconda should probably take these values over to an installer environment as initial values if set.

* In future RHEL releases, Anaconda will allow to define new hostnqn and hostid values (rules apply) similar to setting the iSCSI Initiator Name, in GUI or from kickstart, if not taken over from the previous phase.

* The nvme-cli %post scriptlets will generate hostnqn and hostid files on the target rootfs if they don't exist (the RHEL-1147 Jira change scheduled for inclusion in RHEL 9.3.0). Anaconda should likely overwrite these files with current installer environment values, before generating target initramfs.


Separately we'll need to handle systems without stable DMI UUID records as the current libnvme code (as called from nvme-cli) will generate random value as a fallback.


(please disregard my earlier proposal in comment 24, situation has changed since)

(In reply to Tomáš Bžatek from comment #34)
> Still, Anaconda needs to cover a couple more scenarios. Here's my proposal,
> feel free to propose any change:
> 
> * Presently the installer image should not contain any
> /etc/nvme/{hostnqn,hostid} files - this has been done in bug 2071074 for
> RHEL 9.1.0.

 
> * The dracut 95nvmf module included in RHEL 9.3.0 (Partnerbug 1933787) will
> recognize rd.nvmf.hostnqn and rd.nvmf.hostid kernel commandline arguments
> and will write them to /etc/nvme/{hostnqn,hostid} files in the initramfs
> environment. Anaconda should probably take these values over to an installer
> environment as initial values if set.

The dracut 95nvmf module has a dependency on nvme-cli today... correct?  

If this is true, and nvme-cli is a part of initrd, then why d owe need the  rd.nvmf.hostnqn and rd.nvmf.hostid
files?  Once we have all of the kernel and nvme-cli patches needed to support it, we should just be able to remove the hostid and hostnqn files everywhere.
 
> * In future RHEL releases, Anaconda will allow to define new hostnqn and
> hostid values (rules apply) similar to setting the iSCSI Initiator Name, in
> GUI or from kickstart, if not taken over from the previous phase.

So you want  rd.nvmf.hostnqn and rd.nvmf.hostid to be able to override nvme-cli during install/boot?

> * The nvme-cli %post scriptlets will generate hostnqn and hostid files on
> the target rootfs if they don't exist (the RHEL-1147 Jira change scheduled
> for inclusion in RHEL 9.3.0). Anaconda should likely overwrite these files
> with current installer environment values, before generating target
> initramfs.

I'm thinking that we should just remove all code that generates the hostid/hostnqn and allow nvme-cli to gererate it programatically.

> Separately we'll need to handle systems without stable DMI UUID records as
> the current libnvme code (as called from nvme-cli) will generate random
> value as a fallback.

I think the way we handle these is: they are not supported.

(In reply to John Meneghini from comment #35)
> The dracut 95nvmf module has a dependency on nvme-cli today... correct?  
> 
> If this is true, and nvme-cli is a part of initrd, then why d owe need the 
> rd.nvmf.hostnqn and rd.nvmf.hostid
> files?  Once we have all of the kernel and nvme-cli patches needed to
> support it, we should just be able to remove the hostid and hostnqn files
> everywhere.

This concerns mostly the installer ISO first boot and its dracut run. On an installed system, initramfs gets regenerated with custom files and these overrides won't be needed.
If these rd.nvmf overrides are not supposed to be supported by RHEL, I suggest we remove it.

> So you want  rd.nvmf.hostnqn and rd.nvmf.hostid to be able to override
> nvme-cli during install/boot?

Exactly. Subject to discussion or removal.

> I'm thinking that we should just remove all code that generates the
> hostid/hostnqn and allow nvme-cli to gererate it programatically.

The point is, if these overrides are used, we should honour them through the rest of the installation process.

> > Separately we'll need to handle systems without stable DMI UUID records as
> > the current libnvme code (as called from nvme-cli) will generate random
> > value as a fallback.
> 
> I think the way we handle these is: they are not supported.

Agreed. nvme-cli update in 9.4 will include fixes for this scenario but it can be perhaps officially marked as unsupported/untested.

Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.