Bug 1822697

NetApp thinks this may have started on 8.2 snapshot 3 perhaps?
They say this didn't happen in 8.1, or 7.7, or 8.2 Beta.

Did the nvme-cli package get updated after Beta?

I'm wondering if it could be related to one of these upstream changes to nvme-cli:

commit 48c10dbfaf00777170e209db20f7d07e47f7a983
Author: Andy Lutomirski <luto>
Date:   Thu Oct 3 11:47:02 2019 -0700

    Use a systemd app-specific machine ID for hostnqn

    If /etc/nvme/hostnqn is not present, the fabric commands will ask
    systemd for an app-specific machine ID as a fallback.  This should
    improve functionality if /etc/nvme/hostnqn is not present and should
    allow packagers to avoid creating /etc/nvme/hostnqn.

commit 6180aa0e32ba7d0e8dfa8435d212208569168773
Author: Simon Schricker <sschricker>
Date:   Fri May 17 15:36:37 2019 +0200

    nvme-cli: Add script to determine host NQN

    * Based on the system-UUID, via dmidecode
    * verifies UUID format
    * tries to catch fake/lazy UUIDs

I thought nvme-cli got updated late in 8.2 to pick up the autoconnect changes ?

Hi Steve,

(In reply to Steve Schremmer from comment #2)
> Did the nvme-cli package get updated after Beta?

Yes, we updated to nvme-cli-1.9-5.el8.x86_64.rpm to pickup the autoconnect
change, the below patches went into v1.10 upstream so they are not in the
RHEL8.2 build. Nothing should have changed generating unique hostnqn, not 
sure at the moment what would have broken. Clayton, are you testing with 
nvme-cli-1.9-5.el8.x86_64.rpm?

Also, do remember testing any of the other nvme-cli-1.9-x versions?

I will take a look at the recent changes. Thanks.

> 
> I'm wondering if it could be related to one of these upstream changes to
> nvme-cli:
> 
> commit 48c10dbfaf00777170e209db20f7d07e47f7a983
> Author: Andy Lutomirski <luto>
> Date:   Thu Oct 3 11:47:02 2019 -0700
> 
>     Use a systemd app-specific machine ID for hostnqn
> 
>     If /etc/nvme/hostnqn is not present, the fabric commands will ask
>     systemd for an app-specific machine ID as a fallback.  This should
>     improve functionality if /etc/nvme/hostnqn is not present and should
>     allow packagers to avoid creating /etc/nvme/hostnqn.
> 
> commit 6180aa0e32ba7d0e8dfa8435d212208569168773
> Author: Simon Schricker <sschricker>
> Date:   Fri May 17 15:36:37 2019 +0200
> 
>     nvme-cli: Add script to determine host NQN
> 
>     * Based on the system-UUID, via dmidecode
>     * verifies UUID format
>     * tries to catch fake/lazy UUIDs

Hi Clayton,

Here is 1.9-2 and 1.9-3 in case you want to test an earlier version.

http://people.redhat.com/dmilburn/.bz1822697.98030607080910702011/

> I will take a look at the recent changes. Thanks.
> 
> > 
> > I'm wondering if it could be related to one of these upstream changes to
> > nvme-cli:
> > 
> > commit 48c10dbfaf00777170e209db20f7d07e47f7a983
> > Author: Andy Lutomirski <luto>
> > Date:   Thu Oct 3 11:47:02 2019 -0700
> > 
> >     Use a systemd app-specific machine ID for hostnqn
> > 
> >     If /etc/nvme/hostnqn is not present, the fabric commands will ask
> >     systemd for an app-specific machine ID as a fallback.  This should
> >     improve functionality if /etc/nvme/hostnqn is not present and should
> >     allow packagers to avoid creating /etc/nvme/hostnqn.
> > 
> > commit 6180aa0e32ba7d0e8dfa8435d212208569168773
> > Author: Simon Schricker <sschricker>
> > Date:   Fri May 17 15:36:37 2019 +0200
> > 
> >     nvme-cli: Add script to determine host NQN
> > 
> >     * Based on the system-UUID, via dmidecode
> >     * verifies UUID format
> >     * tries to catch fake/lazy UUIDs

Hi David, To answer comment 4, I can guarantee it was working in the RHEL 8.2 Beta (nvme-cli-1.8.1-3.el8.x86_64.rpm)

I'm uncertain if the issue was in SS1 or SS2, but I can say I definately saw this issue in SS3 (nvme-cli-1.9-2.el8.x86_64.rpm) and RC1 (nvme-cli-1.9-5.el8.x86_64.rpm)

To Comment 5, that actually means I've already tested 1.9-2 and can confirm it exists there too.

Regards,
-Clayton

(In reply to Clayton Skaggs from comment #6)
> Hi David, To answer comment 4, I can guarantee it was working in the RHEL
> 8.2 Beta (nvme-cli-1.8.1-3.el8.x86_64.rpm)
> 
> I'm uncertain if the issue was in SS1 or SS2, but I can say I definately saw
> this issue in SS3 (nvme-cli-1.9-2.el8.x86_64.rpm) and RC1
> (nvme-cli-1.9-5.el8.x86_64.rpm)
> 
> To Comment 5, that actually means I've already tested 1.9-2 and can confirm
> it exists there too.
> 

Thanks Clayton, would you please test nvme-cli-1.9-6? It looks part of post
section in spec file didn't execute.


http://people.redhat.com/dmilburn/.bz1822697.98030607080910702011/

Hi David,

I've tested nvme-cli-1.9-6. That looks to of done the trick. After installing this version my hosts generated unique identifiers. If you can get this into a build of RHEL 8.2 we can get this closed.

Regards,
-Clayton

Hi Clayton,

(In reply to Clayton Skaggs from comment #8)
> Hi David,
> 
> I've tested nvme-cli-1.9-6. That looks to of done the trick. After
> installing this version my hosts generated unique identifiers. If you can
> get this into a build of RHEL 8.2 we can get this closed.
> 

Thanks for retesting, I will have to push it to RHEL8.2.z, at this time
I won't be able to update the package for RHEL8.2.

Hi Clayton,

I was able to put the fix in nvme-cli-1.10.1-1.el8 which
will be our RHEL8.3 package. I can ask for management to
approve nvme-cli-1.9-6.el8 for RHEL8.2.z (RHEL8.2 update
package), but they will need a few words from NetApp
(on this BZ) describing how NetApp is impacted by this 
bug on RHEL8.2. Thanks.

Hi David,

The impact to us or any customers in the field would be that every time they install a new host each host will have identical nqns. These nqns are how we map volumes to hosts in a fabric, so if they are identical across several hosts this can cause a lot of trouble with discovery of luns from these hosts on our array. These nqns would then need to be manually edited into unique values to allow lun discovery to function properly which could prove to be an exhaustive process in a more sizable environment.

Regards,
-Clayton

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nvme-cli bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4476