1. Please describe the problem: Historically CONFIG_LIBNVDIMM was marked "=y" because there was at least one NVDIMM provider marked "=y". However, that is no longer the case: CONFIG_ACPI_NFIT=m CONFIG_X86_PMEM_LEGACY=m More importantly the CONFIG_LIBNVDIMM=y setting currently defeats the NVDIMM security unlock flow. The keys required to unlock NVDIMMs are currently loaded via modprobe.conf rule like the following: install libnvdimm /usr/bin/ndctl load-keys ; /sbin/modprobe --ignore-install libnvdimm $CMDLINE_OPTS With CONFIG_LIBNVDIMM=y built-in there is no trigger to automate the loading of the keys. For reference the key handling implementation follows that of fscrypt where where a master-key protects the per-NVDIMM key material. Longer term the CONFIG_NVDIMM_KEYS dependency will be fixed to depend on CONFIG_LIBNVDIMM=m, or otherwise move the specific dependency to its own module. In the meantime a change to the shipping configuration is needed for pre-v5.2 kernels, and to support CONFIG_NVDIMM_SECURITY=y in post v5.2 kernels. 2. What is the Version-Release number of the kernel: Latest rawhide: kernel-5.1.0-0.rc3.git1.1.fc31.x86_64.rpm 3. Did it work previously in Fedora? If so, what kernel version did the issue *first* appear? Old kernels are available for download at https://koji.fedoraproject.org/koji/packageinfo?packageID=8 : It never worked and the feature was developed on CONFIG_LIBNVDIMM=m builds. 4. Can you reproduce this issue? If so, please provide the steps to reproduce the issue below: There is no trigger to run the "ndctl load-keys" helper prior to the driver attempting to unlock DIMMs. 5. Does this problem occur with the latest Rawhide kernel? To install the Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by ``sudo dnf update --enablerepo=rawhide kernel``: Yes. 6. Are you running any modules that not shipped with directly Fedora's kernel?: No. 7. Please attach the kernel logs. You can get the complete kernel log for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the issue occurred on a previous boot, use the journalctl ``-b`` flag. N/A
Hi Dan, CONFIG_LIBNVDIMM=m should arrive in tomorrow's Rawhide build (kernel-5.1.0-0.rc4.git1.1.fc31).
Hi Dan, for kata usage in Fedora we would like the option to boot from a filesystem image over nvdimm, rather than the slower and more memory consuming initrd default. But this requires non-module NVDIMM in the kernel. This bug suggests that there may be an eventual kernel fix for your issue. Has it been fixed upstream? Any other suggestions how we can resolve this?
@Cole, no there's no option to support DIMM security with libnvdimm built-in, and no future kernel fix being pursued for this case. Is this actual pmem or emulated? If it's emulated pmem why not just run out of it directly? I.e. as an initrfamfs and never pivot_root. That should be even faster than loading the driver and mounting a block based filesystem.
(In reply to Dan Williams from comment #3) > @Cole, no there's no option to support DIMM security with libnvdimm > built-in, and no future kernel fix being pursued for this case. Is this > actual pmem or emulated? If it's emulated pmem why not just run out of it > directly? I.e. as an initrfamfs and never pivot_root. That should be even > faster than loading the driver and mounting a block based filesystem. Kata wraps a qemu+kvm VM around a traditional container image. There's a small appliance OS that runs in the VM, which the host communicates with, and the container is launched in that appliance. kata can pass the appliance rootfs to the VM in two ways: via an initrd via qemu -initrd option, or via nvdimm with qemu -device nvdimm. So this is only dealing with emulated. I'm not a kata expert, but my understanding is: initrd is faster for boot, but nvdimm block access is better for reducing total memory overhead, and the latter criteria is higher on the prio list. kata does not allow combining the initrd and nvdimm options. So if we want to use nvdimm as FS mount, the necessary modules need to be built into the Fedora kernel or the appliance rootfs can not be mounted. I'll ask some of the intel kata guys to chime in here too
I can see a path to supporting locked NVDIMMs *and* CONFIG_LIBNVDIMM=y, but it will take some reworks of both the kernel, and the user space infrastructure for nvdimm-key loading. Effectively instead of relying on 'ndctl load-keys' to happen before the libnvdimm.ko module is loaded, the normal /usr/sbin/request-key flow would need to be enabled. That needs to be combined with a driver rework to defer NVDIMM device probing until the call_usermode_helper() infrastructure is online. I.e. if libnvdimm encounters a locked DIMM defer that device probing until userspace helpers can be called. This would be similar to device-drivers that need a userspace firmware blob to be loaded before device probing can make forward progress. For kata-containers there would never be a locked nvdimm so it would skip the probe deferral and behave as you would expect.
> Historically CONFIG_LIBNVDIMM was marked "=y" because there was at least one > NVDIMM provider marked "=y". However, that is no longer the case: > > CONFIG_ACPI_NFIT=m > CONFIG_X86_PMEM_LEGACY=m > > More importantly the CONFIG_LIBNVDIMM=y setting currently defeats the NVDIMM > security unlock flow. > > The keys required to unlock NVDIMMs are currently loaded via modprobe.conf > rule like the following: > > install libnvdimm /usr/bin/ndctl load-keys ; /sbin/modprobe --ignore-install > libnvdimm $CMDLINE_OPTS Is it required to hook this into libnvdimm module load? Or would hooking this into nfit / nd_e820 module load work too? Possibly it is an option to use (a) CONFIG_LIBNVDIMM=y + CONFIG_VIRTIO_PMEM=y (for qemu), and (b) CONFIG_ACPI_NFIT=m + CONFIG_X86_PMEM_LEGACY=m (for locked dimms)
No, it was broken to tie this to module loading in the first instance, the rework I laid out in comment5 is the way to go, but someone to do the work has been the missing dependency to date.