This may be related to: http://www.novell.com/support/kb/doc.php?id=7010836 and https://bugs.launchpad.net/maas/+bug/1115710 For a MT26448 dual port 10gb ethernet card: 03:00.0 Ethernet controller: Mellanox Technologies MT26448 [ConnectX EN 10GigE, PCIe 2.0 5GT/s] (rev b0) the driver, mlx4_en, is not being loaded automatically. The mlx4_core module is being loaded. after doing a manual modprobe mlx4_en, the devices come up perfectly. Suggestion on novell site is to force the load in /etc/modprobe.d. Software: kmod-12-3.fc18.x86_64 kmod-libs-12-3.fc18.x86_64 Linux r3 3.8.8-202.fc18.x86_64 #1 SMP Wed Apr 17 23:25:17 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
I suppose with kmod works, or with the kernel itself? A similar one that loads mlx4_ib should probably ship with the libmlx4 package.
Although I'm really confused why this can't just be loaded automatically from mlx4_core itself, whether via dependencies or 'request_module'.
The libmlx4 package provides a libmlx4.conf file in /etc/modprobe.d that takes care of loading both the _en and _ib drivers (the _en cards can support RDMA operations via IBoE, so libmlx4 is perfectly appropriate for use with both versions of the card). The reason we use a modprobe.d file is that we configure the card's ports between the loading of mlx4_core and the loading of mlx4_ib and mlx4_en. This configuration requires reading a config file in /etc/rdma and as such it is very difficult and frowned upon to do so from the kernel module directly, hence why we use a user space helper, and hence why we load the modules via a modprobe.d file and not request module or other internal kernel mechanism. It does mean, however, that any time we have mlx4 hardware in the system, we should always be installing libmlx4 whether we ever plan to use the RDMA capabilities of the card or not. If there were some way to tell Anaconda that in the presence of mlx4 hardware to always install libmlx4, that would be my suggestion.
Joel, can you confirm that if you install the libmlx4 package the modules are loaded and you can use the hardware?
Sorry, no can do. Turns out my assistant when he couldn't get the ports to come up, hot RMAed them back to the vendor in exchange for intel cards. It was too late once he brought me in to look at it.
How exactly would anaconda know that mlx4 hardware is present?
You can use lspci to see that the hardware is present. But, really, I would like to see Anaconda just force install libcxgb3, libcxgb4, libmlx4, libmlx5, and rdma at all times. This can be accomplished in the comps file without modifying anaconda. But, these packages are all needed for proper hardware bringup, and in Fedora rawhide, the rdma package now sets up RDMA kernel modules automatically on the basis of a udev rule, so the entire group is fully automated. Bill, can we get comps modified to include the above packages at all times (well, some of the above packages may not have landed in Fedora yet, but any of them that have should be added to the comps base unit as mandatory)?
We don't include all firmware out of the box in the minimal install - I don't see why this would be different?
I don't know of any highly popular cards that we leave non-functional in the default install package set. At least the mlx4 and mlx5 hardware are not functional, not even the Ethernet interfaces, without their appropriate packages.
The libmlx4 package was obsoleted by rdma-core. If there's a desire to add that package to the default package sets, file an issue at https://pagure.io/fedora-comps/issues