Bug 797776

Summary: ISCSI related modules ( such as cxgb3i ) are loaded even if no ISCSI device (such as Chelsio) is installed on some beaker machines
Product: Red Hat Enterprise Linux 5 Reporter: Qian Hong <qhong>
Component: iscsi-initiator-utilsAssignee: Mike Christie <mchristi>
Status: CLOSED DUPLICATE QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: high    
Version: 5.8CC: coughlan, mchristi, nhorman
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-03-02 21:45:00 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Log: configure files for udev none

Description Qian Hong 2012-02-27 09:42:22 UTC
Description of problem:

We first notice this issue via running network driver testing on beaker machines.

Some beaker machine has no Chelsio iscsi device installed, but cxgb3i module is loaded by default. As a dependency, cxgb3 network driver is loaded too. This is annoying for network driver testing, since we want to auto select a machine from beaker with cxgb3 network driver, but most beaker machines with cxgb3 module has no cxgb3 network device at all. 

After that we found be2iscsi, bnx2i are also loaded by default in some machines without ISCSI device. As far as I know the last two are harmless for our network dirver testing, since bnx2i iscsi driver doesn't depends on bnx2 network driver,
and be2iscsi doesn't depends on be2net network driver.

However, until beaker Bug 731615 is fixed, we have no other workaround to automate our cxgb3 driver testing.

Here is a list of machines which have cxgb3 module loaded by default in beaker:
http://goo.gl/Eo5uL
(Take care on another beaker bug: Bug 797720)
The above list give us 241 machines, however, if we search by device instead of by modules, we only get 7 machines:
http://goo.gl/Lbecl

Below is an example with cxgb3i loaded but without any iscsi device installed. 


Version-Release number of selected component (if applicable):
# uname -a
Linux dell-pe700-01.rhts.eng.bos.redhat.com 2.6.18-308.el5 #1 SMP Fri Jan 27 17:21:15 EST 2012 i686 i686 i386 GNU/Linux

# lsb_release -a
LSB Version:	:core-4.0-ia32:core-4.0-noarch:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-ia32:printing-4.0-noarch
Distributor ID:	RedHatEnterpriseServer
Description:	Red Hat Enterprise Linux Server release 5.8 (Tikanga)
Release:	5.8
Codename:	Tikanga

# udevinfo -V
udevinfo, version 095

# lsmod | grep cxgb
cxgb3i                 31177  0 
libcxgbi               54477  1 cxgb3i
cxgb3                 167705  1 cxgb3i
8021q                  25929  1 cxgb3
libiscsi_tcp           22213  3 cxgb3i,libcxgbi,iscsi_tcp
libiscsi2              42693  7 cxgb3i,libcxgbi,be2iscsi,ib_iser,iscsi_tcp,bnx2i,libiscsi_tcp
scsi_transport_iscsi2    37709  8 libcxgbi,be2iscsi,ib_iser,iscsi_tcp,bnx2i,libiscsi2
scsi_mod              144277  12 libcxgbi,be2iscsi,ib_iser,iscsi_tcp,bnx2i,libiscsi2,scsi_transport_iscsi2,scsi_dh,sg,libata,aacraid,sd_mod


# lspci | grep -i chelsio
# lspci | grep -i iscsi
/* no result */

# iscsiadm -m iface
default tcp,<empty>,<empty>,<empty>,<empty>
iser iser,<empty>,<empty>,<empty>,<empty>

# modprobe -rv cxgb3
FATAL: Module cxgb3 is in use.

# modprobe -rv cxgb3i
rmmod /lib/modules/2.6.18-308.el5/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko
rmmod /lib/modules/2.6.18-308.el5/kernel/drivers/scsi/cxgbi/libcxgbi.ko
rmmod /lib/modules/2.6.18-308.el5/kernel/drivers/net/cxgb3/cxgb3.ko
rmmod /lib/modules/2.6.18-308.el5/kernel/net/8021q/8021q.ko


How reproducible:
100%

Steps to Reproduce:
1. Reserve dell-pe700-01.rhts.eng.bos.redhat.com in beaker
2. lsmod | grep cxgb

  
Actual results:
both cxgb3 and cxgb3i are loaded

Expected results:
Neither cxgb3 nor cxgb3i should be loaded

Additional info:
It is great if some one double check, I'm not so similar with ISCSI, not sure whether this should be consider as a bug.

Comment 1 Qian Hong 2012-02-27 09:52:19 UTC
Created attachment 565999 [details]
Log: configure files for udev

.
|-- makedev.d
|   `-- 50-udev.nodes
`-- rules.d
    |-- 05-udev-early.rules
    |-- 40-multipath.rules
    |-- 50-udev.rules
    |-- 51-hotplug.rules
    |-- 60-net.rules
    |-- 60-pcmcia.rules
    |-- 60-raw.rules
    |-- 85-pcscd_ccid.rules
    |-- 90-dm.rules
    |-- 90-hal.rules
    |-- 95-pam-console.rules
    `-- bluetooth.rules

Comment 2 Neil Horman 2012-03-02 14:33:28 UTC
I've taken a look at this, and its a user space problem.  the iscsid start script (/etc/init.d/iscsid) unilaterally loads all the iscsi transport modules.  While this probably makes some sense (you can't know which iscsi transport a user wants to use at the time you start the daemon),  the daemon can do a better job by only loading those transports for which underlying hardware is available (except for iscsi_tcp, which will always need to be loaded).  I expect what it can do is:

1) build a list of iscsi enabled network drivers

2) scan /sys/class/net/<if>/device/driver/module.  If the basename of that symlink matches an item on the list in (1), load the corresponding iscsi_transport (i.e. load cxgb3i if one of the network interfaces is driven by the cxgb3 driver).

3) add some udev rules to monitor for module load events, and, if iscsid is running, load the corresponding iscsi transport module.

If we do that, we should only load the iscsi transport modules for the hardware that we have available on a given system.

Comment 3 Mike Christie 2012-03-02 21:45:00 UTC
This is a dup of
https://bugzilla.redhat.com/show_bug.cgi?id=723669

We are just going to have iscsiadm load the module when there is a command that is run that wants to use it.

For upstream we are going to use the kmod lib. For RHEL it turned out we are not supporting that lib, so I have to do some modprobe magic kexec call from what I understand.

*** This bug has been marked as a duplicate of bug 723669 ***