Bug 232218

Summary: Character devices /dev/twa* for 3ware 9000 series RAID controllers are not created
Product: [Fedora] Fedora Reporter: Kevin Hobbs <kevin.hobbs.1>
Component: smartmontoolsAssignee: Tomas Smetana <tsmetana>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: medium Docs Contact:
Priority: medium    
Version: 8CC: alex, aradford, david.m.highley, jonstanley, joshuadfranklin, kay.sievers, k.georgiou
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
URL: https://www.redhat.com/archives/fedora-list/2007-March/msg00204.html
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2008-05-05 10:34:56 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 427887    

Description Kevin Hobbs 2007-03-14 13:41:39 UTC
The character devices, /dev/twa* are needed by smartd in order to monitor the
disks attached to 3ware 9000 series controllers. We have a 3ware Inc 9550SX
SATA-RAID controller and the device /dev/twa0 does not exist on boot. A udev
rule should exist to create these devices on boot. That udev rule should set the
security context fixed_disk_device_t on the created devices or smartd will not
be able to open them.

We are currently using udev-095-17.fc6, kernel-2.6.19-1.2911.6.5.fc6, and
smartmontools-5.36-3.2.fc6.

I currently have the 3ware monitor daemon 3DM2 installed. It creates the
devices. Once the devices exist, I am able to use smartctl to access the disks.
smartd is not able to access the disks because they do not have the correct
context. When I manually set the context of /dev/twa* to have
fixed_disk_device_t smartd is able to monitor the devices.

Comment 1 Harald Hoyer 2007-03-15 13:11:53 UTC
normally the kernel module provides the name through the /sys filesystem...

Comment 2 Kay Sievers 2007-03-19 22:39:34 UTC
What's the name of the kernel module that this device node talks to?

Udev does not need a rule to create a device node, the kernel-driver needs to
register the device with the kernel driver-core, and udev will create the device
node automatically.

Comment 3 Kevin Hobbs 2007-03-20 14:55:43 UTC
In reference to comment #1: Where in /sys should I look?

In reference to comment #2: The driver is 3w_9xxx. How do I tell if the device
is registered? I see "253 twa" in /proc/devices.

Comment 4 Kay Sievers 2007-03-21 22:18:49 UTC
Seems that the kernel-driver needs a fix, to register the device with the kernel
driver-core. It should probably just register a misc device with the name "twa"
and assign the major/minor to the kernel device-object before registration. When
this is done, udev will automatically create the device-node.

Comment 5 Daniel Walsh 2007-05-22 18:46:33 UTC
*** Bug 240892 has been marked as a duplicate of this bug. ***

Comment 6 Joshua Daniel Franklin 2007-07-05 20:46:50 UTC
Can someone give an example of a driver that does this correctly?

Comment 7 Joshua Daniel Franklin 2007-07-06 23:31:24 UTC
I sent a prospective patch to linux-scsi:
http://marc.info/?l=linux-scsi&m=118376386615731&w=2
I tried using a misc device first and it's cleaner but 
unfortunately they do not show up in /proc/devices which 
breaks smartctl, tw_cli, 3dmd, etc. so I don't think 
that will work. Instead I gleaned info from other 
drivers to create and populate a sysfs class. It is
very easy to drop in the new driver using the tools at:
http://www.truedesign.org/3ware-centos5/

I'm no sysfs expert but I have tested it on my system with a
3ware 7850 running RHEL5 kernel 2.6.18-8.1.6.el5. I'd rather 
not reboot a production system (one with multiple newer 3ware 
cards) just for testing.

Comment 8 Adam Radford 2007-07-09 21:33:24 UTC
I am using Fedora Core 6, and smartmontools-5.36.  If I specify
smartctl -d 3ware,0 /dev/twe0, it automatically creates the device nodes.

Also, "man smartctl" says:

Note  that  if  the special character device nodes /dev/twa? and
              /dev/twe? do not exist, or exist with  the  incorrect  major  or
              minor  numbers,  smartctl  will recreate them on the fly.

What exactly is this bug about?

-Adam


Comment 9 Joshua Daniel Franklin 2007-07-09 21:49:25 UTC
Basically the devices need to have the SELinux security context fixed_disk_device_t.

Do you have SELinux turned on? If so, you should get errors in /var/log/messages
including something like this:

SELinux is preventing /usr/sbin/smartd (fsdaemon_t) "getattr" access to device
/dev/twe0.       
For complete SELinux messages. run sealert -l ff5cfbed-5c29-4a1e-aeac-afbbc0059a36

and sealert says:

SELinux has denied the /usr/sbin/smartd (fsdaemon_t) "getattr" access to
device /dev/twe0. /dev/twe0 is mislabeled, this device has the default label
of the /dev directory, which should not happen.

comment #1 proposes the driver (3w-9xxx) create appropriate sysfs entries so
that udev will automatically set the correct SELinux security context (you can
check with "ls -Z /dev/tw*"). Alternatively smartd (and tw_cli and 3dmd and
3dmd2) could set the context.

Comment 10 Joshua Daniel Franklin 2007-07-09 23:39:48 UTC
I also see that the SELinux Reference Policy added an entry to
modules/kernel/storage.fc for "/dev/tw[a-z][^/]+" on 2007 March 26:
http://oss.tresys.com/projects/refpolicy/changeset/2246
Fedora 7 selinux-policy-2.6.4-25.fc7 includes this change,
but RHEL5 selinux-policy-2.4.6-30.el5 does not.

Arguably the 3ware kernel driver should still register a 
sysfs class, but the problem therefore appears to have
good workaround as of current Fedora, but is still a bug 
in RHEL5.

Comment 11 Kay Sievers 2007-07-09 23:56:02 UTC
In your patch, seems you never remove the registered class-device. That needs to
be fixed.

There will never be more than one device of that type?

And please use "struct device", "struct class_device" is going to be removed
from the kernel.

Comment 12 Bruce Allen 2007-08-24 12:38:05 UTC
You said that smartd could 'set the context'.  I would be happy to modify smartd/smartctl to do this.  But 
I'll need a bit more information about what is needed for smartd/smartctl to 'set the context'.  Could you 
provide a code fragment, or a documentation pointer, or both?

Bruce Allen

Comment 13 Joshua Daniel Franklin 2007-08-27 21:32:07 UTC
Unfortunately it looks like the SELinux Reference Policy
only sets the context on a restorecon, so the workaround
does not really help.

See thread at 
http://sourceforge.net/mailarchive/forum.php?thread_name=46C62EF7.9070600%40wanadoo.fr&forum_name=smartmontools-support

Comment 14 Jon Stanley 2008-01-08 01:50:16 UTC
(This is a mass-update to all current FC6 kernel bugs in NEW state)

Hello,

I'm reviewing this bug list as part of the kernel bug triage project, an attempt
to isolate current bugs in the Fedora kernel.

http://fedoraproject.org/wiki/KernelBugTriage

I am CC'ing myself to this bug, however this version of Fedora is no longer
maintained.

Please attempt to reproduce this bug with a current version of Fedora (presently
Fedora 8). If the bug no longer exists, please close the bug or I'll do so in a
few days if there is no further information lodged.

Thanks for using Fedora!

Comment 15 Kevin Hobbs 2008-01-11 17:55:52 UTC
I upgraded to Fedora 8. I am now able to use smartctl to access the disks
without tdm2. However when smartmond starts I get the following for each device:

Jan 11 12:47:06 backup kernel: audit(1200073626.585:9): avc:  denied  { getattr
} for  pid=2376 comm="smartd" path="/dev/twa0" dev=tmpfs ino=8573
scontext=unconfined_u:system_r:fsdaemon_t:s0
tcontext=unconfined_u:object_r:device_t:s0 tclass=chr_file
Jan 11 12:47:06 backup smartd[2376]: Device: /dev/twa0 [3ware_disk_00], File
exists, open() failed
Jan 11 12:47:06 backup smartd[2376]: Unable to register ATA device /dev/twa0
[3ware_disk_00] at line 3 of file /etc/smartd.conf
Jan 11 12:47:06 backup smartd[2376]: Device /dev/twa0 [3ware_disk_00] not available

Comment 16 Jon Stanley 2008-01-17 06:03:18 UTC
This looks like an SELinux thing at this point.  I've reassigned this over to
selinux-policy for some feedback there.  

Comment 17 Daniel Walsh 2008-01-17 16:58:07 UTC
Who created these disks?  


What does "matchpathcon /dev/twa0"  show?



Comment 18 Joshua Daniel Franklin 2008-01-18 15:58:03 UTC
If you mean the special devices, usually they are created by some sort of
management tools--either 3ware's command-line tw_cli or the 3dmd{,2} daemons, or
smartctl or smartd. Major/minor info comes from /proc/devices. Is there some
selinux code that would be added to smartmontools to apply the correct context
when the device files are created?

Note that my prospective patch (which needs cleaning up) fixes this issue
however I am puzzled that kernel code would be required for something that works
fine after relabeling. Adam Radford (3ware kernel module maintainer) has agreed
has expressed the same.

On my el5.1 test system I have done a relabel manually so this might be
worthless, but here goes (twe0 is for older 3ware cards):

$ matchpathcon /dev/twe0
/dev/twe0       system_u:object_r:fixed_disk_device_t:s0

$ rpm -qa selinux*
selinux-policy-2.4.6-106.el5_1.3
selinux-policy-targeted-2.4.6-106.el5_1.3

Comment 19 Daniel Walsh 2008-03-05 22:25:11 UTC
smartmontools should be using matchpathcon and setfscreatecon or setfilecon to
create the devices with the correct context.  If this is just a script it could
also just execute restorecon after creation of the devices.

So I think this goes back to smartmontools

Comment 20 Tomas Smetana 2008-05-05 10:34:56 UTC
I've committed a patch upstream and added to the recent rawhide version.

Comment 21 Joshua Daniel Franklin 2008-05-05 14:31:18 UTC
Beautiful! If any other tools need to add similar functionality they 
can just look at your patch:

http://smartmontools.cvs.sourceforge.net/smartmontools/sm5/os_linux.cpp?r1=1.105&r2=1.106

Comment 22 Daniel Walsh 2008-08-18 11:08:27 UTC
*** Bug 459369 has been marked as a duplicate of this bug. ***

Comment 23 Aleksandar Milivojevic 2008-10-12 03:42:28 UTC
Joshua, what happened with your kernel patch?  If you ever finished it, it would be a much better solution than relaying on smartmontools.  Kernel/udev should be able to create devices automatically for the hardware that's in the machine.  Applications shouldn't be responsible for creating devices.  Patching smartmontools/tw_cli (was there patch ever for tw_cli?) is simply wrong.  It's udev's job to create devices.  The real solution is to patch 3ware drivers to provide needed info, so that udev can create devices automatically (and correctly).