It is not necessary to reboot Fedora 10 to get the smartd service to generate the 'Permission denied, open failed' error messages.  It is sufficient to start (or stop and start) the smartd service.

Possibly related, the following two 'policy' packages were recently updated at the same time as the 'selinux-policy' packages:

$ sudo grep policycore /var/log/yum.log
Feb 13 10:31:54 Updated: policycoreutils-2.0.57-17.fc10.x86_64
Feb 13 10:34:48 Updated: policycoreutils-gui-2.0.57-17.fc10.x86_64

The previous version of these two packages was 2.0.57-14:

$ sudo grep policycore /var/log/rpmpkgs-20090208
policycoreutils-2.0.57-14.fc10.x86_64.rpm
policycoreutils-gui-2.0.57-14.fc10.x86_64.rpm

Prior to these updates (and reboot), the 'smartd' daemon did not issue the 'Permission denied, open failed' error messages.

Here is a temporary workaround that is working for me (I discovered it by accident while attempting to debug the problem).

1. Shutdown the smartd service if it is running:

   $ sudo service smartd stop

2. Start the smartd daemon directly via the command line, passing it the '--report' option to report all 'ioctl()' calls:

   $ sudo smartd --report ioctl

3. Examine /var/log/messages.  This listing will now show that 3ware_disk_00 and 3ware_disk_01 have been opened without error, interspersed among numerous IOCTL messages.  (I will included a copy of these messages as an attachment to this report.)

4. Shut down the running smartd daemon:

   $ sudo killall -s SIGTERM smartd

5. Start the smartd daemon via 'service':

   $ sudo service smartd start

6. Examine /var/log/messages.  No new error messages have been generated and the messages listed in the 'Expected results' section of the original report have been generated.

This is only a workaround.  Upon reboot, the same problem originally reported recurs and the above steps need to be performed to make it possible for the smartd service to open the /dev/twe0 device.

Created attachment 331955 [details]
Log of 'smartd -r ioctl' and subsequent 'service smartd start' messages.

Are you seeing avc messages in /var/log/audit/audit.log?

Yes, there are avc messages in /var/log/audit/audit.log.

I have (just now) generated a local.te file using the following command:

   audit2allow -m local -l -i /var/log/audit/audit.log > local.te

Here is the text of that local.te file:

----- snip ------

module local 1.0;

require {
        type fsdaemon_t;
        type fixed_disk_device_t;
        class chr_file { create unlink };
}

#============= fsdaemon_t ==============
allow fsdaemon_t fixed_disk_device_t:chr_file { create unlink };

----- snip -----

I notice a small (but significant?) difference from this file when compared with the one I generated for the 'twe2' message in the setroubleshooter, namely, in this file '{ create unlink }' is replaced with simply 'create'.  Is this significant?

I will attach a log of the avc messages in my /var/log/audit/audit.log file to this report.  It is somewhat large, so I will compress it using bzip2.  That is, it will be the result of these commands:

$ grep -i avc /var/log/audit/audit.log > avc.log
$ bzip2 avc.log

Created attachment 332091 [details]
Compressed log of avc messages in /var/log/audit/audit.log

Miroslav, you need to add


allow $1 fixed_disk_device_t:chr_file manage_chr_file_perms;

to 

storage_manage_fixed_disks interface in storage.if


For F9 and F10

(In reply to comment #8)
> Miroslav, you need to add
> 
> 
> allow $1 fixed_disk_device_t:chr_file manage_chr_file_perms;
> 
> to 
> 
> storage_manage_fixed_disks interface in storage.if
> 

If I add the above change to:

   /usr/share/selinux/devel/include/kernel/storage.if

after the line:

   allow $1 fixed_disk_device_t:blk_file manage_blk_file_perms;

(line 209)

and reboot, then should that be sufficient to resolve the problem?  Or are there additional steps that I should take?

Created attachment 332128 [details]
One-line change made to /usr/share/selinux/devel/kernel/storage.if

(In reply to comment #10)
> Created an attachment (id=332128) [details]
> One-line change made to /usr/share/selinux/devel/kernel/storage.if

After making this change, I rebooted and found that the problem had not been resolved.  Presumably, there are additional steps to take, but I was not able to find them in the manual pages that are included in the documentation of the selinux-policy package.

No this is not sufficient.

You would need to rebuild and reinstall the smartmon policy package from the rpm.

Adding the local customization using audit2allow like you stated above is what you need to do Mark, until Miroslav has a new policy out.

(In reply to comment #12)
> Adding the local customization using audit2allow like you stated above is what
> you need to do Mark, until Miroslav has a new policy out.

I would like to test the change and provide some confirmation of the suggested fix for you, if possible.  I appear to need additional instructions, because I have tried what I understand you have suggested and it is not working.  Here are the steps I have tried:

1. Compile and install a new policy file using the local.te listed in Comment #6:

   $ sudo checkmodule -M -m -o local.mod local.te
   $ sudo semodule_package -o local.pp -m local.mod
   $ sudo semodule -i local.pp

   $ sudo service smartd try-restart

Once the 'service' command is issued, the 'Permission denied, open failed' for /dev/twe0 appeared in /var/log/messages, and the setroubleshooter icon appeared due to selinux (correctly) denying smartd file creation privilege for /dev/twe2.

Should I have edited the 'local.te' described above before using it?

In case I misunderstood your instructions, I tried the following, also:

1. Copy the modified 'storage.if' file to /tmp

2. Issue the following command to generate a local.pp policy file:

$ sudo audit2allow -M local < storage.if
compilation failed:
local.te:6:ERROR 'syntax error' at token '' on line 6:


/usr/bin/checkmodule:  error(s) encountered while parsing configuration
/usr/bin/checkmodule:  loading policy configuration from local.te

3. Issue the alternative command:

$ sudo audit2allow -m local -l -i storage.if > local.te
$ cat local.te

module local 1.0;


At this time, I am still able to use 'smartd' after applying the workaround that I described in Comment #3.

Well, if I understand correctly, it still not working with your local policy in Comment #6.

Are you seeing another avc messages in /var/log/audit/audit.log with the active local policy for this ?

(In reply to comment #14)
> Well, if I understand correctly, it still not working with your local policy in
> Comment #6.
> 

That is correct -- it is still not working with the local policy in Comment #6 added.

> Are you seeing another avc messages in /var/log/audit/audit.log with the active
> local policy for this ?

There are fewer avc messages in /var/log/audit/audit.log with the active local policy.  The 'unlink' message is no longer appearing.  I have generated a new local policy based on the messages in audit.log.  Here are the results:

[tmp]$ date
Tue Feb 17 10:06:31 EST 2009
[tmp]$ sudo audit2allow -m local -l -i /var/log/audit/audit.log > local.te
[tmp]$ cat local.te

module local 1.0;

require {
        type fsdaemon_t;
        type fixed_disk_device_t;
        class chr_file create;
}

#============= fsdaemon_t ==============
allow fsdaemon_t fixed_disk_device_t:chr_file create;
[tmp]$ date
Tue Feb 17 10:06:55 EST 2009

Let me add that I do not think that we want the '...chr_file create' policy to work.  If it worked, then 'smartd' would create a /dev/twe2 device file.  'smartd' should not create a /dev/twe2 file.  Instead, 'smartd' should use (read? open?) the /dev/twe0 device file.  When 'smartd' fails its attempt to open /dev/twe0, it falls back on attempting to create /dev/twe2.  (Why does it not attempt to open /dev/twe1?  I do not know.)

But if the /dev/twe0 file does not exist it needs to be able to create it, correct?

(In reply to comment #17)
> But if the /dev/twe0 file does not exist it needs to be able to create it,
> correct?

I do not know with certainty, but I think that the security policy should not allow the 'device create' privilege.  As far as I know, /dev/twe0 has always existed since Fedora was installed (I have had this RAID working since Fedora Core 2, in 2005, but have not had 'smartmontools' working during all of that time).

I thought that 'udev' was responsible for creating /dev/twe0.  Keep in mind that when Fedora is installed via DVD, it has to be able to access the RAID (and its physical drives) without any reference to 'smartmontools'.  So, smartmontools should not *need* to create /dev/twe0.  In order for Fedora to work (to be bootable), it needs to be able to read/write to the RAID, presumably via the RAID device file /dev/tweN, independent of whether 'smartmontools' is installed and active.

It seems to me that the selinux policy with regard to not being able to create the /dev/tweN file is correct.  'smartmontools' apparently was designed to have this fall-back behavior in the event that it could not find the device file listed in its configuration file, but that design decision was (apparently) made in a "selinux" vacuum.

If 'smartmontools' ('smartd') has open/read access to /dev/twe0, then the (apparently redundant) 'create' privilege would not be needed.

AFAIK in some circumstances can be tweX device having wrong major:minor code, that's the reason why smartd removes it and creates new (but different) /dev/tweX. Smartd *needs* to have the possibility to remove and create the device node tweX or it won't work.

(In reply to comment #19)
> AFAIK in some circumstances can be tweX device having wrong major:minor code,
> that's the reason why smartd removes it and creates new (but different)
> /dev/tweX. Smartd *needs* to have the possibility to remove and create the
> device node tweX or it won't work.

Are you sure that is accurate?  Here is the documentation that sounds like what you are referring to in 'man 8 smartctl':

"Note that if the special character device  nodes  /dev/twa?  and
/dev/twe?  do  not  exist,  or exist with the incorrect major or
minor numbers, smartctl will recreate them on  the  fly."

There is no mention that I can find that says 'smartctl' will create a different /dev/tweX.

Here is the result of some commands where I tested 'smartctl' on the valid /dev/twe0 and on the invalid /dev/twe1 and /dev/twe2 device files:

$ date
Wed Feb 18 10:53:14 EST 2009

$ sudo smartctl -H -d 3ware,0 /dev/twe0
smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

$ sudo smartctl -H -d 3ware,1 /dev/twe0
smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

$ sudo smartctl -H -d 3ware,1 /dev/twe1
smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Smartctl open device: /dev/twe1 failed: No such device

$ sudo smartctl -H -d 3ware,0 /dev/twe2
smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

Smartctl open device: /dev/twe2 failed: No such device

$ date
Wed Feb 18 10:54:17 EST 2009

"...smartctl will recreate them on the fly..." = "...will create a
different..."

With different I mean different major:minor number

this is also reason why only first call recreates them

when you call it next time they exists with correct major:minor numbers and they are not recreated

this is the problem that was fixed in #456471 you've mentioned

(In reply to comment #17)
> But if the /dev/twe0 file does not exist it needs to be able to create it,
> correct?

OK, so, based on the above discussion in Comment #19 through Comment #21, my understanding of the answer to the question in Comment #17 is:

"Yes, if the special character device node /dev/twe0 does not exist, then 'smartd' needs to be able to create it, or if /dev/twe0 exists, but has incorrect major or minor numbers then recreate /dev/twe0 with the correct major/minor numbers."

The current erroneous behavior is:

   1) Fail to open /dev/twe0, which is listed in the configuration file
   2) Attempt to create /dev/twe2 (which is not listed in the configuration file), triggering setroubleshooter with the avc error.

(In reply to comment #22)
> The current erroneous behavior is:
> 
>    1) Fail to open /dev/twe0, which is listed in the configuration file

it won't open it because it's wrong file, it needs to recreate them first
= selinux fix is needed

>    2) Attempt to create /dev/twe2 (which is not listed in the configuration
> file), triggering setroubleshooter with the avc error.

it only creates them, and there's no problem with that

smartmontools/os_linux.cpp:int setup_3ware_nodes : 

157: /* Now check if nodes are correct */
158: for (index=0; index<16; index++) {

179:    /* Try to stat the node */
180:    if ((stat(nodestring, &stat_buf))) {
181:      /* Create a new node if it doesn't exist */
182:      if (mknod(nodestring, S_IFCHR|0600, makedev(tw_major, index))) {

190:    /* See if nodes major and minor numbers are correct */
191:    if ((tw_major != (int)(major(stat_buf.st_rdev))) ||
192:        (index    != (int)(minor(stat_buf.st_rdev))) ||
193:        (!S_ISCHR(stat_buf.st_mode))) {


even if you want to use only /dev/twe0 it still needs to unlink and create it (if its wrong) before it is able to open and use them. You can't get rid of the avc error if it only (re)create /dev/twe0 and not /dev/twe{1,2,...}

so all problems here have one common solution: selinux fix

> ... Keep in mind
> that when Fedora is installed via DVD, it has to be able to access the RAID
> (and its physical drives) without any reference to 'smartmontools'.

AFAIK (I've never seen real 3Ware controller) /dev/tweX files are not block devices, but character devices. They are used to communicate with controller, you don't need them if you want to "just use it".

Then it should work with your local policy seems like in Comment #6.

module local 1.0;

require {
        type fsdaemon_t;
        type fixed_disk_device_t;
        class chr_file { create unlink };
}

#============= fsdaemon_t ==============
allow fsdaemon_t fixed_disk_device_t:chr_file { create unlink };

(In reply to comment #24)
> Then it should work with your local policy seems like in Comment #6.
> 
> module local 1.0;
> 
> require {
>         type fsdaemon_t;
>         type fixed_disk_device_t;
>         class chr_file { create unlink };
> }
> 
> #============= fsdaemon_t ==============
> allow fsdaemon_t fixed_disk_device_t:chr_file { create unlink };

I cannot explain why, but it is working now with the local policy (as expected).  It has worked with two reboots without the use of the workaround that I described, above.  This includes a reboot after installing the latest kernel update, version 2.6.27-15-170.2.24.fc10.  I am also able to issue a 'service smartd restart' with triggering the setroubleshooter.  This revokes my problems described in Comment #15.

Previous behavior:

1) Upon reboot, 'smartd' was not able to open /dev/twe0 without error.
2) Using 'service smartd restart' would trigger setroubleshooter with the avc 'create' error for /dev/twe2.
3) Using 'smartd -r ioctl' would correct the /dev/twe0 permissions, and 'service smartd restart' would succeed without error.
4) Upon reboot 'smartd' would fail to open /dev/twe0.

Current behavior:

1) Upon reboot, 'smartd' is able to open /dev/twe0 without error.
2) Using 'service smartd restart' succeeds without error.

I updated the kernel and rebooted this morning at around 10 AM.  Here is some information about the /dev/twe?? files:

ls -l /var/log/boot.log /dev/twe? /dev/twe??
crw------- 1 root root 248,  0 2009-02-19 10:00 /dev/twe0
crw------- 1 root root 248,  1 2009-02-19 10:00 /dev/twe1
crw------- 1 root root 248, 10 2009-02-19 10:00 /dev/twe10
crw------- 1 root root 248, 11 2009-02-19 10:00 /dev/twe11
crw------- 1 root root 248, 12 2009-02-19 10:00 /dev/twe12
crw------- 1 root root 248, 13 2009-02-19 10:00 /dev/twe13
crw------- 1 root root 248, 14 2009-02-19 10:00 /dev/twe14
crw------- 1 root root 248, 15 2009-02-19 10:00 /dev/twe15
crw------- 1 root root 248,  2 2009-02-19 10:00 /dev/twe2
crw------- 1 root root 248,  3 2009-02-19 10:00 /dev/twe3
crw------- 1 root root 248,  4 2009-02-19 10:00 /dev/twe4
crw------- 1 root root 248,  5 2009-02-19 10:00 /dev/twe5
crw------- 1 root root 248,  6 2009-02-19 10:00 /dev/twe6
crw------- 1 root root 248,  7 2009-02-19 10:00 /dev/twe7
crw------- 1 root root 248,  8 2009-02-19 10:00 /dev/twe8
crw------- 1 root root 248,  9 2009-02-19 10:00 /dev/twe9
-rw-r--r-- 1 root root    3.0K 2009-02-19 10:00 /var/log/boot.log
[~]$ uptime
 14:50:19 up  4:50,  3 users,  load average: 0.47, 0.37, 0.28
[~]$ date
Thu Feb 19 14:50:24 EST 2009

Fixed in selinux-policy-3.5.13-46.fc10

(In reply to comment #26)
> Fixed in selinux-policy-3.5.13-46.fc10

I will report here the results that I get when this release of selinux-policy becomes available.

(Today, release 3.5.13-45.fc10 (Fedora Core 10?), became publicly available.  I installed it and rebooted.  The 'smartd' service continues to work without the 'open' error I originally reported.)

(In reply to comment #23)
> (In reply to comment #22)
> 
> 
> even if you want to use only /dev/twe0 it still needs to unlink and create it
> (if its wrong) before it is able to open and use them. You can't get rid of the
> avc error if it only (re)create /dev/twe0 and not /dev/twe{1,2,...}
> 
> so all problems here have one common solution: selinux fix
> 

Isn't the origin of the problem still not fixed?  That is, why was /dev/twe0 created incorrectly or why were its permissions set incorrectly?  (Otherwise, why would 'smartd' need to unlink/create the device file?)  That problem is not an selinux-policy problem, however, and so won't be addressed in this bugzilla report.

(In reply to comment #27)
> (In reply to comment #26)
> > Fixed in selinux-policy-3.5.13-46.fc10
> 
> I will report here the results that I get when this release of selinux-policy
> becomes available.
> 

Although there is no mention of this fix in the changelog of releases *-46 through *-49, I am not seeing any problems, and smartd is working as expected (that is, without any selinux-policy problems).  Because I have made local policy changes, I am not certain that this confirms that the fix is in selinux-policy.

$ rpm -q --changelog selinux-policy
* Fri Mar 13 2009 Miroslav Grepl <mgrepl> 3.5.13-49
- Add gpsd policy
- Fix razor policy
- Fix sysnet/net_conf_t

* Fri Mar 06 2009 Miroslav Grepl <mgrepl> 3.5.13-48
- Fix pcscd policy
- Allow alsa to read hardware state information

* Thu Feb 26 2009 Miroslav Grepl <mgrepl> 3.5.13-47
- Allow ktalkd to write to terminals
- Fix qemu labeling
- Fix mysqld_safe policy

* Thu Feb 19 2009 Miroslav Grepl <mgrepl> 3.5.13-46
- Fix squidGuard labeling
- Allow ftpd to list inotifyfs

How I reported in comment #26, the fix was added to selinux-policy-3.5.13-46.fc10

It should work with an actual selinux-policy for this problem without your local policy.

I am seeing the same behavior, cf. following attachment. I am running a fully updated F10; in particular:
[root@moby ~]# rpm -q smartmontools selinux-policy selinux-policy-targeted
smartmontools-5.38-7.fc10.x86_64
selinux-policy-3.5.13-59.fc10.noarch
selinux-policy-targeted-3.5.13-59.fc10.noarch

Created attachment 345677 [details]
smartd log messages and output of sealert

Doing the restorecon that sealert suggested changed the context of /dev/tw0 from system_u:object_r:device_t:s0 to system_u:object_r:fixed_disk_device_t:s0, but now restarting the smartd service creates a new /dev/twa1 and gives the smae errors on the newly created (and irrelevant) device.

I haven't had time to digest the earlier comments and sort out the if any of the suggested fixes are supposed to work. Sigh.

(In reply to comment #31)
> I am seeing the same behavior, cf. following attachment. I am running a fully
> updated F10; in particular:
> [root@moby ~]# rpm -q smartmontools selinux-policy selinux-policy-targeted
> smartmontools-5.38-7.fc10.x86_64
> selinux-policy-3.5.13-59.fc10.noarch
> selinux-policy-targeted-3.5.13-59.fc10.noarch  

I am running the versions of these packages that you reported, above (including 'x86_64').

For what it's worth, I have not seen any problems since I reported that the fix to selinux-policy had been added.  That includes numerous updates to the kernel, selinux-policy, and selinux-policy-targeted, along with corresponding reboots.  In addition to 'smartd' not having any problems reading the drives, it routinely (daily) runs diagnostic tests on the drives and reports back their status without problems.

None of this helps you to diagnose the source of your problem, but it should be reason for you to believe that a solution can likely be found.

Joe what context is smartmon running under
ps -eZ | grep smartd

The policy says that fsdaemon_t creating a a device in /dev (device_t) will create it labeled fixed_disk_device_t

You are stating that this is not happening.  So either you do not have the correct policy running or something not labeled fsdaemon_t is creating the device.

Dan - sorry I didn't see your last comment until now. FWIW, my problem is solved under F11 w/ selinux 3.6.12-62 and smartmontools 5.38-11