What content/directory are they trying to write in /var/lib?

Also could you re-test it with

# echo "-w /etc/shadow -p w" >> /etc/audit/audit.rules
# service auditd restart
# semanage permissive -a fenced_t

and then attach AVC msgs.

reproducer:
2-node cluster
instructions as in comment #2 on node#1
on node#2; killall -9 corosync

node#1 will automatically run fencing of the other node and fence_apc_snmp will be running in correct context


commands executed from fence_apc_snmp:
---

/usr/bin/snmpwalk -m '' -Oeqn  -v '1' -c 'private' 'XXX:161' '.1.3.6.1.2.1.1.2.0'
.1.3.6.1.2.1.1.2.0 .1.3.6.1.4.1.318.1.3.4.5

Trying APC rPDU
/usr/bin/snmpget -m '' -Oeqn  -v '1' -c 'private' 'XXX:161' '.1.3.6.1.4.1.318.1.1.12.3.5.1.1.4.2'
.1.3.6.1.4.1.318.1.1.12.3.5.1.1.4.2 2

/usr/bin/snmpset -m '' -Oeqn  -v '1' -c 'private' 'XXX:161' '.1.3.6.1.4.1.318.1.1.12.3.3.1.1.4.2' i '1'
.1.3.6.1.4.1.318.1.1.12.3.3.1.1.4.2 1

/usr/bin/snmpget -m '' -Oeqn  -v '1' -c 'private' 'XXX:161' '.1.3.6.1.4.1.318.1.1.12.3.5.1.1.4.2'
.1.3.6.1.4.1.318.1.1.12.3.5.1.1.4.2 1

----
obtained AVC messages:

type=AVC msg=audit(1335191180.860:77): avc:  denied  { read } for  pid=2254 comm="fence_apc_snmp" name="pyconfig-64.h" dev="vda2" ino=131302 scontext=system_u:system_r:fenced_t:s0 tcontext=system_u:object_r:usr_t:s0 tclass=file
type=AVC msg=audit(1335191180.860:77): avc:  denied  { open } for  pid=2254 comm="fence_apc_snmp" name="pyconfig-64.h" dev="vda2" ino=131302 scontext=system_u:system_r:fenced_t:s0 tcontext=system_u:object_r:usr_t:s0 tclass=file
type=AVC msg=audit(1335191180.860:78): avc:  denied  { getattr } for  pid=2254 comm="fence_apc_snmp" path="/usr/include/python2.7/pyconfig-64.h" dev="vda2" ino=131302 scontext=system_u:system_r:fenced_t:s0 tcontext=system_u:object_r:usr_t:s0 tclass=file
type=AVC msg=audit(1335191180.864:79): avc:  denied  { search } for  pid=2254 comm="fence_apc_snmp" name=".local" dev="vda2" ino=16008 scontext=system_u:system_r:fenced_t:s0 tcontext=system_u:object_r:gconf_home_t:s0 tclass=dir
type=AVC msg=audit(1335191180.882:80): avc:  denied  { open } for  pid=2254 comm="fence_apc_snmp" name="1" dev="devpts" ino=4 scontext=system_u:system_r:fenced_t:s0 tcontext=system_u:object_r:devpts_t:s0 tclass=chr_file
type=AVC msg=audit(1335191180.891:81): avc:  denied  { read } for  pid=2255 comm="snmpwalk" name="cert_indexes" dev="vda2" ino=15735 scontext=system_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=dir
type=AVC msg=audit(1335191180.891:82): avc:  denied  { read } for  pid=2255 comm="snmpwalk" name="0" dev="vda2" ino=18257 scontext=system_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file
type=AVC msg=audit(1335191180.891:82): avc:  denied  { open } for  pid=2255 comm="snmpwalk" name="0" dev="vda2" ino=18257 scontext=system_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file
type=AVC msg=audit(1335191180.892:83): avc:  denied  { getattr } for  pid=2255 comm="snmpwalk" path="/var/lib/net-snmp/mib_indexes/0" dev="vda2" ino=18257 scontext=system_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file

I am adding fixes. The problem is 


/var/lib/net-snmp

is mislabeled.

$ restorecon -R -v /var/lib/net-snmp

how did you create this directory?

Unit test [need 2 computers A,B; each step has to be performed on both nodes]

1. install "basic server"
2a. yum install ricci rgmanager
2b. rpm -Uvh selinux-policy selinux-policy-targeted (latest version)
3. add A,B to /etc/hosts; set /etc/hosts.conf to 'Order hosts,bind'
   3.a -> ping A, ping B; host A, host B = should work on both nodes
4. create cluster.conf (only two nodes and fence_apc_snmp as fence device; nodes do not need to be connected to that device at all)
5. (approx same time at both nodes) setenforce 0; /etc/init.d/iptables stop; /etc/init.d/cman start
    5.a -> clustat = you should see cluster now

6. on node A: killall -9 corosync
7. on node B: copy AVC from /var/log/auditd/audit.log

Results:

type=AVC msg=audit(1337245966.458:36): avc:  denied  { read } for  pid=2154 comm="snmpwalk" name="0" dev=dm-0 ino=142597 scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file
type=AVC msg=audit(1337245966.458:36): avc:  denied  { open } for  pid=2154 comm="snmpwalk" name="0" dev=dm-0 ino=142597 scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file
type=AVC msg=audit(1337245966.459:37): avc:  denied  { getattr } for  pid=2154 comm="snmpwalk" path="/var/lib/net-snmp/mib_indexes/0" dev=dm-0 ino=142597 scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file

--

Directory /var/lib/net-snmp is not created during installation. 

Running:
restorecon -R -v /var/lib/net-snmp
reboot
run cluster again (steps 5+)

There are not AVC messages

That's interesting. The directory is definitely created on my machine during the installation of net-snmp package which owns the directory. And the directory is labelled correctly.

# ls -lR /var/lib/net-snmp/
/var/lib/net-snmp/:
total 12
drwx------. 2 root root 4096 Mar 22 14:10 mib_indexes
-rw-------. 1 root root 1149 May 12 13:31 snmpd.conf
-rw-------. 1 root root  681 May 10 20:26 snmptrapd.conf

/var/lib/net-snmp/mib_indexes:
total 4
-rw-r--r--. 1 root root 2476 May 10 22:43 0
# rm -rf /var/lib/net-snmp/
# yum reinstall net-snmp
Loaded plugins: product-id, refresh-packagekit, subscription-manager
Updating certificate-based repositories.
Unable to read consumer identity
Setting up Reinstall Process
Resolving Dependencies
--> Running transaction check
---> Package net-snmp.x86_64 1:5.5-40.el6 will be reinstalled
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package         Arch          Version             Repository              Size
================================================================================
Reinstalling:
 net-snmp        x86_64        1:5.5-40.el6        RHEL-6.3-Server        301 k

Transaction Summary
================================================================================
Reinstall     1 Package(s)

Total download size: 301 k
Installed size: 816 k
Is this ok [y/N]: y
Downloading Packages:
net-snmp-5.5-40.el6.x86_64.rpm                           | 301 kB     00:00     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : 1:net-snmp-5.5-40.el6.x86_64                                 1/1 
Installed products updated.
  Verifying  : 1:net-snmp-5.5-40.el6.x86_64                                 1/1 

Installed:
  net-snmp.x86_64 1:5.5-40.el6                                                  

Complete!
# ls -lR /var/lib/net-snmp/
/var/lib/net-snmp/:
total 0
# ls -dZ /var/lib/net-snmp/
drwxr-xr-x. root root system_u:object_r:snmpd_var_lib_t:SystemLow /var/lib/net-snmp/
#

I don't have package net-snmp installed. Relevant installed packages: net-snmp-libs a net-snmp-utils

So if you remove this directory and run your scenario then the directory exists, right?

Yes, directory is created during fence action.

Net-SNMP maintainer here.

I am not sure I understand the problem correctly, but /var/lib/net-snmp is created by net-snmp package. If someone uses only net-snmp-libs + net-snmp-utils, it is not created during package installation. If snmpwalk then reads MIB files, it creates the directory (probably with wrong selinux context). Is this the problem you see here?

As a solution, I might move the directory from net-snmp to net-snmp-libs.

(In reply to comment #16)
 
> As a solution, I might move the directory from net-snmp to net-snmp-libs.

That would be great.

Yes, I like this solution.

Any chance to get it to RHEL6.3?

Or do we need to find out how the directory is exactly created?

/var/lib/net-snmp is moved to net-snmp-libs in net-snmp-5.5-41.el6.

Created attachment 585780 [details]
Cluster.conf

Yours cluster.conf did not contain relation between fencing device and nodes. Valid cluster.conf is in attachement

Created attachment 585781 [details]
Cluster.conf

Yours cluster.conf did not contain relation between fencing device and nodes. Valid cluster.conf is in attachement

Created attachment 585972 [details]
AVCs logged during automated test

We have created automated test performing the steps described above and with installed net-snmp-utils-5.5-41, AVCs stored in attachement appeared during the test. (in ENFORCING mode)

(In reply to comment #26)
> Created attachment 585972 [details]
> AVCs logged during automated test
> 
> We have created automated test performing the steps described above and with
> installed net-snmp-utils-5.5-41, AVCs stored in attachement appeared during
> the test. (in ENFORCING mode)

If you get AVC msgs in enforcing mode, I need to see also AVC msgs in permissive mode. Thank you.

Only these AVCs showed up in Permisive mode (the test might be affected by the first run):

----
time->Tue May 22 05:06:48 2012
type=SYSCALL msg=audit(1337677608.211:80): arch=c000003e syscall=2 success=yes exit=3 a0=92a0a0 a1=90800 a2=7f2122e62320 a3=0 items=0 ppid=10565 pid=10568 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=2 comm="ccs_config_dump" exe="/usr/sbin/ccs_config_dump" subj=unconfined_u:system_r:corosync_t:s0 key=(null)
type=AVC msg=audit(1337677608.211:80): avc:  denied  { read } for  pid=10568 comm="ccs_config_dump" name="tmp.2GOG8ZUirf" dev=dm-0 ino=1572892 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=unconfined_u:object_r:user_tmp_t:s0 tclass=dir
----
time->Tue May 22 05:07:40 2012
type=SYSCALL msg=audit(1337677660.802:81): arch=c000003e syscall=2 success=yes exit=4 a0=7fffeb17ca30 a1=241 a2=1b6 a3=0 items=0 ppid=10891 pid=10892 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=2 comm="snmpwalk" exe="/usr/bin/snmpwalk" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337677660.802:81): avc:  denied  { write } for  pid=10892 comm="snmpwalk" name="0" dev=dm-0 ino=2502037 scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
type=AVC msg=audit(1337677660.802:81): avc:  denied  { create } for  pid=10892 comm="snmpwalk" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file

Sorry, I found most of them was caused by changing to temp directory. So now in Enforcing:

----
time->Tue May 22 05:32:16 2012
type=SYSCALL msg=audit(1337679136.561:133): arch=40000003 syscall=5 success=no exit=-13 a0=bf8c4490 a1=8241 a2=1b6 a3=8db7a0 items=0 ppid=17623 pid=17624 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpwalk" exe="/usr/bin/snmpwalk" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679136.561:133): avc:  denied  { create } for  pid=17624 comm="snmpwalk" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
----
time->Tue May 22 05:32:16 2012
type=SYSCALL msg=audit(1337679136.920:134): arch=40000003 syscall=5 success=no exit=-13 a0=bfc017c0 a1=8241 a2=1b6 a3=31e7a0 items=0 ppid=17623 pid=17625 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpget" exe="/usr/bin/snmpget" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679136.920:134): avc:  denied  { create } for  pid=17625 comm="snmpget" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
----
time->Tue May 22 05:32:17 2012
type=SYSCALL msg=audit(1337679137.154:135): arch=40000003 syscall=5 success=no exit=-13 a0=bfa77790 a1=8241 a2=1b6 a3=1a77a0 items=0 ppid=17623 pid=17626 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpset" exe="/usr/bin/snmpset" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679137.154:135): avc:  denied  { create } for  pid=17626 comm="snmpset" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
----
time->Tue May 22 05:32:17 2012
type=SYSCALL msg=audit(1337679137.474:136): arch=40000003 syscall=5 success=no exit=-13 a0=bfaaa3e0 a1=8241 a2=1b6 a3=b007a0 items=0 ppid=17623 pid=17627 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpget" exe="/usr/bin/snmpget" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679137.474:136): avc:  denied  { create } for  pid=17627 comm="snmpget" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
----
time->Tue May 22 05:32:17 2012
type=SYSCALL msg=audit(1337679137.704:137): arch=40000003 syscall=5 success=no exit=-13 a0=bfd5cab0 a1=8241 a2=1b6 a3=8667a0 items=0 ppid=17623 pid=17628 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpset" exe="/usr/bin/snmpset" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679137.704:137): avc:  denied  { create } for  pid=17628 comm="snmpset" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
----
time->Tue May 22 05:32:18 2012
type=SYSCALL msg=audit(1337679138.036:138): arch=40000003 syscall=5 success=no exit=-13 a0=bfdc4020 a1=8241 a2=1b6 a3=1ed7a0 items=0 ppid=17623 pid=17629 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpget" exe="/usr/bin/snmpget" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679138.036:138): avc:  denied  { create } for  pid=17629 comm="snmpget" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file



And in Permisive:

time->Tue May 22 05:34:57 2012
type=SYSCALL msg=audit(1337679297.857:140): arch=40000003 syscall=5 success=yes exit=4 a0=bff3c1c0 a1=8241 a2=1b6 a3=48b7a0 items=0 ppid=18180 pid=18181 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts1 ses=11 comm="snmpwalk" exe="/usr/bin/snmpwalk" subj=unconfined_u:system_r:fenced_t:s0 key=(null)
type=AVC msg=audit(1337679297.857:140): avc:  denied  { write } for  pid=18181 comm="snmpwalk" name="0" dev=dm-0 ino=1856783 scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file
type=AVC msg=audit(1337679297.857:140): avc:  denied  { create } for  pid=18181 comm="snmpwalk" name="0" scontext=unconfined_u:system_r:fenced_t:s0 tcontext=unconfined_u:object_r:snmpd_var_lib_t:s0 tclass=file

Ok, I am adding fixes.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0780.html