Bug 869136

Michal,
where is checkquorum.wdmd located?

/usr/share/cluster/checkquorum.wdmd

Ok, I see we have

/usr/share/cluster/checkquorum    --  gen_context(system_u:object_r:bin_t,s0)

which I am fixing.

I added fixes to Fedora. Will backport them. But not sure why

type=SYSCALL msg=audit(1350922595.889:35): arch=40000003 syscall=195 success=no exit=-13 a0=92e1018 a1=bfd3338c a2=df1ff4 a3=3 items=0 ppid=8907 pid=8920 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="which" exe="/usr/bin/which" subj=unconfined_u:system_r:corosync_t:s0 key=(null)
type=AVC msg=audit(1350922595.889:35): avc:  denied  { getattr } for  pid=8920 comm="which" path="/sbin/setfiles" dev=dm-0 ino=139103 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=system_u:object_r:setfiles_exec_t:s0 tclass=file

is needed by corosync.

comm="which" path="/sbin/setfiles"

Hi guys,

checkquorum and checkquorum.wdmd come from cman package, but neither of those calls directly /usr/bin/which nor does corosync init.

so i am a bit puzzled on how you got there.....

(In reply to comment #5)
> I added fixes to Fedora. Will backport them. But not sure why
> 
> type=SYSCALL msg=audit(1350922595.889:35): arch=40000003 syscall=195
> success=no exit=-13 a0=92e1018 a1=bfd3338c a2=df1ff4 a3=3 items=0 ppid=8907
> pid=8920 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0
> fsgid=0 tty=(none) ses=4294967295 comm="which" exe="/usr/bin/which"
> subj=unconfined_u:system_r:corosync_t:s0 key=(null)
> type=AVC msg=audit(1350922595.889:35): avc:  denied  { getattr } for 
> pid=8920 comm="which" path="/sbin/setfiles" dev=dm-0 ino=139103
> scontext=unconfined_u:system_r:corosync_t:s0
> tcontext=system_u:object_r:setfiles_exec_t:s0 tclass=file
> 
> is needed by corosync.
> 
> comm="which" path="/sbin/setfiles"

Miroslav,

WHY is resource agents using corosync context? This is NO NO NO. Nether rgmanager nor pacemaker has something to do with corosync other then using it's services. Corosync itself doesn't need to execute find or which or whichever command.

I would recommend to create extra context for pacemaker and/or resource agents and their own rules. Because if you will allow corosync run everything (because totally unrelated resource agents are call it), whole SELinux is simply useless.

(In reply to comment #7)
> WHY is resource agents using corosync context? This is NO NO NO. Nether rgmanager nor pacemaker has > something to do with corosync other then using it's services. Corosync itself doesn't need to execute find > or which or whichever command.

It's easy to grunt. Maybe you could tell us what corosync really does, so we don't need to guess.

We have this for these resource agents and pacemaker. We know it is hard to confine them so rgmanager is allowed to do "everything".

So the question is why it runs as corosync_t.

(In reply to comment #8)
> (In reply to comment #7)
> > WHY is resource agents using corosync context? This is NO NO NO. Nether rgmanager nor pacemaker has > something to do with corosync other then using it's services. Corosync itself doesn't need to execute find > or which or whichever command.
> 
> It's easy to grunt. Maybe you could tell us what corosync really does, so we
> don't need to guess.

https://github.com/corosync/corosync is what it does. I believe your question should be "what it doesn't". And answer is exec* call.

Michal,
could you ping me? We need to look at the test more.

Ok, I got it.

The problem is cman running as corosync_t which is obviously wrong. Basically cman needs lot of accesses which corosync has but now we know it wants to add more.

So I am just testing the cman_tool running with rgmanager_t and it looks fine. Basically I believe this is a correct labeling for the cman_tool instead of corosync. Also corosync policy can be cleaned.

Michal,
you can test it using

# chcon -t rgmanger_exec_t /usr/sbin/cman_tool /usr/sbin/ccs_tool

> The problem is cman running as corosync_t which is obviously wrong.
> Basically cman needs lot of accesses which corosync has but now we know it
> wants to add more.hmm no guys, that doesn't sound right.

cman (service/core) run within corosync daemon.

cman doesn't need rgmanager, rgmanager needs cman. cman can run without rgmanager even being installed.

ccs_tool is the same as cman_tool. ccs uses libccs to access corosync internals.

I don't see a logical reason to have cman as rgmanager_exec_t.

What I don't understand is:

- why are we seeing this regression from 6.3 to 6.4?
- from which version of fedora are you guys back-porting those rules?
  corosync in f17 and higher works completely different from f16/rhel6.
  so you need to be careful.

Well, these cluster stuff are really hard to test for us. We really need your help. We just run the basic tests.

Why do we get it? This is not a question for us. Is cman_tool supposed to run random scripts? If yes (and it looks so) then rgmanger_exec_t is correct labeling.  SELinux labeling does not depend on installed packages.

Fabio,
are you able to tell me why we get

type=AVC msg=audit(1350922595.889:35): avc:  denied  { getattr } for  pid=8920 comm="which" path="/sbin/setfiles" dev=dm-0 ino=139103 scontext=unconfined_u:system_r:corosync_t:s0 tcontext=system_u:object_r:setfiles_exec_t:s0 tclass=file

which scripts does cman_tool run?

Thank you for your help.

(In reply to comment #15)
> Well, these cluster stuff are really hard to test for us. We really need
> your help. We just run the basic tests.

I am very happy to help you, but i don´t understand selinux, so we need to work together a bit here to understand each other :)

> 
> Why do we get it? This is not a question for us.

all I am asking is what changed in selinux (or policy.. or..) that might have introduced those new AVC. I know more or less all changes that have gone into cluster from 6.3, so pretty much I have an overview of what might be causing stuff.

> Is cman_tool supposed to
> run random scripts?

No, cman_tool only run a very strict set of script. Those are ccs_sync/ccs_config_validate.

ccs_config_validate calls in turns a bunch of stuff. From grepping the code, there is no call to setfiles (might be ccs_sync from ricci package).

Now, I suspect, but can´t be sure that the /usr/share/cluster/*.sh AVC you saw before are due to ccs_config_validate updating its cache. It does so by running those files with a metadata option and extract some information. Tho it´s not real runtime execution as rgmanager does.

The difference is:

cman_tool -> ccs_config_validate -> ccs_update_schema -> (agents $metadata)> cache.

rgmanger -> run agents as root to perform real stuff execution.

It´s a tiny difference if you can spot it.

> If yes (and it looks so) then rgmanger_exec_t is correct
> labeling.

It doesn´t sound right at logical level tho. but it´s just me.

>  SELinux labeling does not depend on installed packages.

Right, but if somebody has to manually fix cman_tool labeling, i doubt they would expect it to be rgmanager since the two are not necessarily related.

> 
> Fabio,
> are you able to tell me why we get
> 
> type=AVC msg=audit(1350922595.889:35): avc:  denied  { getattr } for 
> pid=8920 comm="which" path="/sbin/setfiles" dev=dm-0 ino=139103
> scontext=unconfined_u:system_r:corosync_t:s0
> tcontext=system_u:object_r:setfiles_exec_t:s0 tclass=file

No because none of my code calls directly which or setfiles. so it must be some tools we invoke underneath that does it.

> Thank you for your help.

Can we perhaps meet on IRC and discuss your test setup and see if I can help you trigger the various AVC step-by-step so we can see what´s happening, what´s supposed to be happening etc?

(In reply to comment #16)
> (In reply to comment #15)
> > Well, these cluster stuff are really hard to test for us. We really need
> > your help. We just run the basic tests.
> 
> I am very happy to help you, but i don´t understand selinux, so we need to
> work together a bit here to understand each other :)
> 
> > 
> > Why do we get it? This is not a question for us.
> 
> all I am asking is what changed in selinux (or policy.. or..) that might
> have introduced those new AVC. I know more or less all changes that have
> gone into cluster from 6.3, so pretty much I have an overview of what might
> be causing stuff.
> 
> > Is cman_tool supposed to
> > run random scripts?
> 
> No, cman_tool only run a very strict set of script. Those are
> ccs_sync/ccs_config_validate.
> 
> ccs_config_validate calls in turns a bunch of stuff.

This is important.

> From grepping the code,
> there is no call to setfiles (might be ccs_sync from ricci package).
> 
> Now, I suspect, but can´t be sure that the /usr/share/cluster/*.sh AVC you
> saw before are due to ccs_config_validate updating its cache. It does so by
> running those files with a metadata option and extract some information. Tho
> it´s not real runtime execution as rgmanager does.
> 
> The difference is:
> 
> cman_tool -> ccs_config_validate -> ccs_update_schema -> (agents $metadata)>
> cache.

Which means you will end up with corosync_t which is not obviously the right domain for this. And this is reason why I suggested rgmanager.

> 
> rgmanger -> run agents as root to perform real stuff execution.
>

(In reply to comment #17)

> > No, cman_tool only run a very strict set of script. Those are
> > ccs_sync/ccs_config_validate.
> > 
> > ccs_config_validate calls in turns a bunch of stuff.
> 
> This is important.
> 

> > 
> > cman_tool -> ccs_config_validate -> ccs_update_schema -> (agents $metadata)>
> > cache.
> 
> Which means you will end up with corosync_t which is not obviously the right
> domain for this. And this is reason why I suggested rgmanager.
> 

but it still doesn't answer my question.

This setup has not changed since it was introduced in 6.1 or 6.2.

It's not been a problem in 6.3, but why it's becoming an issue in 6.4?

This code has not changed in between 6.3 and 6.4. So there must be something else triggering those AVC and I can think of a regression in the selinux-policy. I'll double check again all our commits to git, but I am quite sure about it.

No idea. We did not add any changes which could cause this. Basically

(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #15)
> > > Well, these cluster stuff are really hard to test for us. We really need
> > > your help. We just run the basic tests.
> > 
> > I am very happy to help you, but i don´t understand selinux, so we need to
> > work together a bit here to understand each other :)
> > 
> > > 
> > > Why do we get it? This is not a question for us.
> > 
> > all I am asking is what changed in selinux (or policy.. or..) that might
> > have introduced those new AVC. I know more or less all changes that have
> > gone into cluster from 6.3, so pretty much I have an overview of what might
> > be causing stuff.
> > 
> > > Is cman_tool supposed to
> > > run random scripts?
> > 
> > No, cman_tool only run a very strict set of script. Those are
> > ccs_sync/ccs_config_validate.
> > 
> > ccs_config_validate calls in turns a bunch of stuff.
> 
> This is important.
> 
> > From grepping the code,
> > there is no call to setfiles (might be ccs_sync from ricci package).
> > 
> > Now, I suspect, but can´t be sure that the /usr/share/cluster/*.sh AVC you
> > saw before are due to ccs_config_validate updating its cache. It does so by
> > running those files with a metadata option and extract some information. Tho
> > it´s not real runtime execution as rgmanager does.
> > 
> > The difference is:
> > 
> > cman_tool -> ccs_config_validate -> ccs_update_schema -> (agents $metadata)>
> > cache.
> 
> Which means you will end up with corosync_t which is not obviously the right
> domain for this. And this is reason why I suggested rgmanager.
> 

I'm really unsure if this is true. cman_tool itself probably don't need to run under selinux context, because it's not daemon. cman_tool then executes corosync so maybe automatic translation may be way there.

> > 
> > rgmanger -> run agents as root to perform real stuff execution.
> >

You are right here. But there were some issues with cman init script which was running as initrc_t.

But now I am pretty sure the rgmanager_exec_t is a good solution. Basically I could switch cman_* labeling back to bin_t and add accesses for initrc_t but we don't want to do it. And rgmanager_t is "initrc_domain" and already can interact with RHCS domains.

I have already added

commit 8fd238855167d48ea21a9f5f8828a2f59a37f897
Author: Miroslav Grepl <mgrepl>
Date:   Thu Nov 8 19:14:18 2012 +0100

    ccs_tool and cman_tool should not be labeled as corosync_exec_t


to Fedora.

(In reply to comment #21)
> You are right here. But there were some issues with cman init script which
> was running as initrc_t.
> 
> But now I am pretty sure the rgmanager_exec_t is a good solution. Basically
> I could switch cman_* labeling back to bin_t and add accesses for initrc_t
> but we don't want to do it. And rgmanager_t is "initrc_domain" and already
> can interact with RHCS domains.
> 
> I have already added
> 
> commit 8fd238855167d48ea21a9f5f8828a2f59a37f897
> Author: Miroslav Grepl <mgrepl>
> Date:   Thu Nov 8 19:14:18 2012 +0100
> 
>     ccs_tool and cman_tool should not be labeled as corosync_exec_t
> 
> 
> to Fedora.

I really must ask. What Fedora are you talking about? FC17/FC18 doesn't have cman_tool. Actually in fc17+ cluster stack situation is much simpler, because cman and rgmanager are gone and there is really only corosync and pacemaker. This also means, that corosync is no longer executed by cman, but only by it's own init script (yeas, we have systemd unit file, but this just executes init script).

*** Bug 883006 has been marked as a duplicate of this bug. ***

*** This bug has been marked as a duplicate of bug 915151 ***