Bug 2056565

Summary: rabbitmq requires access to tmpfs_t
Product: Red Hat Enterprise Linux 9 Reporter: Takashi Kajinami <tkajinam>
Component: selinux-policyAssignee: Zdenek Pytela <zpytela>
Status: CLOSED ERRATA QA Contact: Milos Malik <mmalik>
Severity: medium Docs Contact:
Priority: medium    
Version: CentOS StreamCC: jpichon, lhh, lvrabec, mmalik, nknazeko, ssekidde, zpytela
Target Milestone: rcKeywords: Triaged
Target Release: 9.1   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: selinux-policy-34.1.38-1.el9 Doc Type: No Doc Update
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-11-15 11:13:14 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Takashi Kajinami 2022-02-21 13:45:33 UTC
Description of problem:

In CI of Puppet OpenStack project, we noticed that beam.smp requires access to tmpfs but
selinux is denying that atm.
The denial is specific to CentOS 9 Stream, possibly because of different rabbitmq version
we have for CentOS 8.

https://zuul.opendev.org/t/openstack/build/8ebb404eea36416aaae78c1c7026eb7b

https://1fce73b181d8a387f794-bf339ca8b211499ccf5ec38f530d7d51.ssl.cf2.rackcdn.com/829987/1/check/puppet-openstack-integration-7-scenario002-tempest-centos-9-stream/8ebb404/logs/audit.log.txt
~~~
type=AVC msg=audit(1645438671.805:3992): avc:  denied  { write } for  pid=30805 comm="beam.smp" name="memfd:vmem" dev="tmpfs" ino=1033 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438671.805:3993): avc:  denied  { read execute } for  pid=30805 comm="beam.smp" path=2F6D656D66643A766D656D202864656C6574656429 dev="tmpfs" ino=1033 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_
r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438732.033:4177): avc:  denied  { write } for  pid=31370 comm="beam.smp" name="memfd:vmem" dev="tmpfs" ino=6168 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438732.033:4178): avc:  denied  { read execute } for  pid=31370 comm="beam.smp" path=2F6D656D66643A766D656D202864656C6574656429 dev="tmpfs" ino=6168 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_
r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438792.289:4740): avc:  denied  { write } for  pid=32385 comm="beam.smp" name="memfd:vmem" dev="tmpfs" ino=1035 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438792.289:4741): avc:  denied  { read execute } for  pid=32385 comm="beam.smp" path=2F6D656D66643A766D656D202864656C6574656429 dev="tmpfs" ino=1035 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_
r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438852.533:4916): avc:  denied  { write } for  pid=64853 comm="beam.smp" name="memfd:vmem" dev="tmpfs" ino=20 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
type=AVC msg=audit(1645438852.533:4917): avc:  denied  { read execute } for  pid=64853 comm="beam.smp" path=2F6D656D66643A766D656D202864656C6574656429 dev="tmpfs" ino=20 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:
tmpfs_t:s0 tclass=file permissive=1
~~~

https://1fce73b181d8a387f794-bf339ca8b211499ccf5ec38f530d7d51.ssl.cf2.rackcdn.com/829987/1/check/puppet-openstack-integration-7-scenario002-tempest-centos-9-stream/8ebb404/logs/ps.txt
~~~
rabbitmq   27672       1   27672  1.6  1.4 557988 118688 /usr/lib64/erlang/erts-12.2.1/bin/beam.smp -W w -MBas ageffcbf -MHas ageffcbf -MBlmbcs 512 -MHlmbcs 512 -MMmcs 30 -P 1048576 -t 5000000 -stbt db -zdbbl 128000 -sbwt none -sbwtdcpu none -sbwtdio none -- -root /usr/lib64/erlang -progname erl -- -home /var/lib/rabbitmq -- -pa  -noshell -noinput -s rabbit boot -boot start_sasl -proto_dist inet6_tcp -syslog logger [] -syslog syslog_error_logger false
~~~


Version-Release number of selected component (if applicable):

https://1fce73b181d8a387f794-bf339ca8b211499ccf5ec38f530d7d51.ssl.cf2.rackcdn.com/829987/1/check/puppet-openstack-integration-7-scenario002-tempest-centos-9-stream/8ebb404/logs/rpm-qa.txt
openstack-selinux-0.8.31-0.20220216142943.80f4ed0.el9.noarch
rabbitmq-server-3.9.10-1.el9s.x86_64

How reproducible:
Always

Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Julie Pichon 2022-02-25 14:51:01 UTC
I'm not 100% sure whether that isn't an issue with labeling here as well. It seems like this is a problem that should also be reported against the main policy as I don't think it's specific to something OpenStack is doing or where it's installing things?


Just noting the SELinux policy version for reference: selinux-policy-34.1.25-1.el9.noarch

Comment 2 Julie Pichon 2022-02-25 15:08:39 UTC
I'm also finding references to this kind of issues happening when /tmp is mounted with tmpfs_t context instead of tmp_t (e.g. [1]). Not sure if this may be the difference between the two versions here, I'm not sure if the output of ls -lZ /tmp is available in the logs, or if that's where the file is being accessed.


[1] https://bugs.centos.org/view.php?id=10069

Comment 3 Takashi Kajinami 2022-03-04 07:51:07 UTC
I've submitted a test patch to check selinux labels of /tmp.
 https://review.opendev.org/c/openstack/puppet-openstack-integration/+/831900

I'll check the type assigned to that directory once ci run completes.

Comment 4 Julie Pichon 2022-03-11 14:31:31 UTC
Sorry for the delay. Have you been able to get the information? Based on your last comment on the review, I will move the bug to CentOS 9 if there is no objection as that seems like a more future-proof place to fix this if the path isn't specific to OpenStack. Thank you!

Comment 5 Takashi Kajinami 2022-06-09 01:17:45 UTC
Sorry this bug has dropped from my memory. I'll put needinfo on me and will update you.

Just fyi, I noticed this is causing rabbitmq to fail to start when selinux is enforced
so this might have higher severity than expected.

Comment 6 Takashi Kajinami 2022-06-09 02:36:33 UTC
I've captured the information in CentOS 9 job.

Auditd complains that rabbitmq tried to access a file with context tmpfs_t .
~~~
type=AVC msg=audit(1654738461.728:4975): avc:  denied  { write } for  pid=40600 comm="beam.smp" name="memfd:vmem" dev="tmpfs" ino=4106 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=1
~~~

However according to the output of `$sudo ls -laZ /tmp`, /tmp has tmp_t instead of tmpfs_t...
~~~
total 1520
drwxrwxrwt. 22 root  root  system_u:object_r:tmp_t:s0             4096 Jun  9 02:26 .
drwxr-xr-x. 21 root  root  system_u:object_r:root_t:s0            4096 Jun  9 01:32 ..
drwx------.  2 zuul  zuul  unconfined_u:object_r:user_tmp_t:s0    4096 Jun  9 01:23 ansible_command_payload_fw0z6y6k
...
~~~

Comment 7 Takashi Kajinami 2022-06-09 06:45:39 UTC
It seems rabbitmq fails to start because of this selinux denial when selinux is enforced.

~~~
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: Starting RabbitMQ broker...
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 rabbitmq-server[38490]: beam/jit/x86/beam_asm.cpp:167:pick_allocator(): Internal error: jit: Cannot allocate executable memory. Use the interpreter instead.
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: Created slice Slice /system/systemd-coredump.
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: Started Process Core Dump (PID 38498/UID 0).
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd-coredump[38499]: Resource limits disable core dumping for process 38490 (beam.smp).
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd-coredump[38499]: Process 38490 (beam.smp) of user 985 dumped core.
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: rabbitmq-server.service: Main process exited, code=dumped, status=6/ABRT
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: rabbitmq-server.service: Failed with result 'core-dump'.
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: Failed to start RabbitMQ broker.
Jun 08 18:11:27 centos-9-stream-ovh-gra1-0029941684 systemd[1]: systemd-coredump: Deactivated successfully.
~~~

~~~
type=AVC msg=audit(1654711887.924:3177): avc:  denied  { write } for  pid=38490 comm="beam.smp" name="memfd:vmem" dev="tmpfs" ino=10 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:tmpfs_t:s0 tclass=file permissive=0
type=SYSCALL msg=audit(1654711887.924:3177): arch=c000003e syscall=77 success=no exit=-13 a0=10 a1=1000 a2=7f3c4e1aec80 a3=55f11a394210 items=0 ppid=1 pid=38490 auid=4294967295 uid=985 gid=985 euid=985 suid=985 fsuid=985 egid=985 sgid=985 fsgid=985 tty=(none) ses=4294967295 comm="beam.smp" exe="/usr/lib64/erlang/erts-12.1.5/bin/beam.smp" subj=system_u:system_r:rabbitmq_t:s0 key=(null)ARCH=x86_64 SYSCALL=ftruncate AUID="unset" UID="rabbitmq" GID="rabbitmq" EUID="rabbitmq" SUID="rabbitmq" FSUID="rabbitmq" EGID="rabbitmq" SGID="rabbitmq" FSGID="rabbitmq"
type=PROCTITLE msg=audit(1654711887.924:3177): proctitle=2F7573722F6C696236342F65726C616E672F657274732D31322E312E352F62696E2F6265616D2E736D70002D570077002D4D426173006167656666636266002D4D486173006167656666636266002D4D426C6D62637300353132002D4D486C6D62637300353132002D4D4D6D6373003330002D500031303438353736002D74
type=AVC msg=audit(1654711887.925:3178): avc:  denied  { execmem } for  pid=38490 comm="beam.smp" scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:system_r:rabbitmq_t:s0 tclass=process permissive=0
type=SYSCALL msg=audit(1654711887.925:3178): arch=c000003e syscall=9 success=no exit=-13 a0=0 a1=20000 a2=7 a3=22 items=0 ppid=1 pid=38490 auid=4294967295 uid=985 gid=985 euid=985 suid=985 fsuid=985 egid=985 sgid=985 fsgid=985 tty=(none) ses=4294967295 comm="beam.smp" exe="/usr/lib64/erlang/erts-12.1.5/bin/beam.smp" subj=system_u:system_r:rabbitmq_t:s0 key=(null)ARCH=x86_64 SYSCALL=mmap AUID="unset" UID="rabbitmq" GID="rabbitmq" EUID="rabbitmq" SUID="rabbitmq" FSUID="rabbitmq" EGID="rabbitmq" SGID="rabbitmq" FSGID="rabbitmq"
~~~

Comment 9 Takashi Kajinami 2022-06-09 06:49:59 UTC
I've moved this to selinux-policy because the policy rules for rabbitmq is implemented in that package instead of openstack-selinux.

Comment 12 Takashi Kajinami 2022-06-09 07:02:36 UTC
I've submitted a PR to address denials for write/read/execute which I initially reported.
Interestingly we also find denials about execmem but I'd leave it now and address it separately if needed ...

Comment 14 Zdenek Pytela 2022-06-09 09:08:08 UTC
To backport:
commit 944c765970794e8aae72f15ea3f630aa09234d14 (HEAD -> rawhide, upstream/rawhide)
Author: Takashi Kajinami <tkajinam>
Date:   Thu Jun 9 15:54:38 2022 +0900

    Allow rabbitmq to access its private memfd: objects

Comment 19 Takashi Kajinami 2022-06-13 07:23:48 UTC
OK I also confirmed the denials mentioned in commet:18 is reproduced in our CI jobs.

type=AVC msg=audit(1654884684.383:4434): avc:  denied  { getattr } for  pid=40772 comm="10_dirty_io_sch" path="/run/systemd/notify" dev="tmpfs" ino=36 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:init_var_run_t:s0 tclass=sock_file permissive=1
type=AVC msg=audit(1654884684.383:4435): avc:  denied  { read } for  pid=40772 comm="10_dirty_io_sch" name="notify" dev="tmpfs" ino=36 scontext=system_u:system_r:rabbitmq_t:s0 tcontext=system_u:object_r:init_var_run_t:s0 tclass=sock_file permissive=1


Will submit a fix for that as well.

Comment 20 Takashi Kajinami 2022-06-13 07:34:50 UTC
(In reply to Takashi Kajinami from comment #19)
> OK I also confirmed the denials mentioned in commet:18 is reproduced in our
> CI jobs.
> 
> type=AVC msg=audit(1654884684.383:4434): avc:  denied  { getattr } for 
> pid=40772 comm="10_dirty_io_sch" path="/run/systemd/notify" dev="tmpfs"
> ino=36 scontext=system_u:system_r:rabbitmq_t:s0
> tcontext=system_u:object_r:init_var_run_t:s0 tclass=sock_file permissive=1
> type=AVC msg=audit(1654884684.383:4435): avc:  denied  { read } for 
> pid=40772 comm="10_dirty_io_sch" name="notify" dev="tmpfs" ino=36
> scontext=system_u:system_r:rabbitmq_t:s0
> tcontext=system_u:object_r:init_var_run_t:s0 tclass=sock_file permissive=1
> 
> 
> Will submit a fix for that as well.

I've submitted a fix for this.
 https://github.com/fedora-selinux/selinux-policy/pull/1231

Please let me know in case I should create a separate bug.

Comment 21 Takashi Kajinami 2022-06-22 01:01:19 UTC
Hi Zdenek,

Could you please check comment:20 and the additional patch I proposed when you have time ?

Comment 26 Takashi Kajinami 2022-07-09 16:47:45 UTC
So we need to allow rabbitmq_t to read init_var_run_t:sock_file.

In the second patch, we used init_stream_connect, but this internally uses stream_connect_pattern, which then uses write_sock_file_perms, which does not allow read.

IIUC there are a few options we have now.
 1) Update that stream_connect_pattern (and dgram_send_pattern ?) to allow read 
 2) Update init_stream_connect to allow read
 3) Introduce a new macro to additionally allow read

I feel like the 1 would be preferable but am concerned that it can affect multiple rules depending on it.


@Zdenek May I ask for your suggestion ?

Comment 28 Zdenek Pytela 2022-07-14 08:37:41 UTC
I've just written a more generic solution:
https://github.com/fedora-selinux/selinux-policy/pull/1283

Comment 38 errata-xmlrpc 2022-11-15 11:13:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (selinux-policy bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:8283