Bug 2223069

Summary: Selinux denials are reported after following "Chapter 13. Managing Custom File Type Content" chapter step by step
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED MIGRATED QA Contact: Shweta Singh <shwsingh>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.13.1CC: ahumbe, dalley, egolov, osousa, rlavi, shwsingh, zhunting
Target Milestone: UnspecifiedKeywords: MigratedToJIRA, Regression, Triaged
Target Release: UnusedFlags: shwsingh: needinfo? (dalley)
shwsingh: needinfo? (dalley)
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: pulpcore-selinux-2.0.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2250343 (view as bug list) Environment:
Last Closed: 2024-06-06 16:24:35 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 2263815    
Bug Blocks:    

Description Sayan Das 2023-07-15 06:45:17 UTC
Description of problem:

If an user of a new Satellite 6.11\6.12\6.13 follows the "Creating a Local Source for a Custom File Type Repository" section from Content Management guide, despite everything is working, auditd will log *cosmetic* selinux denials on read\open\ioctl actions on the target files.

If someone has upgraded from Satellite 6.9 to 6.11\6.12\6.13, then the same denials would not be reproducible. 


Version-Release number of selected component (if applicable):

Satellite 6.11 ( RHEL 7 and RHEL 8 )
Satellite 6.12
Satellite 6.13


How reproducible:

100%

Steps to Reproduce and Actual Results:

# sestatus
SELinux status:                 enabled
SELinuxfs mount:                /sys/fs/selinux
SELinux root directory:         /etc/selinux
Loaded policy name:             targeted
Current mode:                   enforcing
Mode from config file:          enforcing
Policy MLS status:              enabled
Policy deny_unknown status:     allowed
Memory protection checking:     actual (secure)
Max kernel policy version:      33


# semanage fcontext -l | grep pulp
/etc/pulp/certs(/.*)?                              all files          system_u:object_r:httpd_config_t:s0 
/etc/pulp/certs/database_fields.symmetric.key      all files          system_u:object_r:pulpcore_etc_t:s0 
/etc/pulp/certs/galaxy_signing_service.*           all files          system_u:object_r:pulpcore_etc_t:s0 
/etc/pulp/certs/token_private_key.pem              all files          system_u:object_r:pulpcore_etc_t:s0 
/etc/pulp/certs/token_public_key.pem               all files          system_u:object_r:pulpcore_etc_t:s0 
/etc/pulp/settings.py                              all files          system_u:object_r:pulpcore_etc_t:s0 
/usr/libexec/pulpcore/.*                           regular file       system_u:object_r:pulpcore_exec_t:s0 
/usr/libexec/pulpcore/gunicorn                     regular file       system_u:object_r:pulpcore_server_exec_t:s0 
/usr/local/lib/pulp/bin/gunicorn                   regular file       system_u:object_r:pulpcore_server_exec_t:s0 
/usr/local/lib/pulp/bin/pulpcore-worker            regular file       system_u:object_r:pulpcore_exec_t:s0 
/usr/local/lib/pulp/bin/rq                         regular file       system_u:object_r:pulpcore_exec_t:s0 
/var/lib/pulp/(media|artifact)(/.*)?               all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/.ansible(/.*)?                       all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/.cache(/.*)?                         all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/assets(/.*)?                         all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/devel(/.*)?                          all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/pulpcore_static(/.*)?                all files          system_u:object_r:httpd_sys_content_t:s0 
/var/lib/pulp/sign-metadata.sh                     regular file       system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/tmp(/.*)?                            all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/pulp/upload(/.*)?                         all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/lib/soe/software(/.*)?                        all files          system_u:object_r:pulpcore_var_lib_t:s0 
/var/log/galaxy_api_access.log                     all files          system_u:object_r:pulpcore_log_t:s0 
/var/run/pulpcore-(api|content)\.sock              all files          system_u:object_r:pulpcore_server_var_run_t:s0 
/var/run/pulpcore-api(/.*)?                        all files          system_u:object_r:pulpcore_server_var_run_t:s0 
/var/run/pulpcore-content(/.*)?                    all files          system_u:object_r:pulpcore_server_var_run_t:s0 
/var/run/pulpcore.*                                all files          system_u:object_r:pulpcore_var_run_t:s0 


# rpm -q python39-pulp_manifest
python39-pulp_manifest-3.0.0-3.el8pc.noarch


# mkdir -p /var/lib/pulp/local_repos/my_file_repo

# ls -ld /var/lib/pulp/local_repos/my_file_repo -Z
drwxr-xr-x. 2 root root unconfined_u:object_r:var_lib_t:s0 6 Jul 14 07:44 /var/lib/pulp/local_repos/my_file_repo


# satellite-installer --foreman-proxy-content-pulpcore-additional-import-paths /var/lib/pulp/local_repos --foreman-proxy-content-pulpcore-additional-import-paths /var/lib/soe/software
2023-07-14 07:52:22 [NOTICE] [root] Loading installer configuration. This will take some time.
...
...

  The full log is at /var/log/foreman-installer/satellite.log
Package versions are being locked.

# cat /etc/pulp/settings.py | grep IMPORT
ALLOWED_IMPORT_PATHS = ["/var/lib/pulp/sync_imports", "/var/lib/pulp/imports", "/var/lib/pulp/local_repos", "/var/lib/soe/software"]


# ls -ld /var/lib/pulp/local_repos/my_file_repo -Z
drwxr-xr-x. 2 root root unconfined_u:object_r:var_lib_t:s0 6 Jul 14 07:44 /var/lib/pulp/local_repos/my_file_repo


# restorecon -RFv /var/lib/pulp/local_repos/my_file_repo
Relabeled /var/lib/pulp/local_repos/my_file_repo from unconfined_u:object_r:var_lib_t:s0 to system_u:object_r:var_lib_t:s0

# ls -ld /var/lib/pulp/local_repos/my_file_repo -Z
drwxr-xr-x. 2 root root system_u:object_r:var_lib_t:s0 6 Jul 14 07:44 /var/lib/pulp/local_repos/my_file_repo

# ls -ld /var/lib/pulp/local_repos -Z
drwxrwx---. 3 pulp pulp system_u:object_r:var_lib_t:s0 26 Jul 14 07:44 /var/lib/pulp/local_repos


# touch /var/lib/pulp/local_repos/my_file_repo/test.txt
# pulp-manifest /var/lib/pulp/local_repos/my_file_repo

# ls /var/lib/pulp/local_repos/my_file_repo
PULP_MANIFEST  test.txt

# ls -ldZ /var/lib/pulp/local_repos /var/lib/pulp/local_repos/my_file_repo /var/lib/pulp/local_repos/my_file_repo/*
drwxrwx---. 3 pulp pulp system_u:object_r:var_lib_t:s0     26 Jul 14 07:44 /var/lib/pulp/local_repos
drwxr-xr-x. 2 root root system_u:object_r:var_lib_t:s0     43 Jul 14 07:50 /var/lib/pulp/local_repos/my_file_repo
-rw-r--r--. 1 root root unconfined_u:object_r:var_lib_t:s0 76 Jul 14 07:50 /var/lib/pulp/local_repos/my_file_repo/PULP_MANIFEST
-rw-r--r--. 1 root root unconfined_u:object_r:var_lib_t:s0  0 Jul 14 07:50 /var/lib/pulp/local_repos/my_file_repo/test.txt

# restorecon -RFv /var/lib/pulp/local_repos
Relabeled /var/lib/pulp/local_repos/my_file_repo/test.txt from unconfined_u:object_r:var_lib_t:s0 to system_u:object_r:var_lib_t:s0
Relabeled /var/lib/pulp/local_repos/my_file_repo/PULP_MANIFEST from unconfined_u:object_r:var_lib_t:s0 to system_u:object_r:var_lib_t:s0


# hammer repository info --name Myfiles --product File --organization RedHat | grep -i -B3 URL
Red Hat Repository: no
Content Type:       file
Mirroring Policy:   Content Only
Url:                file:///var/lib/pulp/local_repos/my_file_repo

--> After syncing from UI:

# hammer repository info --name Myfiles --product File --organization RedHat  | tail -10
GPG Key:            

Sync:               
    Status:         Success
    Last Sync Date: 1 minute
Created:            2023/07/14 11:58:55
Updated:            2023/07/14 11:58:57
Content Counts:     
    Files: 1


So, My selinux was always in enforcing mode and even if my sync was successful, I can see these denials 

time->Fri Jul 14 08:00:18 2023
type=PROCTITLE msg=audit(1689336018.528:4016): proctitle=2F7573722F62696E2F707974686F6E332E39002F7573722F62696E2F70756C70636F72652D776F726B6572
type=SYSCALL msg=audit(1689336018.528:4016): arch=c000003e syscall=16 success=no exit=-25 a0=e a1=5401 a2=7f97c4ba9bf0 a3=1c3279463920e1 items=0 ppid=44797 pid=45287 auid=4294967295 uid=993 gid=991 euid=993 suid=993 fsuid=993 egid=991 sgid=991 fsgid=991 tty=(none) ses=4294967295 comm="pulpcore-worker" exe="/usr/bin/python3.9" subj=system_u:system_r:pulpcore_t:s0 key=(null)
type=AVC msg=audit(1689336018.528:4016): avc:  denied  { ioctl } for  pid=45287 comm="pulpcore-worker" path="/var/lib/pulp/local_repos/my_file_repo/PULP_MANIFEST" dev="dm-0" ino=46314752 ioctlcmd=0x5401 scontext=system_u:system_r:pulpcore_t:s0 tcontext=system_u:object_r:var_lib_t:s0 tclass=file permissive=1

Now, NOTE that, 

* auditd reports the denial as "permissive=1" even though selinux is in enforcing mode.

* The denial happens as pulpcore_t trying to ioctl access on var_lib_t and that is not allowed. 

# sesearch -A -s pulpcore_t -p ioctl | grep pulpcore | grep "var_lib"
allow pulpcore_t pulpcore_server_var_lib_t:dir { add_name create getattr ioctl link lock open read remove_name rename reparent rmdir search setattr unlink write };
allow pulpcore_t pulpcore_server_var_lib_t:file { append create getattr ioctl link lock open read rename setattr unlink write };
allow pulpcore_t pulpcore_var_lib_t:dir { add_name create getattr ioctl link lock open read remove_name rename reparent rmdir search setattr unlink write };
allow pulpcore_t pulpcore_var_lib_t:file { append create execute execute_no_trans getattr ioctl link lock map open read rename setattr unlink write };
allow pulpcore_t pulpcore_var_lib_t:lnk_file { append create getattr ioctl link lock read rename setattr unlink write };


I would expect it to be var_lib_t only based on this default definition:

/var/lib(/.*)?                                     all files          system_u:object_r:var_lib_t:s0 


Now, To stop the denials, I would have to set up an additional selinux context i.e. 

# semanage fcontext -a -t pulpcore_var_lib_t "/var/lib/pulp/local_repos(/.*)?"

# restorecon -RFv /var/lib/pulp/local_repos
Relabeled /var/lib/pulp/local_repos from system_u:object_r:var_lib_t:s0 to system_u:object_r:pulpcore_var_lib_t:s0
Relabeled /var/lib/pulp/local_repos/my_file_repo from system_u:object_r:var_lib_t:s0 to system_u:object_r:pulpcore_var_lib_t:s0
Relabeled /var/lib/pulp/local_repos/my_file_repo/test.txt from system_u:object_r:var_lib_t:s0 to system_u:object_r:pulpcore_var_lib_t:s0
Relabeled /var/lib/pulp/local_repos/my_file_repo/PULP_MANIFEST from system_u:object_r:var_lib_t:s0 to system_u:object_r:pulpcore_var_lib_t:s0

# ls -ldZ /var/lib/pulp/local_repos /var/lib/pulp/local_repos/my_file_repo /var/lib/pulp/local_repos/my_file_repo/*
drwxrwx---. 3 pulp pulp system_u:object_r:pulpcore_var_lib_t:s0 26 Jul 14 07:44 /var/lib/pulp/local_repos
drwxr-xr-x. 2 root root system_u:object_r:pulpcore_var_lib_t:s0 43 Jul 14 07:50 /var/lib/pulp/local_repos/my_file_repo
-rw-r--r--. 1 root root system_u:object_r:pulpcore_var_lib_t:s0 76 Jul 14 07:50 /var/lib/pulp/local_repos/my_file_repo/PULP_MANIFEST
-rw-r--r--. 1 root root system_u:object_r:pulpcore_var_lib_t:s0  0 Jul 14 07:50 /var/lib/pulp/local_repos/my_file_repo/test.txt


And then no denials would be seen ( whether cosmetic or not ).


For users upgrading from Satellite 6.9, They would have an additional rule in place which a newly install Sat 6.10\11\12\13 would never have i.e. 

/var/lib/pulp(/.*)?                                all files          system_u:object_r:httpd_sys_rw_content_t:s0 

Due to this, Any files created in /var/lib/pulp/local_repos would have httpd_sys_rw_content_t label and since pulpcore_t is allowed to access httpd_sys_rw_content_t, no denials would be logged. 

# sesearch -A -s pulpcore_t -p ioctl | grep http
allow pulpcore_t httpd_sys_rw_content_t:dir { add_name create getattr ioctl link lock open read remove_name rename reparent rmdir search setattr unlink write };
allow pulpcore_t httpd_sys_rw_content_t:file { append create getattr ioctl link lock open read rename setattr unlink write };
allow pulpcore_t httpd_sys_rw_content_t:lnk_file { append create getattr ioctl link lock read rename setattr unlink write };



Expected results:

There is no global context set for "/var/lib/pulp(/.*)?" itself. 

If we expect any other custom-hosted content inside /var/lib/pulp should have same context as "/var/lib/pulp/(media|artifact)(/.*)? " i.e.

/var/lib/pulp/(media|artifact)(/.*)?               all files          system_u:object_r:pulpcore_var_lib_t:s0

Then add a rule for the same. 

Or else ensure that following is created on any new installations of Satellite 6.11\12\13 as well i.e. 

/var/lib/pulp(/.*)?                                all files          system_u:object_r:httpd_sys_rw_content_t:s0 


Additional info:


I also tried the chapter "Creating a Remote File Type Repository" where we are instructed to expose the context over HTTP by placing the files inside "/var/www/html/pub/".

For any files created inside /var/www/html/pub/, the context would be "httpd_sys_content_t"

But as we saw above pulpcore_t cannot access httpd_sys_content_t but it can httpd_sys_rw_content_t

So i assumed when i will sync the repo, It will give me similar denials but It does not. 

Perhaps that is because we are accessing the file over HTTP and hence the first process that accesses the file would be the webserver i.e. foreman_rails_t and if that is true then it is allowed to access\read\view\ioctl on both httpd_sys_content_t and httpd_sys_content_t 

Anyways, this is just a speculation but perhaps the reason behind *no denials* could be something different.

Comment 3 Daniel Alley 2023-08-22 01:22:59 UTC
This ought to be addressed now via https://github.com/pulp/pulpcore-selinux/pull/69

However this line from the description confuses me: "For users upgrading from Satellite 6.9, They would have an additional rule in place which a newly install Sat 6.10\11\12\13 would never have"

If we're shipping the selinux policy in a package alongside Satellite, how would users that upgraded from 6.9 have a rule in place that new installations would not?  Wouldn't the selinux policy have been swapped wholesale?  Where do these remanents come from?

Comment 4 Sayan Das 2023-08-22 08:47:56 UTC
Yeah, I would expect it to happen in that way only i.e. existing policies should be overwritten with newer ones. 

But It seems the following remains intact from 6.9, even if that satellite\capsule has been upgraded to 6.11\6.12\6.13 

/var/lib/pulp(/.*)?                                all files          system_u:object_r:httpd_sys_rw_content_t:s0 


Maybe the postinstall scriptlet of pulpcore-selinux requires some modifications here ?

Comment 5 Daniel Alley 2023-08-22 17:23:14 UTC
Dennis makes the point that 

"pulpcore-selinux policy is distinct from the pulp-selinux policy that shipped with pulp 2 so it's possible that the old selinux policy was still installed"

It doesn't look like the specfile declares any obsoletes on the old package, so that sounds plausible.

Comment 6 Sayan Das 2023-08-22 17:37:39 UTC
Very much possible as Sat 6.9 is shipped with both pulp-selinux (Pulp2) and pulpcore-selinux (Pulp3) but that means, the pre/postuninstall scriptlet of pulp-selinux never removed the existing policies. 


# rpm -q --scripts pulp-selinux
preinstall scriptlet (using /bin/sh):
# Record old version so we can limit which restorecon statement are executed later
test -e /var/lib/rpm-state/pulp || mkdir -p /var/lib/rpm-state/pulp
oldversion=$(rpm -qa pulp-selinux)
echo ${oldversion:13} > /var/lib/rpm-state/pulp/old-version

exit 0
postinstall scriptlet (using /bin/sh):
# Enable SELinux policy modules
if /usr/sbin/selinuxenabled ; then
 /usr/share/pulp/selinux/server/enable.sh /usr/share
fi

# restorcecon wasn't reading new file contexts we added when running under 'post' so moved to 'posttrans'
# Spacewalk saw same issue and filed BZ here: https://bugzilla.redhat.com/show_bug.cgi?id=505066
preuninstall scriptlet (using /bin/sh):
# Clean up after package removal
if [ $1 -eq 0 ]; then
/usr/share/pulp/selinux/server/uninstall.sh
/usr/share/pulp/selinux/server/relabel.sh
rm -r /var/lib/rpm-state/pulp
fi
exit 0
posttrans scriptlet (using /bin/sh):
if /usr/sbin/selinuxenabled ; then
 cat /var/lib/rpm-state/pulp/old-version | xargs /usr/share/pulp/selinux/server/relabel.sh
 rm /var/lib/rpm-state/pulp/old-version
fi





# cat /usr/share/pulp/selinux/server/uninstall.sh
#!/bin/sh

PACKAGE_NAMES=( "pulp-celery" "pulp-server" )
SELINUX_VARIANTS="targeted"
MODULE_TYPE="apps"
INSTALL_DIR="/usr/share"

for NAME in ${PACKAGE_NAMES[@]}
do
    for selinuxvariant in ${SELINUX_VARIANTS}
    do
        /usr/sbin/semodule -s ${selinuxvariant} -r ${NAME} &> /dev/null || :
        rm -f ${INSTALL_DIR}/${selinuxvariant}/${NAME}.pp
    done
done

Comment 9 Shweta Singh 2024-02-08 11:25:20 UTC
FailedQA

Version Tested: Satellite 6.15.0 Snap 8.0

"python311-pulp_manifest" package is missing in 6.15.0 and this is a blocker for verifying this Bug https://bugzilla.redhat.com/show_bug.cgi?id=2223069.

Comment 10 Daniel Alley 2024-02-08 14:08:21 UTC
Instead of marking this failed, shouldn't we create a brand new blocking packaging BZ?  The packaging team isn't going to know to look at this one.

Comment 12 Shweta Singh 2024-03-11 15:29:59 UTC
FailedQA

Version Tested: Satellite 6.15.0 Snap 10.1

Verification Steps:
1. Follow steps mentioned in https://bugzilla.redhat.com/show_bug.cgi?id=2223069#c0 

Observation:
1. auditd reports the denial as "permissive=1" even though selinux is in enforcing mode.
2. The denial happens as pulpcore_t trying to ioctl access on var_lib_t and that is not allowed.

Comment 13 Daniel Alley 2024-03-13 03:42:28 UTC
I'm looking into this.  In the interim, I would NOT recommend holding up the release, as this is only a cosmetic issue as-described.

Comment 17 Shweta Singh 2024-03-14 06:54:15 UTC
This bug is a regression on 6.15.0 as we don't see the denials on 6.14.3.

Comment 18 Daniel Alley 2024-03-14 13:45:25 UTC
@Shweta this is a new test run on 6.14.3, or are you referring to the one done to verify the clone?

Comment 19 Daniel Alley 2024-04-11 04:36:11 UTC
^

Comment 20 Shweta Singh 2024-04-11 06:37:21 UTC
@dalley This is (failed to)verified on 6.15.0. We have a clone of this BZ on 6.14.3 which is working as expected. So I am assuming that this is regression on 6.15.0.

Comment 21 Shweta Singh 2024-04-11 06:37:41 UTC
@dalley This is (failed to)verified on 6.15.0. We have a clone of this BZ on 6.14.3 which is working as expected. So I am assuming that this is regression on 6.15.0.

Comment 22 Eric Helms 2024-06-06 16:24:35 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "SAT-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.