Bug 2011361
| Summary: | [RFE] Avoid having packages and selinux-policy packages from being installed in the same yum transaction to prevent broken labels | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Kenny Tordeurs <ktordeur> | ||||
| Component: | Infrastructure | Assignee: | satellite6-bugs <satellite6-bugs> | ||||
| Status: | CLOSED WONTFIX | QA Contact: | Lukas Pramuk <lpramuk> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 6.9.0 | CC: | egolov, ehelms, ekohlvan, inecas, jkrajice, ttereshc | ||||
| Target Milestone: | Unspecified | Keywords: | FutureFeature, Reopened, RFE | ||||
| Target Release: | Unused | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2021-10-28 22:55:32 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
This issue can be fixed by doing a relabel `# echo "-vv" > /.autorelabel` # ls -lZ /var/opt/rh/rh-mongodb34/run/mongodb/ ~~~ -rw-r--r--. mongodb mongodb system_u:object_r:var_t:s0 mongod.pid ~~~ ls -lZ /var/opt/rh/rh-mongodb34/run ~~~ drwxr-xr-x. mongodb root system_u:object_r:var_t:s0 mongodb ~~~ However this should not have to be required when nothing has changed on the permissions side to cause this. This Capsule was freshly installed. Example from a Capsule getting the same issue. ~~~ systemctl status rh-mongodb34-mongod ● rh-mongodb34-mongod.service - High-performance, schema-free document-oriented database Loaded: loaded (/usr/lib/systemd/system/rh-mongodb34-mongod.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Thu 2021-10-07 09:56:23 CEST; 24min ago Process: 1090 ExecStart=/opt/rh/rh-mongodb34/root/usr/libexec/mongodb-scl-helper enable $RH_MONGODB34_SCLS_ENABLED -- /opt/rh/rh-mongodb34/root/usr/bin/mongod $OPTIONS run (code=exited, status=1/FAILURE) Oct 07 09:56:22 capsule.exercise.lan systemd[1]: Starting High-performance, schema-free document-oriented database... Oct 07 09:56:23 capsule.exercise.lan mongodb-scl-helper[1090]: about to fork child process, waiting until server is ready for connections. Oct 07 09:56:23 capsule.exercise.lan mongodb-scl-helper[1090]: forked process: 1213 Oct 07 09:56:23 capsule.exercise.lan mongod.27017[1213]: [main] ERROR: Cannot write pid file to /var/opt/rh/rh-mongodb34/run/mongodb/mongod.pid: Permission denied Oct 07 09:56:23 capsule.exercise.lan mongodb-scl-helper[1090]: ERROR: child process failed, exited with error number 1 Oct 07 09:56:23 capsule.exercise.lan systemd[1]: rh-mongodb34-mongod.service: control process exited, code=exited status=1 Oct 07 09:56:23 capsule.exercise.lan systemd[1]: Failed to start High-performance, schema-free document-oriented database. Oct 07 09:56:23 capsule.exercise.lan systemd[1]: Unit rh-mongodb34-mongod.service entered failed state. Oct 07 09:56:23 capsule.exercise.lan systemd[1]: rh-mongodb34-mongod.service failed. ~~~ # ls -lZ /var/opt/rh/rh-mongodb34/run ~~~ drwxr-xr-x. mongodb root system_u:object_r:var_t:s0 mongodb ~~~ # getfacl /var/opt/rh/rh-mongodb34/run/mongodb/ ~~~ getfacl: Removing leading '/' from absolute path names # file: var/opt/rh/rh-mongodb34/run/mongodb/ # owner: mongodb # group: root user::rwx group::r-x other::r-x ~~~ [root@capsule ~]# rpm -qa | grep capsule ~~~ satellite-capsule-6.9.6.1-1.el7sat.noarch ~~~ (In reply to Kenny Tordeurs from comment #1) > This issue can be fixed by doing a relabel `# echo "-vv" > /.autorelabel` > > # ls -lZ /var/opt/rh/rh-mongodb34/run/mongodb/ > ~~~ > -rw-r--r--. mongodb mongodb system_u:object_r:var_t:s0 mongod.pid > ~~~ > > ls -lZ /var/opt/rh/rh-mongodb34/run > ~~~ > drwxr-xr-x. mongodb root system_u:object_r:var_t:s0 mongodb > ~~~ > > However this should not have to be required when nothing has changed on the > permissions side to cause this. If a relabel helps, then I wonder what's wrong. The output of restorecon -Rv /var/opt/rh/rh-mongodb34 can help since that should show you what it changes. Created attachment 1830325 [details]
restorecon -Rvn output
The problem is that the MongoDB packages were installed in the same yum transaction which also contained selinux-policy updates, and this results in broken labels. This is why our docs (https://access.redhat.com/documentation/en-us/red_hat_satellite/6.9/html/installing_capsule_server/installing-capsule-server#installing-capsule-server-packages_capsule) tell to run "yum update" before installing the packages I'm confirming that the issue is gone by ensuring a `yum update` takes place before the actual installation of the Capsule/Satellite. However I wanted to bring the point forward if someone recently installed/deployed a fresh system they might easily skip the `yum update` step assuming it's not required as the box was just installed. Could we not get foreman-maintain involved for Satellite/Capsule to also perform the complete installation and avoid this type of issue as it would have the correct order of the steps as it's an easy step to overlook, maybe this could be an RFE? old wontfix BZ of the same issue: https://bugzilla.redhat.com/show_bug.cgi?id=1873319 yes this can be an RFE, I guess After some discussion, we are choosing to close as Satellite 6.10 will drop mongodb and the RHSCL product owns the mongodb packages so solving the specific issue with mongodb would need to happen there. If this issue shows up more broadly, in more of the Satellite selinux packages please re-open and we will investigate. |
Description of problem: Some SElinux permission issue on mongod.pid causing mongo to fail to start. ERROR: Cannot write pid file to /var/opt/rh/rh-mongodb34/run/mongodb/mongod.pid: Permission denied Version-Release number of selected component (if applicable): 6.9.4 How reproducible: Uncertain, manually reproducible by changing the SELinux context. Steps to Reproduce: 1. Happens because of SElinux issues 2. 3. Actual results: ERROR: Cannot write pid file to /var/opt/rh/rh-mongodb34/run/mongodb/mongod.pid: Permission denied Expected results: No issues Additional info: [root@ktordeur-satellite-latest ~]# systemctl status rh-mongodb34-mongod ● rh-mongodb34-mongod.service - High-performance, schema-free document-oriented database Loaded: loaded (/usr/lib/systemd/system/rh-mongodb34-mongod.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2021-10-06 14:46:41 CEST; 27min ago Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan systemd[1]: Starting High-performance, schema-free document-oriented database... Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan mongodb-scl-helper[9440]: about to fork child process, waiting until server is ready for connections. Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan mongodb-scl-helper[9440]: forked process: 9445 Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[9445]: [main] ERROR: Cannot write pid file to /var/opt/rh/rh-mongodb34/run/mongodb/mongod.pid: Permission denied Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan mongodb-scl-helper[9440]: ERROR: child process failed, exited with error number 1 Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan systemd[1]: rh-mongodb34-mongod.service: control process exited, code=exited status=1 Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan systemd[1]: Failed to start High-performance, schema-free document-oriented database. Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan systemd[1]: Unit rh-mongodb34-mongod.service entered failed state. Oct 06 14:46:41 ktordeur-satellite-latest.sysmgmt.lan systemd[1]: rh-mongodb34-mongod.service failed. # ls -lZ /var/opt/rh/rh-mongodb34/run drwxr-xr-x. mongodb root system_u:object_r:var_t:s0 mongodb # getfacl /var/opt/rh/rh-mongodb34/run/mongodb/ ~~~ getfacl: Removing leading '/' from absolute path names # file: var/opt/rh/rh-mongodb34/run/mongodb/ # owner: mongodb # group: root user::rwx group::r-x other::r-x ~~~ # sudo -u mongodb touch /var/opt/rh/rh-mongodb34/run/mongodb/mongod.pid [root@ktordeur-satellite-latest ~]# ls -lZ /var/opt/rh/rh-mongodb34/run/mongodb/ ~~~ -rw-r--r--. mongodb mongodb unconfined_u:object_r:var_t:s0 mongod.pid ~~~ ~~~ type=AVC msg=audit(1633526478.479:49897): avc: denied { write } for pid=12152 comm="mongod" name="mongodb" dev="dm-0" ino=68799459 scontext=system_u:system_r:mongod_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=1 type=AVC msg=audit(1633526478.479:49897): avc: denied { add_name } for pid=12152 comm="mongod" name="mongod.pid" scontext=system_u:system_r:mongod_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=dir permissive=1 type=AVC msg=audit(1633526478.479:49897): avc: denied { create } for pid=12152 comm="mongod" name="mongod.pid" scontext=system_u:system_r:mongod_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 type=AVC msg=audit(1633526478.479:49897): avc: denied { write open } for pid=12152 comm="mongod" path="/var/opt/rh/rh-mongodb34/run/mongodb/mongod.pid" dev="dm-0" ino=68796704 scontext=system_u:system_r:mongod_t:s0 tcontext=system_u:object_r:var_t:s0 tclass=file permissive=1 ~~~ # systemctl start rh-mongodb34-mongod.service Job for rh-mongodb34-mongod.service failed because the control process exited with error code. See "systemctl status rh-mongodb34-mongod.service" and "journalctl -xe" for details. [root@ktordeur-satellite-latest ~]# setenforce 0 [root@ktordeur-satellite-latest ~]# systemctl start rh-mongodb34-mongod.service [root@ktordeur-satellite-latest ~]# systemctl status rh-mongodb34-mongod.service ● rh-mongodb34-mongod.service - High-performance, schema-free document-oriented database Loaded: loaded (/usr/lib/systemd/system/rh-mongodb34-mongod.service; enabled; vendor preset: disabled) Active: active (running) since Wed 2021-10-06 15:25:59 CEST; 11s ago Process: 12241 ExecStart=/opt/rh/rh-mongodb34/root/usr/libexec/mongodb-scl-helper enable $RH_MONGODB34_SCLS_ENABLED -- /opt/rh/rh-mongodb34/root/usr/bin/mongod $OPTIONS run (code=exited, status=0/SUCCESS) Main PID: 12246 (mongod) Tasks: 20 CGroup: /system.slice/rh-mongodb34-mongod.service └─12246 /opt/rh/rh-mongodb34/root/usr/bin/mongod -f /etc/opt/rh/rh-mongodb34/mongod.conf run Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] ** We suggest setting it to 'never' Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'. Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] ** We suggest setting it to 'never' Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongodb/diagnostic.data' Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongod.27017[12246]: [thread1] waiting for connections on port 27017 Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan mongodb-scl-helper[12241]: child process started successfully, parent exiting Oct 06 15:25:59 ktordeur-satellite-latest.sysmgmt.lan systemd[1]: Started High-performance, schema-free document-oriented database.