Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2181008

Summary: SELinux prevents rootless container from starting up
Product: Red Hat Enterprise Linux 9 Reporter: Laszlo Ersek <lersek>
Component: container-selinuxAssignee: Jindrich Novy <jnovy>
Status: CLOSED MIGRATED QA Contact: atomic-bugs <atomic-bugs>
Severity: high Docs Contact:
Priority: unspecified    
Version: 9.1CC: bbaude, dwalsh, jnovy, lsm5, mboddu, mheon, pthomas, tsweeney, umohnani, zpytela
Target Milestone: rcKeywords: MigratedToJIRA
Target Release: ---Flags: pm-rhel: mirror+
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-09-11 19:36:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Laszlo Ersek 2023-03-22 20:24:25 UTC
*** Description of problem:

(I don't know what component to file this for; please adjust as needed.)

When downloading a container image that the libnbd project uses in CI on
gitlab.com, the container cannot be entered with SELinux enforcing.

*** Version-Release number of selected component (if applicable):

- podman: 4.2.0-11.el9_1
- selinux: 3.4-3.el9
- container-selinux: 2.189.0-1.el9
- selinux-policy: 34.1.43-1.el9_1.2

*** How reproducible:

Always.

*** Steps to Reproduce:

cd $HOME
podman system reset -f
rm -rf .local/share/containers
mkdir x
cd x
podman run -it --rm --userns=keep-id -v .:/repo:z -w /repo \
    registry.gitlab.com/nbdkit/libnbd/ci-alpine-edge:latest \
    bash

*** Actual results:

> Trying to pull registry.gitlab.com/nbdkit/libnbd/ci-alpine-edge:latest...
> Getting image source signatures
> Copying blob 0ded2f83af0e done
> Copying blob 88ecf269dec3 done
> Copying config a3b4bffb18 done
> Writing manifest to image destination
> Storing signatures
> Error relocating /usr/lib/libreadline.so.8: RELRO protection failed: Permission denied
> Error relocating /lib/ld-musl-x86_64.so.1: RELRO protection failed: Permission denied
> Error relocating /usr/lib/libncursesw.so.6: RELRO protection failed: Permission denied
> Error relocating /bin/bash: RELRO protection failed: Permission denied

*** Expected results:

> Trying to pull registry.gitlab.com/nbdkit/libnbd/ci-alpine-edge:latest...
> Getting image source signatures
> Copying blob 0ded2f83af0e done
> Copying blob 88ecf269dec3 done
> Copying config a3b4bffb18 done
> Writing manifest to image destination
> Storing signatures
> bash-5.2$

*** Additional info:

(1) The "id" command outputs:

> uid=1000(lacos) gid=1000(lacos)
> groups=1000(lacos),10(wheel),18(dialout),135(mock),975(libvirt),1001(lmda)
> context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

(2) The expected result is achievable when setting SELinux to
permissive.

(3) With SELinux permissive, a single AVC is generated. "sealert -a"
reports:

> SELinux is preventing /bin/bash from read access on the file
> /usr/lib/libreadline.so.8.2.
>
> *****  Plugin restorecon (99.5 confidence) suggests   ************************
>
> If you want to fix the label.
> /usr/lib/libreadline.so.8.2 default label should be lib_t.
> Then you can run restorecon. The access attempt may have been stopped
> due to insufficient permissions to access a parent directory in which
> case try to change the following command accordingly.
> Do
> # /sbin/restorecon -v /usr/lib/libreadline.so.8.2
>
> *****  Plugin catchall (1.49 confidence) suggests   **************************
>
> If you believe that bash should be allowed read access on the
> libreadline.so.8.2 file by default.
> Then you should report this as a bug.
> You can generate a local policy module to allow this access.
> Do
> allow this access for now by executing:
> # ausearch -c 'bash' --raw | audit2allow -M my-bash
> # semodule -X 300 -i my-bash.pp
>
>
> Additional Information:
> Source Context                system_u:system_r:container_t:s0:c62,c364
> Target Context                unconfined_u:object_r:user_home_t:s0
> Target Objects                /usr/lib/libreadline.so.8.2 [ file ]
> Source                        bash
> Source Path                   /bin/bash
> Port                          <Unknown>
> Host                          <Unknown>
> Source RPM Packages           bash-5.1.8-6.el9_1.x86_64
> Target RPM Packages
> SELinux Policy RPM            selinux-policy-targeted-34.1.43-1.el9_1.2.noarch
> Local Policy RPM              selinux-policy-targeted-34.1.43-1.el9_1.2.noarch
> Selinux Enabled               True
> Policy Type                   targeted
> Enforcing Mode                Permissive
> Host Name                     lacos-laptop-9.usersys.redhat.com
> Platform                      Linux lacos-laptop-9.usersys.redhat.com
>                               5.14.0-162.18.1.el9_1.x86_64 #1 SMP
>                               PREEMPT_DYNAMIC Thu Feb 9 04:28:41 EST 2023 x86_64
>                               x86_64
> Alert Count                   1
> First Seen                    2023-03-22 12:57:44 CET
> Last Seen                     2023-03-22 12:57:44 CET
> Local ID                      0db129a5-552f-49b2-b3bc-ec206978affb
>
> Raw Audit Messages
> type=AVC msg=audit(1679486264.987:145): avc:  denied  { read } for
> pid=2752 comm="bash" path="/usr/lib/libreadline.so.8.2" dev="dm-3"
> ino=2907654 scontext=system_u:system_r:container_t:s0:c62,c364
> tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1
>
>
> type=SYSCALL msg=audit(1679486264.987:145): arch=x86_64
> syscall=mprotect success=yes exit=0 a0=7f761e694000 a1=3000 a2=1
> a3=55744feb9c80 items=0 ppid=2749 pid=2752 auid=1000 uid=1000 gid=1000
> euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0
> ses=2 comm=bash exe=/bin/bash
> subj=system_u:system_r:container_t:s0:c62,c364 key=(null)ARCH=x86_64
> SYSCALL=mprotect AUID=lacos UID=lacos GID=lacos EUID=lacos SUID=lacos
> FSUID=lacos EGID=lacos SGID=lacos FSGID=lacos
>
> Hash: bash,container_t,user_home_t,file,read

Note that any complaints about "/usr/lib/libreadline.so.8.2" having
wrong labels are presumably bogus, given that this file exists within
the container.

(4) After the described failure, I tried

restorecon -FvvR ~/.local/share/containers
restorecon -FvvR ~/x

This relabels a big bunch of files, but then the same "podman" command
fails the same way.

The new AVC is effectively identical to the previous one; here's the
diff between the "sealert -a" outputs:

> @@ -24,7 +24,7 @@
>
>
>  Additional Information:
> -Source Context                system_u:system_r:container_t:s0:c62,c364
> +Source Context                system_u:system_r:container_t:s0:c436,c873
>  Target Context                unconfined_u:object_r:user_home_t:s0
>  Target Objects                /usr/lib/libreadline.so.8.2 [ file ]
>  Source                        bash
> @@ -44,15 +44,15 @@
>                                PREEMPT_DYNAMIC Thu Feb 9 04:28:41 EST 2023 x86_64
>                                x86_64
>  Alert Count                   1
> -First Seen                    2023-03-22 12:57:44 CET
> -Last Seen                     2023-03-22 12:57:44 CET
> -Local ID                      0db129a5-552f-49b2-b3bc-ec206978affb
> +First Seen                    2023-03-22 13:01:49 CET
> +Last Seen                     2023-03-22 13:01:49 CET
> +Local ID                      2771711b-e2af-4c92-840d-36573a4fb12a
>
>  Raw Audit Messages
> -type=AVC msg=audit(1679486264.987:145): avc:  denied  { read } for  pid=2752 comm="bash" path="/usr/lib/libreadline.so.8.2" dev="dm-3" ino=2907654 scontext=system_u:system_r:container_t:s0:c62,c364 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1
> +type=AVC msg=audit(1679486509.713:167): avc:  denied  { read } for  pid=3168 comm="bash" path="/usr/lib/libreadline.so.8.2" dev="dm-3" ino=2907654 scontext=system_u:system_r:container_t:s0:c436,c873 tcontext=unconfined_u:object_r:user_home_t:s0 tclass=file permissive=1
>
>
> -type=SYSCALL msg=audit(1679486264.987:145): arch=x86_64 syscall=mprotect success=yes exit=0 a0=7f761e694000 a1=3000 a2=1 a3=55744feb9c80 items=0 ppid=2749 pid=2752 auid=1000 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=2 comm=bash exe=/bin/bash subj=system_u:system_r:container_t:s0:c62,c364 key=(null)ARCH=x86_64 SYSCALL=mprotect AUID=lacos UID=lacos GID=lacos EUID=lacos SUID=lacos FSUID=lacos EGID=lacos SGID=lacos FSGID=lacos
> +type=SYSCALL msg=audit(1679486509.713:167): arch=x86_64 syscall=mprotect success=yes exit=0 a0=7f6318db1000 a1=3000 a2=1 a3=562c3fdd6c80 items=0 ppid=3165 pid=3168 auid=1000 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=2 comm=bash exe=/bin/bash subj=system_u:system_r:container_t:s0:c436,c873 key=(null)ARCH=x86_64 SYSCALL=mprotect AUID=lacos UID=lacos GID=lacos EUID=lacos SUID=lacos FSUID=lacos EGID=lacos SGID=lacos FSGID=lacos
>
>  Hash: bash,container_t,user_home_t,file,read

(5) This is similar to bug 1969996 and bug 2019324, but the instructions
described there don't work here.

Comment 1 Laszlo Ersek 2023-03-22 20:34:43 UTC
... if it matters, the kernel is 5.14.0-162.18.1.el9_1.x86_64

Comment 2 Tom Sweeney 2023-03-23 19:09:39 UTC
@dwalsh PTAL

Comment 3 Daniel Walsh 2023-03-25 11:19:39 UTC
This is caused by a broken install of container-selinux

sudo dnf -y reinstall container-selinux

And see if it blows up.

If it does not then try the restorecon -R -v ~/.local/containers

If it does, then try to update to a newer version of container-selinux

sudo dnf -y update container-selinux

If not is available then attempt to downgrade.

sudo dnf -y downgrade container-selinux

This is caused by a mismatch beteen selinux tooling and the container-selinux package.

Comment 4 Laszlo Ersek 2023-03-25 11:57:24 UTC
(In reply to Daniel Walsh from comment #3)
> This is caused by a broken install of container-selinux
>
> sudo dnf -y reinstall container-selinux
>
> And see if it blows up.

as root:

> # dnf reinstall container-selinux
> Updating Subscription Management repositories.
> Last metadata expiration check: 1:38:23 ago on Sat Mar 25 11:03:14 2023.
> Dependencies resolved.
> ======================================================================================================================
>  Package                    Architecture    Version                   Repository                                 Size
> ======================================================================================================================
> Reinstalling:
>  container-selinux          noarch          3:2.189.0-1.el9           rhel-9-for-x86_64-appstream-rpms           53 k
>
> Transaction Summary
> ======================================================================================================================
>
> Total download size: 53 k
> Installed size: 57 k
> Is this ok [y/N]: y
> Downloading Packages:
> container-selinux-2.189.0-1.el9.noarch.rpm                                             38 kB/s |  53 kB     00:01
> ----------------------------------------------------------------------------------------------------------------------
> Total                                                                                  38 kB/s |  53 kB     00:01
> Running transaction check
> Transaction check succeeded.
> Running transaction test
> Transaction test succeeded.
> Running transaction
>   Preparing        :                                                                                              1/1
>   Running scriptlet: container-selinux-3:2.189.0-1.el9.noarch                                                     1/2
>   Reinstalling     : container-selinux-3:2.189.0-1.el9.noarch                                                     1/2
>   Running scriptlet: container-selinux-3:2.189.0-1.el9.noarch                                                     1/2
>   Cleanup          : container-selinux-3:2.189.0-1.el9.noarch                                                     2/2
>   Running scriptlet: container-selinux-3:2.189.0-1.el9.noarch                                                     2/2
>   Verifying        : container-selinux-3:2.189.0-1.el9.noarch                                                     1/2
>   Verifying        : container-selinux-3:2.189.0-1.el9.noarch                                                     2/2
> Installed products updated.
>
> Reinstalled:
>   container-selinux-3:2.189.0-1.el9.noarch
>
> Complete!

As the normal user, repeated the repro steps from comment#0.

The symptom persists.

(In reply to Daniel Walsh from comment #3)
> If it does not then try the restorecon -R -v ~/.local/containers

I don't have a ~/.local/containers directory.

> If it does, then try to update to a newer version of container-selinux
>
> sudo dnf -y update container-selinux

There is no newer version of container-selinux available via the
"rhel-9-for-x86_64-appstream-rpms" repository / Subscription Management,
but I can try the latest build from Brew...

Ugh, I'd rather not. The latest build in Brew (buildID=2433735)
represents a huge version jump (from 2.189.0-1.el9 to 2.206.0-2.el9);
it's a candidate for RHEL-9.3, and I don't even have RHEL-9.2 installed.

> If not is available then attempt to downgrade.
>
> sudo dnf -y downgrade container-selinux

As root:

> # dnf downgrade container-selinux
> Updating Subscription Management repositories.
> Last metadata expiration check: 1:44:47 ago on Sat Mar 25 11:03:14 2023.
> Dependencies resolved.
> ======================================================================================================================
>  Package                   Architecture   Version                      Repository                                Size
> ======================================================================================================================
> Downgrading:
>  container-selinux         noarch         3:2.188.0-1.el9_0            rhel-9-for-x86_64-appstream-rpms          52 k
>
> Transaction Summary
> ======================================================================================================================
> Downgrade  1 Package
>
> Total download size: 52 k
> Is this ok [y/N]: y
> Downloading Packages:
> container-selinux-2.188.0-1.el9_0.noarch.rpm                                           21 kB/s |  52 kB     00:02
> ----------------------------------------------------------------------------------------------------------------------
> Total                                                                                  21 kB/s |  52 kB     00:02
> Running transaction check
> Transaction check succeeded.
> Running transaction test
> Transaction test succeeded.
> Running transaction
>   Preparing        :                                                                                              1/1
>   Running scriptlet: container-selinux-3:2.188.0-1.el9_0.noarch                                                   1/2
>   Downgrading      : container-selinux-3:2.188.0-1.el9_0.noarch                                                   1/2
>   Running scriptlet: container-selinux-3:2.188.0-1.el9_0.noarch                                                   1/2
>   Cleanup          : container-selinux-3:2.189.0-1.el9.noarch                                                     2/2
>   Running scriptlet: container-selinux-3:2.189.0-1.el9.noarch                                                     2/2
>   Running scriptlet: container-selinux-3:2.188.0-1.el9_0.noarch                                                   2/2
>   Verifying        : container-selinux-3:2.188.0-1.el9_0.noarch                                                   1/2
>   Verifying        : container-selinux-3:2.189.0-1.el9.noarch                                                     2/2
> Installed products updated.
>
> Downgraded:
>   container-selinux-3:2.188.0-1.el9_0.noarch
>
> Complete!

As the normal user, repeated the repro steps from comment#0.

The symptom persists.

(In reply to Daniel Walsh from comment #3)
> This is caused by a mismatch beteen selinux tooling and the
> container-selinux package.

Is it possible to keep Requires: directives up to date in the
appropriate RPM spec files? I can imagine the RH CDN being stale, but a
failed update (impossible to satisfy inter-package dependencies) is
clearer than podman-run failing.

Thanks!
Laszlo

Comment 8 Daniel Walsh 2023-09-06 18:39:04 UTC
The issue is the homedir is not labeled correctly.  Running restorecon -R -v $HOME should change all of the container labels in the homedir, as long as container-selinux was installed correctly.

Comment 9 RHEL Program Management 2023-09-11 19:36:29 UTC
Issue migration from Bugzilla to Jira is in process at this time. This will be the last message in Jira copied from the Bugzilla bug.

Comment 10 RHEL Program Management 2023-09-11 19:36:40 UTC
This BZ has been automatically migrated to the issues.redhat.com Red Hat Issue Tracker. All future work related to this report will be managed there.

Due to differences in account names between systems, some fields were not replicated.  Be sure to add yourself to Jira issue's "Watchers" field to continue receiving updates and add others to the "Need Info From" field to continue requesting information.

To find the migrated issue, look in the "Links" section for a direct link to the new issue location. The issue key will have an icon of 2 footprints next to it, and begin with "RHEL-" followed by an integer.  You can also find this issue by visiting https://issues.redhat.com/issues/?jql= and searching the "Bugzilla Bug" field for this BZ's number, e.g. a search like:

"Bugzilla Bug" = 1234567

In the event you have trouble locating or viewing this issue, you can file an issue by sending mail to rh-issues. You can also visit https://access.redhat.com/articles/7032570 for general account information.