As noted above cri-o 1.35 starts an runs on F44 Cloud (as a vm).

Forgot to add:

[vagrant@localhost crio]$ uname -a
Linux localhost.localdomain 7.0.7-200.fc44.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 14 17:19:46 UTC 2026 x86_64 GNU/Linux


On the F44 Cloud vm, selinux labels look correct? 

[vagrant@localhost crio]$ ls -laZ /var/run/crio/
total 4
drwxr-xr-x.  5 root root system_u:object_r:container_var_run_t:s0  140 May 15 18:10 .
drwxr-xr-x. 46 root root system_u:object_r:var_run_t:s0           1160 May 15 18:10 ..
srw-rw----.  1 root root system_u:object_r:container_var_run_t:s0    0 May 15 18:10 crio.sock
drwxr-x---.  2 root root system_u:object_r:container_var_run_t:s0   40 May 15 18:10 exec-pid-dir
drwxr-xr-x.  2 root root system_u:object_r:container_var_run_t:s0   40 May 15 18:10 exits
drwx------.  2 root root system_u:object_r:container_var_run_t:s0   40 May 15 18:10 seccomp
-rw-r--r--.  1 root root system_u:object_r:container_var_run_t:s0   16 May 15 18:10 version

[vagrant@localhost crio]$ ls -laZ /usr/bin/crio
-rwxr-xr-x. 1 root root system_u:object_r:container_runtime_exec_t:s0 75403848 Apr  2 00:00 /usr/bin/crio
[vagrant@localhost crio]$

Some notes (thanks to Jonathan's help with CoreOS)

1. cri-o works on fedora 44 cloud 7.0.7 kernel.
2. Fails on fcos testing (7.0.7 kernel) with the noted error above (shared library permission denied)
3. 'sudo crio status config' also returns the same error.
4. Turn off selinux enforcing and cri-o works (e.g. systemctl start crio.service)
5. copy crio binary to /var/tmp preserving ownership/context and the binary works ('sudo /var/tmp/crio status config' returns a result (with selinux enforcing).
6. Disable composefs ($ sudo rpm-ostree kargs --append='ostree.prepare-root.composefs=0') and cri-o works.
7. cri-o works on fcos stable (6.x kernel).

Also verified that cri-o1.35 fails in the same way. Expected.

sudo ausearch -m AVC -ts recent results in man repetitions of the following types of entries:

"time->Sun May 24 15:43:17 2026
type=AVC msg=audit(1779637397.144:90): avc:  denied  { search } for  pid=1526 comm="lsblk" name="mount" dev="tmpfs" ino=820 scontext=system_u:system_r:bootupd_t:s0 tcontext=system_u:object_r:mount_var_run_t:s0 tclass=dir permissive=1"

with a variety of 'name' and 'dev' combinations along with variations in { read }, { write }, { connectto }, and { search }

and a number of the following crio related entries, each with a different pid.

"time->Sun May 24 15:46:00 2026
type=AVC msg=audit(1779637560.775:183): avc:  denied  { execmem } for  pid=1476 comm="crio" scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=process permissive=0"

Within the last not-very-many hours, Fedora CoreOS stable has promoted from v44.20260419.3.1 to v44.20260510.3.1 --- and that includes a kernel-6.19.14-101.fc44.[arch] to kernel-7.0.8-200.fc44.[arch] update.  I assume there's no choice to e.g., pause the CoreOS rollout.  I also assume there isn't a reasonable hotfix that can be patched onto cri-o1.35 in a hurry.  (I'll have a dig in the next 12--24 hours, but I've not had the pleasure of adventuring into this code before.)

The coincidence of the audit message with systemd trying to bring up crio.service makes me suspect that #c4 has exactly the right audit message.  On my machines, I see:

```
% journalctl --no-hostname --output=short-monotonic --boot=0
[...]
[60158.141724] systemd[1]: Starting crio.service...
[60158.158444] (crio)[87237]: crio.service: Referenced but unset environment variable evaluates to an empty string: CRIO_CONFIG_OPTIONS, CRIO_METRICS_OPTIONS, CRIO_NETWORK_OPTIONS, CRIO_RUNTIME_OPTIONS, CRIO_STORAGE_OPTIONS
[60158.160330] audit[87237]: AVC avc:  denied  { execmem } for  pid=87237 comm="crio" scontext=system_u:system_r:kernel_t:s0 tcontext=system_u:system_r:kernel_t:s0 tclass=process permissive=0
[60158.161449] crio[87237]: /usr/bin/crio: error while loading shared libraries: cannot make segment writable for relocation: Permission denied
[60158.162795] systemd[1]: crio.service: Main process exited, code=exited, status=127/n/a
[60158.163127] systemd[1]: crio.service: Failed with result 'exit-code'.
[60158.163806] systemd[1]: Failed to start crio.service.
[60158.163329] audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=crio comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
[...]
```

(Mildly hilarious addenda to #c3: doing `rpm-ostree install strace`, then `rpm-ostree apply-live`, also made the issue go away.)

For completeness, a brief precis of my production-canary machine that took this update and is now sad ---

```
core@blyss:~$ uname -a
Linux blyss.[domain] 7.0.8-200.fc44.x86_64 #1 SMP PREEMPT_DYNAMIC Fri May 15 14:03:46 UTC 2026 x86_64 GNU/Linux

core@blyss:~$ grep -E '^(NAME|VERSION|RELEASE_TYPE)=' /etc/os-release 
NAME="Fedora Linux"
VERSION="44.20260510.3.1 (CoreOS)"
RELEASE_TYPE=stable

core@blyss:~$ rpm -q cri-o1.35
cri-o1.35-1.35.3-1.fc44.x86_64

core@blyss:~$ ldd $(which crio)
	linux-vdso.so.1 (0x00007ffff7fc3000)
	libresolv.so.2 => /lib64/libresolv.so.2 (0x00007ffff180a000)
	libseccomp.so.2 => /lib64/libseccomp.so.2 (0x00007ffff17de000)
	libgpgme.so.45 => /lib64/libgpgme.so.45 (0x00007ffff1787000)
	libc.so.6 => /lib64/libc.so.6 (0x00007ffff158c000)
	libassuan.so.0 => /lib64/libassuan.so.0 (0x00007ffff1577000)
	libgpg-error.so.0 => /lib64/libgpg-error.so.0 (0x00007ffff1548000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc5000)

core@blyss:~$ rpm -qf /lib64/ld-linux-x86-64.so.2 /lib64/libresolv.so.2 /lib64/libseccomp.so.2 /lib64/libgpgme.so.45 /lib64/libc.so.6 /lib64/libassuan.so.0 /lib64/libgpg-error.so.0
glibc-2.43-4.fc44.x86_64
glibc-2.43-4.fc44.x86_64
libseccomp-2.6.0-3.fc44.x86_64
gpgme-2.0.1-4.fc44.x86_64
glibc-2.43-4.fc44.x86_64
libassuan-2.5.7-5.fc44.x86_64
libgpg-error-1.58-2.fc44.x86_64
```

It is interesting, but perhaps not necessarily relevant, that we also took a version bump of glibc-2.43-2.fc44 to glibc-2.43-4.fc44 (FEDORA-2026-79d2a70ad1; FEDORA-2026-ced72ab158), and of gpgme-2.0.1-3.fc44 to gpgme-2.0.1-4.fc44 (FEDORA-2026-16846f7d5d).  I had a quick peek at the summaries on Bodhi but saw nothing that jumped out as relevant.

If I do

```
core@blyss:~$ setarch $(uname -m) --addr-no-randomize strace -o x -ff -X verbose -v crio
crio: error while loading shared libraries: cannot make segment writable for relocation: Permission denied
```

and squint at the output file, I have some perhaps interesting observations:

```
brk(NULL)                               = 0x555a2eafe000
[... lots of syscalls elided ...]
mmap(NULL, 8192, 0x3 /* PROT_READ|PROT_WRITE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x7f58158a7000
mmap(NULL, 12288, 0x3 /* PROT_READ|PROT_WRITE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x7f58158a4000
arch_prctl(0x1002 /* ARCH_SET_FS */, 0x7f58158a4780) = 0
set_tid_address(0x7f58158a4da8)         = 2501
set_robust_list(0x7f58158a4a60, 24)     = 0
rseq({cpu_id_start=0, cpu_id=-1 /* RSEQ_CPU_ID_UNINITIALIZED */, rseq_cs=NULL, flags=0, node_id=0, mm_cid=0, slice_ctrl={request=0, granted=0, __reserved=0}, __reserved=0}, 33, 0, 0x53053053) = 0
mprotect(0x7f5815ad8000, 16384, 0x1 /* PROT_READ */) = 0
mprotect(0x7f58158d4000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f58158e9000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f5815b3a000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f5815b67000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f5815b78000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x555a0b61f000, 27312128, 0x7 /* PROT_READ|PROT_WRITE|PROT_EXEC */) = -1 EACCES (Permission denied)
writev(2, [{iov_base="crio", iov_len=4}, {iov_base=": ", iov_len=2}, {iov_base="error while loading shared libra"..., iov_len=36}, {iov_base=": ", iov_len=2}, {iov_base="", iov_len=0}, {iov_base="", iov_len=0}, {iov_base="cannot make segment writable for"..., iov_len=43}, {iov_base=": ", iov_len=2}, {iov_base="Permission denied", iov_len=17}, {iov_base="\n", iov_len=1}], 10) = 107
exit_group(127)                         = ?
+++ exited with 127 +++
```

Comparing the addresses to what had been previously loaded, the sequence of `mprotect` calls that set `PROT_READ` patch up parts of the mappings for, respectively, -lc, -lgpg-error, -lassuan, -lgpgme, -lseccomp, -lresolv; fine.  But the next `mprotect`, which tries to grab 6668 pages / 0x1a0c000 bytes / ~26 MiB, confuses me.  I *think* this is supposed be <https://forge.sourceware.org/glibc/glibc-mirror/src/tag/glibc-2.43/elf/dl-reloc.c#L281-L286>.

Of the ELF objects loaded, I only see one with a section that's anywhere near the right size: the first of the `LOAD` mappings of `/usr/bin/crio` itself.

```
lisbon% llvm-objdump --all-headers --syms --disassemble --source --demangle crio
[...]
Program Header:
[...]
    LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**12
         filesz 0x0000000001a0bf69 memsz 0x0000000001a0bf69 flags r-x
    LOAD off    0x0000000001a0c000 vaddr 0x0000000001a0c000 paddr 0x0000000001a0c000 align 2**12
         filesz 0x00000000025a2afc memsz 0x00000000025a2afc flags r--
    LOAD off    0x0000000003faef80 vaddr 0x0000000003faff80 paddr 0x0000000003faff80 align 2**12
         filesz 0x000000000078ea21 memsz 0x00000000027e8a80 flags rw-
[...]
```

Interestingly, we also went from selinux-policy-43.6-1.fc44 to selinux-policy-44.1-1.fc44 (FEDORA-2026-e0ab4349fe); but I don't see anything relevant in the patch set: <https://github.com/fedora-selinux/selinux-policy/compare/v43.6...v44.1>. 

Comparing the `strace ... crio` output from a machine that hasn't taken the update from v44.20260419.3.1 to v44.20260510.3.1 yet, it turns out that the mprotect r+w+x is an expected behaviour (?!) --- starting from an equivalent point in the trace, which is immediately after the last shlib fd is closed:

```
mmap(NULL, 8192, 0x3 /* PROT_READ|PROT_WRITE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x7f3722cda000
mmap(NULL, 12288, 0x3 /* PROT_READ|PROT_WRITE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x7f3722cd7000
arch_prctl(0x1002 /* ARCH_SET_FS */, 0x7f3722cd7740) = 0
set_tid_address(0x7f3722cd7d68)         = 806100
set_robust_list(0x7f3722cd7a20, 24)     = 0
rseq({cpu_id_start=0, cpu_id=-1 /* RSEQ_CPU_ID_UNINITIALIZED */, rseq_cs=NULL, flags=0, node_id=0, mm_cid=0, slice_ctrl={request=0, granted=0, __reserved=0}, __reserved=0}, 32, 0, 0x53053053) = 0
mprotect(0x7f3722f0b000, 16384, 0x1 /* PROT_READ */) = 0
mprotect(0x7f3722d07000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f3722d1c000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f3722f6d000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f3722f9a000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x7f3722fab000, 4096, 0x1 /* PROT_READ */) = 0
mprotect(0x55efd99a1000, 27312128, 0x7 /* PROT_READ|PROT_WRITE|PROT_EXEC */) = 0
mprotect(0x55efdb3ad000, 39464960, 0x3 /* PROT_READ|PROT_WRITE */) = 0
mprotect(0x55efdb3ad000, 39464960, 0x1 /* PROT_READ */) = 0
mprotect(0x55efd99a1000, 27312128, 0x5 /* PROT_READ|PROT_EXEC */) = 0
mprotect(0x55efdd950000, 7020544, 0x1 /* PROT_READ */) = 0
mprotect(0x7f3722ff5000, 8192, 0x1 /* PROT_READ */) = 0
prlimit64(0, 0x3 /* RLIMIT_STACK */, NULL, {rlim_cur=8388608 /* 8192*1024 */, rlim_max=18446744073709551615 /* RLIM64_INFINITY */}) = 0
getrandom("\x9c\x84\xbd\xc9\x41\x35\x85\xc8", 8, 0x1 /* GRND_NONBLOCK */) = 8
munmap(0x7f3722fb1000, 17603)           = 0
brk(NULL)                               = 0x55effd26c000
```

That behaviour, and the AVC-deny noted earlier, both sound like exactly the shape of the nasty LSM / overlayfs interaction described in PR <https://lore.kernel.org/lkml/9a2a59bd8d9548fb5dab128f4859fa3d@paul-moore.com/>, and referenced by <https://github.com/alicefr/bink/issues/8#issuecomment-4499582027>: those patches were, indeed, landed in 7.1-rc1, and have since been picked up by stable-7.0.y.

... and made it into 7.0.4.  But #c2 and #c6 both cite stable versions later than that?

*** Bug 2483872 has been marked as a duplicate of this bug. ***

Still present with:

[core@cosa-devsh ~]$ uname -a
Linux cosa-devsh 7.0.9-205.fc44.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 21 16:31:48 UTC 2026 x86_64 GNU/Linux

A workaround is noted at https://github.com/coreos/fedora-coreos-tracker/issues/2148#issuecomment-4586710639 (thank you @deadlyhawk) (copied below):

cat > /tmp/crio-execmem.cil <<'EOF'
(typeattributeset cil_gen_require kernel_t)
(allow kernel_t self (process (execmem)))
EOF

semodule -i /tmp/crio-execmem.cil
systemctl restart crio

(In reply to Jashank Jeremy from comment #7)
>
> 
> ... and made it into 7.0.4.  But #c2 and #c6 both cite stable versions later
> than that?

I searched the Fedora 7.0.9 kernel for the specific commit related to https://lore.kernel.org/lkml/9a2a59bd8d9548fb5dab128f4859fa3d@paul-moore.com/ and did not find it.

I did find it in the forthcoming 7.1 kernel and a follow on patch.

I used ' git log --grep lsm-pr --since="2026-03-01" ' on a local copy of the Fedora kernel-ark repo.

The nature of stable means the PR merge won't appear, but the cherrypicked commits will (with different IDs).

On the mainline: <https://gitlab.com/cki-project/kernel-ark/-/commit/82544d36b1729153c8aeb179e84750f0c085d3b1>
On stable/7.0.y: <https://gitlab.com/cki-project/kernel-ark/-/commit/cd0e707a927a70cdfd8bc5a512a9719a87f5ed51>

Moreover, Gitlab reports that the stable cherrypick is in these tags, here in version order --- v7.0.4 kernel-7.0.4-0 v7.0.5 v7.0.6 kernel-7.0.6-0 v7.0.7 kernel-7.0.7-0 v7.0.8 kernel-7.0.8-0 v7.0.9 kernel-7.0.9-0 kernel-7.0.9-1 kernel-7.0.9-3 kernel-7.0.9-4 kernel-7.0.9-5 v7.0.10 kernel-7.0.10-0 kernel-7.0.10-1 v7.0.11 kernel-7.0.11-1

I also note the presence of `selinux_mmap_backing_file`, newly-defined in that series in `kernel/security/selinux/hooks.c`, in the kernel symbol table...

```
% grep selinux_mmap_backing_file /lib/modules/7.0.8-200.fc44.x86_64/System.map
ffffffff81bcff50 t __pfx_selinux_mmap_backing_file
ffffffff81bcff60 t selinux_mmap_backing_file
``

I do not claim to have a coherent argument of what workaround works best, and I certainly do not claim to have a general sense of threat models and risk tolerance.  But personally, and especially in the present atmosphere of a plethora of extraordinary kernel bugs, I am much more averse to a SELinux policy that does `(allow kernel_t self (process (execmem)))`.  On my canary machine, I instead set `ostree.prepare-root.composefs=0`.  (On the rest of my CoreOS fleet, I've frozen Zincati, and the nature of my workload means I'm not averse to staying in that state for a few months.)

One interesting addenda from a F44 non-CoreOS box with kernel-7.0.8-200.fc44.x86_64: the SELinux test suite grew some cases for _mmap(2)_ and _mprotect(2)_, as <https://github.com/SELinuxProject/selinux-testsuite/commit/7fe74d71d49f462570e9f8339add026d0768f915>, and those cases don't fail, so I think there's some subtlety to this issue that hasn't yet been captured by the SELinux folks.

I haven't enough deep Linux-kernel / SELinux / overlayfs expertise (other than what I've gleaned in a hurry to get to this point) to really have a good idea of where to go next.  It seems like there's still a bug that the mmap-and-mprotect patch series in `lsm/lsm-pr-20260410` hasn't fixed, and I think this probably needs the attention of some SELinux folks.  I also I note that source- and target-context `system_u:system_r:kernel_t:s0` smells deeply suspicious: I do not know how or why a normal user-space process, nor its backing-file (even through an overlayfs constructed by ostree magick), has ended up in that context.

(In reply to Jashank Jeremy from comment #6)
> (Mildly hilarious addenda to #c3: doing `rpm-ostree install strace`, then
> `rpm-ostree apply-live`, also made the issue go away.)
> 

Thank you for your detailed notes information. Just for kicks I did the following:



sudo rpm-ostree install cri-o1.36 containernetworking-plugins
sudo rpm-ostree apply-live
sudo sed -i '/\/usr\/libexec\/cni/d' /etc/crio/crio.conf
sudo systemctl start crio.service
(with this error: error: Running tmpfiles for /run and /var: Running systemd worker: exit status: 73)

And cri-o will start without a problem. Since only dabble occasionally with CoreOS I am not sure where to go next.

(In reply to Brad Smith from comment #12)
> And cri-o will start without a problem. Since only dabble occasionally with
> CoreOS I am not sure where to go next.

You probably need to reboot to reproduce the problem, since apply-live appears to be a workaround.

I've spent some more time thinking about this, and I'm now fairly convinced that my assertion on comment #11 is true:

> It seems like there's still a bug that the mmap-and-mprotect patch series in `lsm/lsm-pr-20260410` hasn't fixed, and I think this probably needs the attention of some SELinux folks.

I suspect that

* either there's additional subtlety to the overlayfs breakage, due to composefs' stacking of an EROFS-with-redirects as the top layer of the overlayfs, which yields some sort of backing-file confusion;

* or there is a really nasty bug in the patch, causing some sort of credential confusion: we're either using the wrong process-credentials, or the wrong file-credentials, or the wrong backing-file credentials, or something even more catastrophic is going on.

Unfortunately, the more I think about this, the more my head hurts.  I'm inclined to think the latter case proposed above is true: my first pass reading through this bit of the SELinux implementation has left me with some ideas, but also led me to the point of "but how do *any* dynamically-loaded executables work," and then my face fell off.  I don't know what the escalation path is to SELinux/LSM folks, but this really needs some expert guidance by subject-matter experts.

(Incidentally, RedHatters --- I am extremely disappointed to see <https://github.com/openshift/microshift/pull/6903>.  There's ample evidence above that the `lsm-pr-20260410` patch series did *not* fix this problem.  And, as noted on comment #11, I think it is **deeply unwise** to set `allow kernel_t self:process execmem`: please consider carefully what this allows, and in the current climate of intricate and alarming kernel bugs, why one might like to avoid messing with the execute bit on effectively arbitrary memory in protected mode...)

w.r.t the workaround confusion on comment #12: remember that, in the context of an atomic system, `rpm-ostree apply-live` is a strange impurity that will almost certainly violate some or other invariants of the system, and one probably shouldn't want to use it.  I'm not _quite_ sure what `rpm-ostree apply-live` is doing under the hood, but I suspect this is adjusting the composefs/overlayfs in some way that means whatever backing-file-confusion bug is going on isn't triggered.

Hello everyone,

The kernel_t denials are actually caused by an oversight in kernel commit 82544d36b172 ("selinux: fix overlayfs mmap() and mprotect() access checks") [a.k.a. lsm-pr-20260410, a.k.a. fix for CVE-2026-46054]. I have a reproducer and a working fix and I'm currently working to get it upstream. Stay tuned, I will try to keep this bug updated...

As for the policy workaround - allowing kernel_t the execmem permission - that's actually OK and mostly harmless. The execmem permission doesn't restrict the kernel itself, only userspace tasks, so allowing it doesn't make things worse w.r.t. kernel vulnerabilities. It does apply to user tasks that started from the initramfs before the SELinux policy was loaded (these currently end up with the kernel_t label), but those normally shouldn't occur at all (barring some bugs or race conditions), so the potential attack surface is very small there.

Ondrej - thank you for the update and information.

Update: I sent a patch upstream and the v2 has been accepted [1] and should get to the mainline kernel soon (and then propagate to stable releases shortly after).

[1] https://web.git.kernel.org/pub/scm/linux/kernel/git/pcmoore/selinux.git/commit/?h=stable-7.2&id=9fe595fad54d4ac6a402edb3f60bec859d52cea6