Bug 1882136
| Summary: | created toolbox fails to start: open /proc/sys/net/ipv4/ping_group_range: Permission denied: OCI runtime permission denied error | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Martin Pitt <mpitt> |
| Component: | podman | Assignee: | Lokesh Mandvekar <lsm5> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 33 | CC: | bbaude, debarshir, dustymabe, dwalsh, harrymichal, jnovy, lsm5, mheon, pehunt, rh.container.bot, santiago, umohnani, vrutkovs |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-10-09 04:14:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Martin Pitt
2020-09-23 21:18:55 UTC
I forgot: Since last week's update, podman and toolbox remained the same. The meaningful package updates in "sudo rpm-ostree db diff" are: conmon 2:2.0.21-0.3.dev.git5a6b2ac.fc33 -> 2:2.0.21-3.fc33 kernel 5.8.8-300.fc33 -> 5.8.10-300.fc33 CC'ing Debarshi as that affects toolbox. Hmm, I tried to downgrade to the previous conmon: curl https://kojipkgs.fedoraproject.org//packages/conmon/2.0.21/0.3.dev.git5a6b2ac.fc33/x86_64/conmon-2.0.21-0.3.dev.git5a6b2ac.fc33.x86_64.rpm | rpm2cpio | cpio -ivdD /tmp/ ./usr/bin/conmon sudo mount -o bind /tmp/usr/bin/conmon /usr/bin/conmon sudo setenforce 0 But that doesn't seem to help, still the same error. Conversely, I booted back into the previous deployment and ran the conmon binary from 2:2.0.21-3.fc33, and that works fine. So it's not conmon after all. Looks like toolbox needs to adjust its configuration to some recent kernel changes somehow? It's this Podman issue: https://github.com/containers/podman/issues/7766 Thanks Debarshi! Confirming the bug with recent updates: podman-2.1.1-2.fc33.x86_64 runc-1.0.0-279.dev.gitdedadbf.fc33.x86_64 kernel-5.8.12-300.fc33.x86_64 containers-common-1.1.1-10.fc33.x86_64 crun-0.15-3.fc33.x86_64 I don't have a containers.conf anywhere (that's not created by default apparently), and notice that https://github.com/containers/podman/issues/7766 is slightly different: That has Error: write to /proc/sys/net/ipv4/ping_group_range: Invalid argument: OCI runtime error and running "podman run --uidmap 0:10000:10000 quay.io/libpod/testimage:20200902 true" *does* work. The bug here is open /proc/sys/net/ipv4/ping_group_range: Permission denied: OCI runtime permission denied error Nevertheless, it's certainly related, as creating a containers.conf and disabling the sysctl works: mkdir ~/.config/containers; printf '[containers]\ndefault_sysctls = []\n' > ~/.config/containers/containers.conf This is a good enough workaround for now \o/ which unblocks the workflow, and doesn't need me to go back to an old ostree to recreate toolboxes \o/ This somehow seems to be part of a container definition. If I do rm ~/.config/containers/containers.conf toolbox create -c noconf printf '[containers]\ndefault_sysctls = []\n' > ~/.config/containers/containers.conf toolbox create -c emptysysctl Then noconf fails, and emptysysctl succeeds a "toolbox enter". I looked at diff -u <(podman inspect x) <(podman inspect y) which unfortunately has a lot of noise due to the unsorted Mounts and Binds maps, but after sorting these a little there really is no significant difference between. .local/share/containers/storage/overlay-containers/<uuid>/ is identical for both, they just have an empty userdata/artifacts/ subdir (other containers have a config.json and ctr.log). The config in .local/share/containers/storage/overlay-containers/containers.json also looks very similar, the only difference that isn't trivial (like UUID or timestamps) is "flags": { - "MountLabel": "system_u:object_r:container_file_t:s0:c816,c938", + "MountLabel": "system_u:object_r:container_file_t:s0:c304,c863", "ProcessLabel": "" } (Not sure if anything meaningful is encoded there) We believe this is fixed in podman-2.1.1-10.fc33 and newer. Can you verify? Confirmed. I updated to latest Fedora, including podman-2.1.1-11.fc33.x86_64, and it once again works with the default configuration. Thank you! |