Bug 1979497

Summary: podman v3.2.2 - cannot get logs when running in namespace with /var/log overmounted
Product: Red Hat Enterprise Linux 8 Reporter: Valentin Rothberg <vrothber>
Component: podmanAssignee: Jindrich Novy <jnovy>
Status: CLOSED ERRATA QA Contact: Joy Pu <ypu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 8.4CC: bbaude, dwalsh, jligon, jnovy, lsm5, mheon, pthomas, tsweeney, umohnani, ypu
Target Milestone: betaKeywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: podman-3.2.3-0.7.el8 or newer Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-11-09 17:40:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Valentin Rothberg 2021-07-06 08:14:57 UTC
Description of problem:

Podman v3.2.2 scheduled for RHEL 8.4.0.2 has a regression in getting logs when running inside a mount namespace with a changed /var/log.  That ultimately prevents accessing journald and Podman error out.

The issue has been reported upstream:
https://github.com/containers/podman/issues/10863

Version-Release number of selected component (if applicable):

Podman v3.2.2


How reproducible:

Always.


Additional info:

Already fixed in the main branch.  I will tackle the backports and report back once they've been merged into Podman v3.2.

Comment 1 Valentin Rothberg 2021-07-09 09:49:46 UTC
The fix has been backported (https://github.com/containers/podman/pull/10871) and merged into Podman's v3.2 branch. Assigning to Jindrich for packaging.

Comment 7 Joy Pu 2021-08-30 09:44:44 UTC
Test podman-3.3.1-4.module+el8.5.0+12418+ce3480d6.x86_64 with given steps in issue link and the error message is not show up again. So set this to verified, details:
$ unshare -Urm env _CONTAINERS_ROOTLESS_UID="$(id -u "${USER}")" _CONTAINERS_USERNS_CONFIGURED="true" sh -c 'mount -t tmpfs tmpfs /var/log && podman logs --cgroup-manager=cgroupfs --log-level=debug -fn "$(podman --cgroup-manager=cgroupfs run -d docker.io/library/alpine sh -c "sleep 2; echo hi")"'
WARN[0000] additional gid=1 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=2 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=3 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=4 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=6 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=10 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=11 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=20 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=26 is not present in the user namespace, skip setting it 
WARN[0000] additional gid=27 is not present in the user namespace, skip setting it 
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called logs.PersistentPreRunE(podman logs --cgroup-manager=cgroupfs --log-level=debug -fn cdc8d2c3127901f18eb802ceb096238255736aba8a9677d820e11daad866b31a) 
DEBU[0000] cached value indicated that overlay is supported 
DEBU[0000] Merged system config "/usr/share/containers/containers.conf" 
DEBU[0000] cached value indicated that overlay is supported 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
DEBU[0000] Initializing boltdb state at /home/test/.local/share/containers/storage/libpod/bolt_state.db 
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /home/test/.local/share/containers/storage 
DEBU[0000] Using run root /tmp/podman-run-1000/containers 
DEBU[0000] Using static dir /home/test/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /tmp/run-1000/libpod/tmp       
DEBU[0000] Using volume path /home/test/.local/share/containers/storage/volumes 
DEBU[0000] cached value indicated that overlay is supported 
DEBU[0000] Set libpod namespace to ""                   
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] cached value indicated that overlay is supported 
DEBU[0000] cached value indicated that metacopy is not being used 
DEBU[0000] cached value indicated that native-diff is usable 
DEBU[0000] backingFs=xfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false 
DEBU[0000] Initializing event backend file              
DEBU[0000] configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
INFO[0000] Found CNI network podman (type=bridge) at /home/test/.config/cni/net.d/87-podman.conflist 
DEBU[0000] Default CNI network name podman is unchangeable 
INFO[0000] Setting parallel job count to 7              
DEBU[0000] Failed to add podman to systemd sandbox cgroup: dial unix /run/user/0/bus: connect: permission denied 
DEBU[0000] Initializing event backend file              
hi
DEBU[0002] Called logs.PersistentPostRunE(podman logs --cgroup-manager=cgroupfs --log-level=debug -fn cdc8d2c3127901f18eb802ceb096238255736aba8a9677d820e11daad866b31a) 
DEBU[0002] [graphdriver] trying provided driver "overlay" 
DEBU[0002] cached value indicated that overlay is supported 
DEBU[0002] cached value indicated that metacopy is not being used 
DEBU[0002] backingFs=xfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false

Comment 9 errata-xmlrpc 2021-11-09 17:40:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4154