Bug 1891220
| Summary: | Unable to run podman rootless, Error: cannot re-exec process | ||
|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Peter Portante <pportant> |
| Component: | podman | Assignee: | Giuseppe Scrivano <gscrivan> |
| Status: | CLOSED EOL | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 31 | CC: | acui, bbaude, container-sig, debarshir, dwalsh, gscrivan, jnovy, lsm5, mheon, rh.container.bot, santiago |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2020-11-24 20:17:32 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Peter Portante
2020-10-24 14:33:21 UTC
Hi Peter, have you tried v2.1.1? [pportant@gandalf ~]$ podman run -it --name demo --rm centos:8 /bin/bash Error: cannot re-exec process [pportant@gandalf ~]$ podman --version podman version 2.1.1 Adding Giuseppe to CC - this looks like the rootless C code failing. What does $ podman unshare cat /proc/self/uid_map output? [pportant@gandalf ~]$ podman unshare cat /proc/self/uid_map Error: cannot re-exec process Do you have a podman running in your user session? ps -ef | grep podman filecap /usr/bin/newuidmap; filecap /usr/bin/newgidmap set file capabilities rootid effective /usr/bin/newuidmap setuid set file capabilities rootid effective /usr/bin/newgidmap setgid [pportant@gandalf rpm]$ ps -ef | grep podman pportant 1454363 3096635 0 16:51 pts/1 00:00:00 grep --color=auto podman [pportant@gandalf rpm]$ filecap /usr/bin/newuidmap; filecap /usr/bin/newgidmap set file capabilities effective /usr/bin/newuidmap setuid set file capabilities effective /usr/bin/newgidmap setgid [pportant@gandalf rpm]$ Ok I am out of options. Looks like something is going wrong when it creates the usernamespaced podman. Giuseppe any ideas? If you want to hop on the box and debug let me know. I can also try to update to Fedora 32 or 33 and retry. By anychance do you have your homedir mounted noexec or nosuid? [pportant@gandalf images]$ sudo mount /dev/mapper/system-home on /home type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota) sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel) proc on /proc type proc (rw,nosuid,nodev,noexec,relatime) devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=16367324k,nr_inodes=4091831,mode=755) securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime) tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel) devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000) tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755) cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate) pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel) none on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700) configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime) /dev/mapper/system-root on / type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota) selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime) systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19913) debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel) hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M) mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel) fusectl on /sys/fs/fuse/connections type fusectl (rw,nosuid,nodev,noexec,relatime) tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel) /dev/mapper/system-var on /var type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota) /dev/sda2 on /boot type ext4 (rw,relatime,seclabel) /dev/mapper/system-var_log on /var/log type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota) /dev/mapper/system-var_tmp on /var/tmp type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota) /dev/mapper/system-opt on /opt type xfs (rw,relatime,seclabel,attr2,inode64,logbufs=8,logbsize=32k,noquota) sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime) tmpfs on /run/user/42 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=3277644k,mode=700,uid=42,gid=42) tracefs on /sys/kernel/debug/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel) tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=3277644k,mode=700,uid=1000,gid=1000) tmpfs on /run/netns type tmpfs (rw,nosuid,nodev,seclabel,mode=755) could you show me the output for: $ cat /proc/sys/user/max_user_namespaces $ unshare -U sleep 100 & $ newuidmap $! 0 $(id -u) 1 1 100000 65536 $ newgidmap $! 0 $(id -g) 1 1 100000 65536 [pportant@gandalf images]$ cat /proc/sys/user/max_user_namespaces
127869
# The above has not changed since I posted this bug, see the "additional info" section of the description.
[pportant@gandalf images]$ unshare -U sleep 100 &
[1] 1537131
[pportant@gandalf images]$ jobs
[1]+ Running unshare -U sleep 100 &
[pportant@gandalf images]$ newuidmap $! 0 $(id -u) 1 1 100000 65536
[pportant@gandalf images]$ newgidmap $! 0 $(id -g) 1 1 100000 65536
[pportant@gandalf images]$ jobs
[1]+ Running unshare -U sleep 100 &
[pportant@gandalf images]$ ps jx
PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
147540 147554 147540 147540 ? -1 S 1000 0:18 sshd: pportant@pts/0
147554 147558 147558 147558 pts/0 1537147 Ss 1000 0:08 -bash
1 294224 294224 294224 ? -1 Ss 1000 0:01 /usr/lib/systemd/systemd --user
294224 294234 294224 294224 ? -1 S 1000 0:00 (sd-pam)
294224 294359 294359 294359 ? -1 Ss 1000 0:00 /usr/bin/dbus-broker-launch --scope user
294359 294360 294359 294359 ? -1 S 1000 0:00 dbus-broker --log 4 --controller 11 --machine-id dece48160fd04636874c91c57e2a853c --max-bytes 100000000000000 --max-fds 25000000000000 --max-matches 5000000000
751901 751915 751901 751901 ? -1 S 1000 0:00 sshd: pportant@pts/4
751915 751917 751917 751917 pts/4 751917 Ss+ 1000 0:00 /bin/sh
294224 896590 896590 896590 ? -1 Ssl 1000 0:00 /usr/bin/gnome-keyring-daemon --start --foreground --components=secrets
1133593 1133604 1133593 1133593 ? -1 S 1000 0:01 sshd: pportant@pts/2
1133604 1133609 1133609 1133609 pts/2 1133609 Ss+ 1000 0:00 -bash
147558 1537131 1537131 147558 pts/0 1537147 S 1000 0:00 sleep 100
147558 1537147 1537147 147558 pts/0 1537147 R+ 1000 0:00 ps jx
1 3096634 3096604 3096604 ? -1 S 1000 4:58 mosh-server new -c 256 -s -l LANG=en_US.UTF-8
3096634 3096635 3096635 3096635 pts/1 3096635 Ss+ 1000 0:15 -bash
Thanks, the issue seems to be in Podman as newuidmap/newgidmap work fine. Do you have a /run/user/1000/libpod/pause.pid file? If you do, what is its value? If it is possible, I could try accessing your node and see what is happening. [pportant@gandalf images]$ ls -ld /run/user/1000/libpod/pause.pid -rw-------. 1 pportant pportant 7 Aug 9 23:36 /run/user/1000/libpod/pause.pid [pportant@gandalf images]$ cat /run/user/1000/libpod/pause.pid 2214917<NO EOL> I'll contact you offline about accessing the machine directly. it seems the error could be triggered by having an invalid pause pid file, as well as an invalid conmon pid file pointing to another process. In this case the best solution is to use "podman system migrate" and restart the pause process. I've opened a PR to give a better error message when it happens: https://github.com/containers/podman/pull/8173 This message is a reminder that Fedora 31 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '31'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 31 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed. |