Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Description of problem:
When runing podman container that specifies --cap-add CAP_SYS_ADMIN(This option specifies that perform a range of system administration operations [1]) and performing nfs mount in it, but the result fails. which is a different behavior from docker that specifiles "--cap-add SYS_ADMIN" . ( If this behavior is your expected.)
Version-Release number of selected component (if applicable):
kernel-4.18.0-147.el8.x86_64
Steps to Reproduce:
[root@xxx ~]# uname -r
4.18.0-147.el8.x86_64
[root@SERVER ~]# mkdir /export
mkdir: cannot create directory ‘/export’: File exists
[root@SERVER ~]# echo "/export *(rw,no_root_squash)" > /etc/exports
[root@SERVER ~]# systemctl restart nfs-server
[root@SERVER ~]# exportfs -v
/export <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash)
###############
Test in podman:
###############
[root@SERVER ~]# podman run --cap-add CAP_SYS_ADMIN -it centos:8
^^^^^^^^^^^^^^^ This option specifies that perform a range of system administration operations
[root@a179291fad10 /]# yum install -y nfs-utils &> /dev/null
[root@a179291fad10 /]# showmount -e $SERVER
Export list for $SERVER:
/export *
[root@a179291fad10 /]# mount $SERVER:/export /mnt -vvv
mount.nfs: timeout set for Thu Oct 24 02:59:30 2019
mount.nfs: trying text-based options 'vers=4.2,addr=10.73.4.209,clientaddr=10.88.0.5'
mount.nfs: mount(2): No such device <<<<<<<<<<<<<<<<< fail
mount.nfs: No such device
(If this is your expectation, do you need to return a clear prompt so that the customer can easily understand the cause of the error)
###############
Test in docker:
###############
[root@SERVER ~]# docker run --cap-add SYS_ADMIN -it centos:8
^^^^^^^^^
[root@0d85c14ee55c /]# showmount -e $SERVER
Export list for $SERVER:
/export *
[root@0d85c14ee55c /]# mount $SERVER:/export /mnt -vvv <<<<<<<<<<< pass
mount.nfs: timeout set for Thu Oct 24 03:18:43 2019
mount.nfs: trying text-based options 'vers=4.2,addr=10.73.4.169,clientaddr=172.17.0.2'
[root@0d85c14ee55c /]# df -h | grep /mnt
Filesystem 1K-blocks Used Available Use% Mounted on
$SERVER:/export 52399104 4132864 48266240 8% /mnt
Additional info:
[1] http://man7.org/linux/man-pages/man7/capabilities.7.html> CAP_SYS_ADMIN
Note: this capability is overloaded; see Notes to kernel
developers, below.
* Perform a range of system administration operations
including: quotactl(2), mount(2), umount(2), pivot_root(2),
setdomainname(2);
...
Don't choose CAP_SYS_ADMIN if you can possibly avoid it! A vast
proportion of existing capability checks are associated with this
capability (see the partial list above). It can plausibly be
called "the new root", since on the one hand, it confers a wide
range of powers, and on the other hand, its broad scope means that
this is the capability that is required by many privileged
programs.
If you run the container as privileged (`--privileged` instead of `--cap-add CAP_SYS_ADMIN`), does it work? Alternatively, try disabling SELinux (`sudo setenforce 0`) before running the container.
(In reply to Matthew Heon from comment #1)
> If you run the container as privileged (`--privileged` instead of `--cap-add
> CAP_SYS_ADMIN`), does it work? Alternatively, try disabling SELinux (`sudo
> setenforce 0`) before running the container.
Yes, It works, but whether it is a mistake of podman runtime?
Well it would be good to know what device nfs mount is complaining about.
Any chance you could strace the mount command to see if it gives you more information.
(In reply to Daniel Walsh from comment #6)
> Any chance this is just an issue of the network not being setup correctly in
> Podman. Can you ping the NFS Server?
Yes, I can
(In reply to Matthew Heon from comment #7)
> Which of the two things I suggested fixed this - `--privileged` or
> `setenforce 0`?
Up to your strategies, we just focus on the NFS related operations.
Zhi, we are trying to figure out if this is an SELinux issue or some other issue.
By setenforce 0, then we can see if SELinux is blocking the mount, If SELinux is disabled
or setenforce 0 does not fix the issue, then we have to look elsewhere.
@zhi Please gather the AVC's
ausearch -m avc -ts recent
This might not be something that we want to fix in container selinux, but I need the AVCs to be sure.
podman run --security-opt label=disabled --cap-add CAP_SYS_ADMIN -it centos:8
Should turn off SELinux separation for the container.
THere is a boolean that allows this
setsebool -P domain_kernel_load_modules 1
But I would prefer that you did not do this.
A better solution would be to have the system just load the kernel module.
Can you get the system to load the kernel module on boot?
Something like:
echo nfs > /etc/modules-load.d/nfs
Description of problem: When runing podman container that specifies --cap-add CAP_SYS_ADMIN(This option specifies that perform a range of system administration operations [1]) and performing nfs mount in it, but the result fails. which is a different behavior from docker that specifiles "--cap-add SYS_ADMIN" . ( If this behavior is your expected.) Version-Release number of selected component (if applicable): kernel-4.18.0-147.el8.x86_64 Steps to Reproduce: [root@xxx ~]# uname -r 4.18.0-147.el8.x86_64 [root@SERVER ~]# mkdir /export mkdir: cannot create directory ‘/export’: File exists [root@SERVER ~]# echo "/export *(rw,no_root_squash)" > /etc/exports [root@SERVER ~]# systemctl restart nfs-server [root@SERVER ~]# exportfs -v /export <world>(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) ############### Test in podman: ############### [root@SERVER ~]# podman run --cap-add CAP_SYS_ADMIN -it centos:8 ^^^^^^^^^^^^^^^ This option specifies that perform a range of system administration operations [root@a179291fad10 /]# yum install -y nfs-utils &> /dev/null [root@a179291fad10 /]# showmount -e $SERVER Export list for $SERVER: /export * [root@a179291fad10 /]# mount $SERVER:/export /mnt -vvv mount.nfs: timeout set for Thu Oct 24 02:59:30 2019 mount.nfs: trying text-based options 'vers=4.2,addr=10.73.4.209,clientaddr=10.88.0.5' mount.nfs: mount(2): No such device <<<<<<<<<<<<<<<<< fail mount.nfs: No such device (If this is your expectation, do you need to return a clear prompt so that the customer can easily understand the cause of the error) ############### Test in docker: ############### [root@SERVER ~]# docker run --cap-add SYS_ADMIN -it centos:8 ^^^^^^^^^ [root@0d85c14ee55c /]# showmount -e $SERVER Export list for $SERVER: /export * [root@0d85c14ee55c /]# mount $SERVER:/export /mnt -vvv <<<<<<<<<<< pass mount.nfs: timeout set for Thu Oct 24 03:18:43 2019 mount.nfs: trying text-based options 'vers=4.2,addr=10.73.4.169,clientaddr=172.17.0.2' [root@0d85c14ee55c /]# df -h | grep /mnt Filesystem 1K-blocks Used Available Use% Mounted on $SERVER:/export 52399104 4132864 48266240 8% /mnt Additional info: [1] http://man7.org/linux/man-pages/man7/capabilities.7.html > CAP_SYS_ADMIN Note: this capability is overloaded; see Notes to kernel developers, below. * Perform a range of system administration operations including: quotactl(2), mount(2), umount(2), pivot_root(2), setdomainname(2); ... Don't choose CAP_SYS_ADMIN if you can possibly avoid it! A vast proportion of existing capability checks are associated with this capability (see the partial list above). It can plausibly be called "the new root", since on the one hand, it confers a wide range of powers, and on the other hand, its broad scope means that this is the capability that is required by many privileged programs.