1558836 – Permission denied error with Posix Ceph backend

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1558836 - Permission denied error with Posix Ceph backend

Summary: Permission denied error with Posix Ceph backend

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	selinux-policy
Sub Component:
Version:	7.6
Hardware:	x86_64
OS:	Linux
Priority:	urgent
Severity:	urgent
Target Milestone:	pre-dev-freeze
Target Release:	7.7
Assignee:	Lukas Vrabec
QA Contact:	Milos Malik
Docs Contact:
URL:
Whiteboard:
Duplicates (2):	1685799 1724018 (view as bug list)
Depends On:
Blocks:	1653106 1672178 1753535
TreeView+	depends on / blocked

Reported:	2018-03-21 05:58 UTC by 曾浩
Modified:	2019-10-29 05:52 UTC (History)
CC List:	19 users (show)
Fixed In Version:	selinux-policy-3.13.1-235.el7
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Clones:	1753535 (view as bug list)
Environment:
Last Closed:	2019-08-06 12:51:45 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
ovirt (3.95 KB, text/plain) 2018-03-21 05:58 UTC, 曾浩	no flags	Details
View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2019:2127	0	None	None	None	2019-08-06 12:52:06 UTC

Description 曾浩 2018-03-21 05:58:40 UTC

Created attachment 1410959 [details]
ovirt

Description of problem:

vm is started error!

Version-Release number of selected component (if applicable):

ovirt-node-ng-installer-ovirt-4.2-2018031106

How reproducible:


Steps to Reproduce:
1. storage --> domain --> create domain -->data master（posix Cephfs（one address））, ISO and Export use NFS
2. create vm --> instance images（20gb）--> boot options --> CDROM(attach iso)
3. start vm

Actual results:
vm test down with error,<path> Permission denied.
1. login the ovirt node with ssh，and use command ls -l,
[root@hptestenv70 vdsm]# ls -l /rhev/data-center/mnt/10.148.181.227:6789:_hptestvm/6c13c677-43a9-4396-b0b5-26f0ed380334/images/b149218c-9501-4ae8-9592-76e842a3cb9e/
总用量 20972545
-rw-rw---- 1 vdsm kvm 21474836480 3月  21 13:29 c6722078-9a19-4614-8122-524c28abf9af
-rw-rw---- 1 vdsm kvm     1048576 3月  21 13:29 c6722078-9a19-4614-8122-524c28abf9af.lease
-rw-r--r-- 1 vdsm kvm         313 3月  21 13:29 c6722078-9a19-4614-8122-524c28abf9af.meta



Expected results:


Additional info:

Comment 1 Fred Rolland 2018-05-29 08:35:27 UTC

I reproduced the issue on my env.
It is a SELinux issue.
If I disable SELinux on the host running the VM, the VM starts successfully.

Comment 2 Fred Rolland 2018-05-29 08:45:11 UTC

ausearch -m AVC,USER_AVC -ts recent

type=PROCTITLE msg=audit(1527583258.543:92781): proctitle=2F7573722F6C6962657865632F71656D752D6B766D002D6E616D650067756573743D434550484653564D2C64656275672D746872656164733D6F6E002D53002D6F626A656374007365637265742C69643D6D61737465724B6579302C666F726D61743D7261772C66696C653D2F7661722F6C69622F6C6962766972742F71656D
type=SYSCALL msg=audit(1527583258.543:92781): arch=c000003e syscall=2 success=no exit=-13 a0=55a5c41931e0 a1=80800 a2=0 a3=fffffffffffff498 items=0 ppid=1 pid=11716 auid=4294967295 uid=107 gid=107 euid=107 suid=107 fsuid=107 egid=107 sgid=107 fsgid=107 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/libexec/qemu-kvm" subj=system_u:system_r:svirt_t:s0:c440,c805 key=(null)
type=AVC msg=audit(1527583258.543:92781): avc:  denied  { read } for  pid=11716 comm="qemu-kvm" name="36b34034-5b5b-454d-a117-0f71129c9493" dev="ceph" ino=1099511627789 scontext=system_u:system_r:svirt_t:s0:c440,c805 tcontext=system_u:object_r:cephfs_t:s0 tclass=file
----
time->Tue May 29 11:40:58 2018
type=PROCTITLE msg=audit(1527583258.543:92782): proctitle=2F7573722F6C6962657865632F71656D752D6B766D002D6E616D650067756573743D434550484653564D2C64656275672D746872656164733D6F6E002D53002D6F626A656374007365637265742C69643D6D61737465724B6579302C666F726D61743D7261772C66696C653D2F7661722F6C69622F6C6962766972742F71656D
type=SYSCALL msg=audit(1527583258.543:92782): arch=c000003e syscall=4 success=no exit=-13 a0=55a5c41931e0 a1=7ffd1106f400 a2=7ffd1106f400 a3=fffffffffffff498 items=0 ppid=1 pid=11716 auid=4294967295 uid=107 gid=107 euid=107 suid=107 fsuid=107 egid=107 sgid=107 fsgid=107 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/libexec/qemu-kvm" subj=system_u:system_r:svirt_t:s0:c440,c805 key=(null)
type=AVC msg=audit(1527583258.543:92782): avc:  denied  { getattr } for  pid=11716 comm="qemu-kvm" path="/rhev/data-center/mnt/10.35.1.36:6789:_ovirt/56c92d21-d086-4692-9835-29edb47ee522/images/3b95e159-d4a3-4491-91dc-3bfdc1e5f1f4/36b34034-5b5b-454d-a117-0f71129c9493" dev="ceph" ino=1099511627789 scontext=system_u:system_r:svirt_t:s0:c440,c805 tcontext=system_u:object_r:cephfs_t:s0 tclass=file
----
time->Tue May 29 11:40:58 2018
type=PROCTITLE msg=audit(1527583258.543:92783): proctitle=2F7573722F6C6962657865632F71656D752D6B766D002D6E616D650067756573743D434550484653564D2C64656275672D746872656164733D6F6E002D53002D6F626A656374007365637265742C69643D6D61737465724B6579302C666F726D61743D7261772C66696C653D2F7661722F6C69622F6C6962766972742F71656D
type=SYSCALL msg=audit(1527583258.543:92783): arch=c000003e syscall=2 success=no exit=-13 a0=55a5c41931e0 a1=84002 a2=0 a3=0 items=0 ppid=1 pid=11716 auid=4294967295 uid=107 gid=107 euid=107 suid=107 fsuid=107 egid=107 sgid=107 fsgid=107 tty=(none) ses=4294967295 comm="qemu-kvm" exe="/usr/libexec/qemu-kvm" subj=system_u:system_r:svirt_t:s0:c440,c805 key=(null)
type=AVC msg=audit(1527583258.543:92783): avc:  denied  { read write } for  pid=11716 comm="qemu-kvm" name="36b34034-5b5b-454d-a117-0f71129c9493" dev="ceph" ino=1099511627789 scontext=system_u:system_r:svirt_t:s0:c440,c805 tcontext=system_u:object_r:cephfs_t:s0 tclass=file

Comment 3 Fred Rolland 2018-05-29 08:45:35 UTC

journalctl

May 29 11:40:58 vdsm42 systemd[1]: Started Virtual Machine qemu-5-CEPHFSVM.
May 29 11:40:58 vdsm42 systemd-machined[13090]: New machine qemu-5-CEPHFSVM.
May 29 11:40:58 vdsm42 systemd[1]: Starting Virtual Machine qemu-5-CEPHFSVM.
May 29 11:40:58 vdsm42 kvm[11721]: 1 guest now active
May 29 11:40:58 vdsm42 libvirtd[18910]: 2018-05-29 08:40:58.556+0000: 18910: error : qemuMonitorIORead:588 : Unable to read from monitor: Connection reset by peer
May 29 11:40:58 vdsm42 libvirtd[18910]: 2018-05-29 08:40:58.556+0000: 18910: error : qemuProcessReportLogError:1862 : internal error: qemu unexpectedly closed the monitor: 2018-05-29T08:40:58.544994Z qemu-kvm: -
May 29 11:40:58 vdsm42 kvm[11723]: 0 guests now active
May 29 11:40:58 vdsm42 systemd-machined[13090]: Machine qemu-5-CEPHFSVM terminated.
May 29 11:40:58 vdsm42 libvirtd[18910]: 2018-05-29 08:40:58.757+0000: 18913: error : qemuProcessReportLogError:1862 : internal error: process exited while connecting to monitor: 2018-05-29T08:40:58.544994Z qemu-
May 29 11:40:58 vdsm42 vdsm[19103]: WARN File: /var/lib/libvirt/qemu/channels/3738a61b-8a75-42e2-be00-7d0c9fea50c0.ovirt-guest-agent.0 already removed
May 29 11:40:58 vdsm42 vdsm[19103]: WARN File: /var/lib/libvirt/qemu/channels/3738a61b-8a75-42e2-be00-7d0c9fea50c0.org.qemu.guest_agent.0 already removed

Comment 4 Fred Rolland 2018-05-29 13:10:37 UTC

# cat /etc/redhat-release 
CentOS Linux release 7.5.1804 (Core) 
# rpm -qa | grep selin
libselinux-2.5-12.el7.i686
libselinux-2.5-12.el7.x86_64
selinux-policy-targeted-3.13.1-192.el7_5.3.noarch
libselinux-python-2.5-12.el7.x86_64
libselinux-utils-2.5-12.el7.x86_64
selinux-policy-3.13.1-192.el7_5.3.noarch

Comment 5 Fred Rolland 2018-05-29 14:14:06 UTC

Lukas,
Can you take a look at this issue?

You handled something similar here:
https://bugzilla.redhat.com/show_bug.cgi?id=1315332

Thanks

Comment 6 Fred Rolland 2018-06-06 13:06:27 UTC

Lukas, any updates?

Comment 7 Lukas Vrabec 2018-06-06 13:22:52 UTC

Hi, 

It's bug int SELinux policy, will fix it.

Comment 17 Lukas Vrabec 2019-03-06 12:09:22 UTC

*** Bug 1685799 has been marked as a duplicate of this bug. ***

Comment 23 Milos Malik 2019-04-29 12:19:54 UTC

# rpm -qa selinux\* | sort
selinux-policy-3.13.1-244.el7.noarch
selinux-policy-devel-3.13.1-244.el7.noarch
selinux-policy-targeted-3.13.1-244.el7.noarch
# sesearch -s svirt_t -t cephfs_t -A
Found 6 semantic av rules:
   allow virt_domain file_type : dir { getattr search open } ; 
   allow virt_domain cephfs_t : dir { getattr search open } ; 
   allow virt_domain cephfs_t : file { ioctl read write getattr lock append open } ; 
   allow domain file_type : file map ; 
   allow domain file_type : chr_file map ; 
   allow domain file_type : blk_file map ; 

#

There is no allow rule for lnk_file. SELinux denial which affects the lnk_file class is not recorded in this bug, but I believe that such allow rule should be also added to policy.

Comment 24 Milos Malik 2019-04-29 12:21:51 UTC

If following rule is not present, then svirt_t processes cannot read symbolic links stored on CephFS and that could be a problem for users.

Comment 29 Konstantin Shalygin 2019-07-16 10:15:16 UTC

When selinux-policy-3.13.1-235.el7 will be released? oVirt still have issue even with ISO domains.

Comment 30 Zdenek Pytela 2019-07-16 14:27:24 UTC

A fix for this bugzilla should be a part of RHEL 7.7 which is expected to go general availability later this quarter.

Comment 31 Fred Rolland 2019-07-17 09:04:03 UTC

*** Bug 1724018 has been marked as a duplicate of this bug. ***

Comment 33 errata-xmlrpc 2019-08-06 12:51:45 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:2127

Comment 34 Konstantin Shalygin 2019-10-29 05:52:27 UTC

Works for me at el 7.7, thanks Zdenek.

Note You need to log in before you can comment on or make changes to this bug.