Description of problem: The once working changes to support virtio-fs described in "Red Hat Bugzilla – Bug 1899703" seem once again to be failing. Problems with lsattr, dnf/yum/rpm broken (cpio fails with ... Error unpacking rpm package dnf-4.7.0-1.fc34.noarch Upgrading : python3-dnf-plugins-core-4.0.21-1.fc34.noarch 8/20 error: unpacking of archive failed on file /usr/bin/dnf;60b1b277: cpio: (error 0x2) error: dnf-4.7.0-1.fc34.noarch: install failed error: lsetfilecon: (/etc/dnf/plugins/copr.conf, system_u:object_r:etc_t:s0) Operation not permitted error: Plugin selinux: hook fsm_file_prepare failed ... Version-Release number of selected component (if applicable): guest: fc34 5.11.16-300.fc34.x86_64 Host: # /usr/lib/qemu/virtiofsd -V using FUSE kernel interface version 7.32 uname 5.11.0-17-generic How reproducible: Always Steps to Reproduce: 1. touch ~/not-again 2 lsattr ~/not-again Actual results: lsattr: Function not implemented While reading flags on oops-again Expected results: A list of attributes. Additional info: dpkg / rpm / yum broken too, with on the guest, and May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Queue 1 gave evalue: 1 available: in: 40 out: 68 May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Waiting for Queue 1 event May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000478] fv_queue_worker: elem 3: with 2 out desc of length 68 May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000478] unique: 1405570, opcode: GETXATTR (22), nodeid: 10277, insize: 68, pid: 2386 May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000478] lo_getxattr(ino=10277, name=security.capability size=24) May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000478] unique: 1405570, error: -61 (No data available), outsize: 16 May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000478] virtio_send_msg: elem 3: with 2 in desc of length 40 May 29 15:12:36 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 Guest fstab: myfs / virtiofs defaults 0 0 Host FS is btrfs with xattr /usr/lib/qemu/virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock,log_level=debug -d --syslog
(In reply to Harry Coin from comment #0) > Description of problem: > The once working changes to support virtio-fs described in "Red Hat Bugzilla > – Bug 1899703" seem once again to be failing. Problems with lsattr, > dnf/yum/rpm broken (cpio fails with > > ... > Error unpacking rpm package dnf-4.7.0-1.fc34.noarch > Upgrading : python3-dnf-plugins-core-4.0.21-1.fc34.noarch > 8/20 > error: unpacking of archive failed on file /usr/bin/dnf;60b1b277: cpio: > (error 0x2) > error: dnf-4.7.0-1.fc34.noarch: install failed > error: lsetfilecon: (/etc/dnf/plugins/copr.conf, system_u:object_r:etc_t:s0) > Operation not permitted > error: Plugin selinux: hook fsm_file_prepare failed > > ... > > Version-Release number of selected component (if applicable): > guest: fc34 5.11.16-300.fc34.x86_64 > Host: > # /usr/lib/qemu/virtiofsd -V > using FUSE kernel interface version 7.32 > > uname 5.11.0-17-generic Are there any SELinux denials (`ausearch -m avc`) on the guest? Is it possible that SELinux is enabled on the host? Based on the uname I guess it's Debian or Ubuntu, but I've heard of people trying to get SELinux working on those as well... If not SELinux, it may be something else on the host that prevents the daemon to set the security.selinux extended attribute on the backing files - perhaps it is denied by AppArmor or the process is missing some capability? Also, you mention that this was working before - was it on the same system? Can you identify what software component introduced the breakage (and the last working version)? > How reproducible: > Always > > Steps to Reproduce: > 1. touch ~/not-again > 2 lsattr ~/not-again > > > Actual results: > > lsattr: Function not implemented While reading flags on oops-again lsattr(1) is for getting e2fs-specific attributes, not extended attributes. I think you meant to use getfattr(1).
The guest had (before the breakage) and has selinux booting in permissive mode. So there are no denials. The host had and has btrfs, no selinux installed (ubuntu), with xattr on.
How to repeat/test virtiofs lsetfilecon regression that breaks dpkg, prevents upgrades/fixes. [root@registry1 ~]# getenforce Permissive [root@registry1 ~]# cat lsetfilecon.c #include <selinux/selinux.h> #include <stdio.h> #include <errno.h> void perror(const char *s); int main(int argc,char *argv[]){ int i; i= lsetfilecon("/usr/bin/rngtest","system_u:object_r:bin_t:s0"); //i= lsetfilecon("/usr/bin/rngtest;60b9120b","system_u:object_r:bin_t:s0"); printf("ret %lx\n",i); perror("\n"); return 0; } [root@registry1 ~]# gcc lsetfilecon.c -lselinux -o lsetfilecon [root@registry1 ~]# ./lsetfilecon ret ffffffff : Operation not permitted [root@registry1 ~]# ls -l /usr/bin/rngtest -rwxr-xr-x. 1 root root 21176 Apr 27 18:26 /usr/bin/rngtest [root@registry1 ~]# uname -a Linux registry1.xxxx 5.11.19-300.fc34.x86_64 #1 SMP Fri May 7 14:17:15 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux On the host: root@noc1:/vmsystems/registry1/usr/bin# getfattr -m - -d rngtest # file: rngtest security.selinux="system_u:object_r:bin_t:s0" ls -l rngtest -rwxr-xr-x. 1 root root 21176 Apr 27 18:26 rngtest #/usr/lib/qemu/virtiofsd --version using FUSE kernel interface version 7.32 #uname -a 5.11.0-18-generic #19-Ubuntu SMP Fri May 7 14:22:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
thanks for the reproducer. Ran it quickly on a file on virtiofs and I can see error. Error seems to be coming from guest kernel (and not host kernel). Will need to debug little more to figure out which component is returning the error and why.
Thanks, I should have written 'dnf/yum' instead of 'dpkg'. It's Friday and I've been juggling so many host/guest distro combinations I lost track of to whom I was writing about what.
After booting with /.autorelabel, even with in the host 'xattrmap=:map::user.virtiofs.:', all the files that are soft links in the guest get listed in the boot log like selinux-autorelabel[834]: /bin/setfiles: Could not set context for <filename>: Operation not permitted. However, the relabeling succeeds for other than softlinks. For example, in the host for what would be /var in the guest, adding the xattr map and the autorelabel leads to: # file: var security.selinux user.virtiofs.security.selinux
(In reply to Vivek Goyal from comment #4) > thanks for the reproducer. Ran it quickly on a file on virtiofs and I can > see error. Error seems to be coming from guest kernel (and not host kernel). > Will need to debug little more to figure out which component is returning > the error and why. Ok, following is returning -EOPNOTSUPP in selinux_inode_setxattr() (security/selinux/hooks.c). sbsec = selinux_superblock(inode->i_sb); if (!(sbsec->flags & SBLABEL_MNT)) { return -EOPNOTSUPP; } I am wondering what's this flag SBLABEL_MNT and who is supposed to set it. Ondrej, would you know.
(In reply to Harry Coin from comment #6) > After booting with /.autorelabel, even with in the host > 'xattrmap=:map::user.virtiofs.:', all the files that are soft links in the > guest get listed in the boot log like > selinux-autorelabel[834]: /bin/setfiles: Could not set context for > <filename>: Operation not permitted. > However, the relabeling succeeds for other than softlinks. > > For example, in the host for what would be /var in the guest, adding the > xattr map and the autorelabel leads to: > > # file: var > security.selinux > user.virtiofs.security.selinux Hmm..., so it failed with soft links. Adding david gilbert. He wrote this code. He might have an idea. BTW, this issue of xattr remapping and softlink, we should probably discuss in a separate email thread. Otherwise these two issues will create lot of confusion in same bug.
This commit added the code. Still not sure what does flag mean. commit 53e0c2aa9a59a48e3798ef193d573ade85aa80f5 Author: Ondrej Mosnacek <omosnace> Date: Fri Dec 21 21:18:53 2018 +0100 selinux: do not override context on context mounts
DejaVu? https://lore.kernel.org/patchwork/patch/774906/
NFS related history: From: "J. Bruce Fields" <***@redhat.com> Before calling into the filesystem, vfs_setxattr calls security_inode_setxattr, which ends up calling selinux_inode_setxattr in our case. That returns -EOPNOTSUPP whenever SBLABEL_MNT is not set. SBLABEL_MNT was supposed to be set by sb_finish_set_opts, which sets it only if selinux_is_sblabel_mnt returns true. The selinux_is_sblabel_mnt logic was broken by eadcabc697e9 "SELinux: do all flags twiddling in one place", which didn't take into the account the SECURITY_FS_USE_NATIVE behavior that had been introduced for nfs with eb9ae686507b "SELinux: Add new labeling type native labels". This caused setxattr's of security labels over NFSv4.2 to fail.
Frankly speaking I don't understand all this magic SELinux flags and what are different options and what' the right thing to do for virtiofs. I am hoping Ondrej and other SELinux folks can help me out here with what's the right fix here.
Contrary to the advertising, it's not even good enough to get common programs to upgrade by turning off xattrs on the host, and set enforce to permissive on the guest. While most packages upgrade properly, 'gnome-shell' errors out on upgrade -- not being able to tell the difference between 'not supported' and 'not permitted.' However, it appears you can upgrade a kernel wedged by this bug by turning off xattrs, enough to load the next kernel when it happens this bug is fixed someday. As follows: /etc/fstab: myfs / virtiofs context="system_u:object_r:fs_t" 0 0 then in the host, turn off xattrs and posix on. eg: /usr/lib/qemu/virtiofsd --fd=49 -o source=/vmsystems/fedora_generic,no_xattr,flock,posix_lock This worked in my lab upgrading a few hundred files with stock kernel 5.11.16-300.fc34.x86_64 in the guest. xattrs are still entirely broken, but at least this ugly hack will allow dnf/yum upgrade to install the fix and other packages when available. gnome-shell bug is a longstanding one: https://www.reddit.com/r/Fedora/comments/j61mmi/gnome_transaction_errors/
(In reply to Vivek Goyal from comment #7) > Ok, following is returning -EOPNOTSUPP in selinux_inode_setxattr() > (security/selinux/hooks.c). > sbsec = selinux_superblock(inode->i_sb); > if (!(sbsec->flags & SBLABEL_MNT)) { > return -EOPNOTSUPP; > } > > I am wondering what's this flag SBLABEL_MNT and who is supposed to set it. That's strange, that flag should be set in sb_finish_set_opts() when he SECURITY_FS_USE_XATTR behavior is selected. What does `seinfo --genfs | grep virtiofs` and `seinfo --fs_use | grep virtiofs` show (on the guest)? When you list files/dirs inside the mount (ls -lZ), does it show the labels from the host?
(In reply to Ondrej Mosnacek from comment #14) > > That's strange, that flag should be set in sb_finish_set_opts() when he > SECURITY_FS_USE_XATTR behavior is selected. What does `seinfo --genfs | grep > virtiofs` and `seinfo --fs_use | grep virtiofs` show (on the guest)? I am running a Fedora 32 guest. I have updated it to latest. When I run above two commands, I don't see anything in there for virtiofs. $ seinfo --fs_use | grep virtiofs $ seinfo --genfs | grep virtiofs > When > you list files/dirs inside the mount (ls -lZ), does it show the labels from > the host? On host I labeled foo.txt as `system_u:object_r:etc_t:s0` $ ls -Z foo.txt system_u:object_r:etc_t:s0 foo.txt But inside guest it shows as unlabeled_t. ls -Z foo.txt system_u:object_r:unlabeled_t:s0 foo.txt So does not look like selinux recognizes that filesystem supports xattr.
(In reply to Vivek Goyal from comment #15) > I am running a Fedora 32 guest. I have updated it to latest. That's too old (and EOL now). I believe the selinux-policy support for virtiofs landed in F34. > When I run above two commands, I don't see anything in there for virtiofs. > > $ seinfo --fs_use | grep virtiofs > $ seinfo --genfs | grep virtiofs > > > When > > you list files/dirs inside the mount (ls -lZ), does it show the labels from > > the host? > > On host I labeled foo.txt as `system_u:object_r:etc_t:s0` > > $ ls -Z foo.txt > system_u:object_r:etc_t:s0 foo.txt > > But inside guest it shows as unlabeled_t. > > ls -Z foo.txt > system_u:object_r:unlabeled_t:s0 foo.txt > > So does not look like selinux recognizes that filesystem supports xattr. That all matches what I would expect to see on F32. Can you try it with F34 as the guest?
(In reply to Ondrej Mosnacek from comment #16) > (In reply to Vivek Goyal from comment #15) > > I am running a Fedora 32 guest. I have updated it to latest. > > That's too old (and EOL now). I believe the selinux-policy support for > virtiofs landed in F34. > > > When I run above two commands, I don't see anything in there for virtiofs. > > > > $ seinfo --fs_use | grep virtiofs > > $ seinfo --genfs | grep virtiofs > > > > > When > > > you list files/dirs inside the mount (ls -lZ), does it show the labels from > > > the host? > > > > On host I labeled foo.txt as `system_u:object_r:etc_t:s0` > > > > $ ls -Z foo.txt > > system_u:object_r:etc_t:s0 foo.txt > > > > But inside guest it shows as unlabeled_t. > > > > ls -Z foo.txt > > system_u:object_r:unlabeled_t:s0 foo.txt > > > > So does not look like selinux recognizes that filesystem supports xattr. > > That all matches what I would expect to see on F32. Can you try it with F34 > as the guest? I tried on F34 and things look good there. I can see the labels as set on host inside guest. And lsetfilecon succeeds too. # seinfo --genfs | grep virtiofs genfscon virtiofs / system_u:object_r:virtiofs_t:s0 # seinfo --fs_use | grep virtiofs fs_use_xattr virtiofs system_u:object_r:fs_t:s0; # ls -Z /mnt/virtiofs/foo.txt system_u:object_r:bin_t:s0 /mnt/virtiofs/foo.txt # ./lsetfilecon /mnt/virtiofs/foo.txt "system_u:object_r:etc_t:s0" lsetfilecon(/mnt/virtiofs/foo.txt, system_u:object_r:etc_t:s0) succeeded # ls -Z /mnt/virtiofs/foo.txt system_u:object_r:etc_t:s0 /mnt/virtiofs/foo.txt
Harry, so with fedora 34, I don't see the problem of lsetfilecon. I am running kernel 5.12.9-300.fc34.x86_64 Can you update your guest to latest and see if problem is still there.
(In reply to Ondrej Mosnacek from comment #16) > (In reply to Vivek Goyal from comment #15) > > I am running a Fedora 32 guest. I have updated it to latest. > > That's too old (and EOL now). I believe the selinux-policy support for > virtiofs landed in F34. > > > That all matches what I would expect to see on F32. Can you try it with F34 > as the guest? Tried with F33 as well. It is same as F32 and looks like virtiofs specific selinux changes are not there.
(In reply to Vivek Goyal from comment #19) > > Tried with F33 as well. It is same as F32 and looks like virtiofs specific > selinux changes are not there. I think that's the reason that if I am using virtiofs as rootfs for F33, I can't login if selinux is enabled and enforced. I have to either disable SELinux or boot with enforcing=0 to be able to login into the guest. I see following avc on F33. type=AVC msg=audit(1623273429.833:119): avc: denied { transition } for pid=673 comm="login" path="/usr/bin/bash" dev="virtiofs" ino=3409080 scontext=system_u:system_r:kernel_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0 tclass=process permissive=0
Even after upgrading to F34, I can't login into the guest (virtiofs root) with selinux in enforced. With enforced=0, I see following avc. type=AVC msg=audit(1623274846.031:175): avc: denied { dyntransition } for pid=14467 comm="sshd" scontext=system_u:system_r:kernel_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0 tclass=process permissive=1
No joy. Same error. Did you try the reproducer? [root@fedora ~]# mount | grep myfs myfs on / type virtiofs (rw,relatime,seclabel) [root@fedora ~]# ls -lZ /usr/bin/rngtest -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 21176 May 24 12:27 /usr/bin/rngtest [root@fedora ~]# uname -a Linux fedora.1.quietfountain.com 5.12.9-300.fc34.x86_64 #1 SMP Thu Jun 3 13:51:40 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux [root@fedora ~]# cat lsetfilecon.c #include <selinux/selinux.h> #include <stdio.h> #include <errno.h> void perror(const char *s); int main(int argc,char *argv[]){ int i; i= lsetfilecon("/usr/bin/rngtest","system_u:object_r:bin_t:s0"); //i= lsetfilecon("/usr/bin/rngtest;60b9120b","system_u:object_r:bin_t:s0"); printf("ret %lx\n",i); perror("\n"); return 0; } [root@fedora ~]# gcc lsetfilecon.c -lselinux -o lsetfilecon [root@fedora ~]# ./lsetfilecon ret ffffffff : Operation not permitted [root@fedora ~]# [root@fedora ~]# strace ./lsetfilecon execve("./lsetfilecon", ["./lsetfilecon"], 0x7ffc629b61c0 /* 29 vars */) = 0 brk(NULL) = 0xee3000 arch_prctl(0x3001 /* ARCH_??? */, 0x7ffcfca42880) = -1 EINVAL (Invalid argument) access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 newfstatat(3, "", {st_mode=S_IFREG|0644, st_size=61688, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 61688, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fd887b5c000 close(3) = 0 openat(AT_FDCWD, "/lib64/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20p\0\0\0\0\0\0"..., 832) = 832 newfstatat(3, "", {st_mode=S_IFREG|0755, st_size=171376, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd887b5a000 mmap(NULL, 177640, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd887b2e000 mprotect(0x7fd887b34000, 139264, PROT_NONE) = 0 mmap(0x7fd887b34000, 106496, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0x7fd887b34000 mmap(0x7fd887b4e000, 28672, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x20000) = 0x7fd887b4e000 mmap(0x7fd887b56000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x27000) = 0x7fd887b56000 mmap(0x7fd887b58000, 5608, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd887b58000 close(3) = 0 openat(AT_FDCWD, "/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260|\2\0\0\0\0\0"..., 832) = 832 pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784 pread64(3, "\4\0\0\0 \0\0\0\5\0\0\0GNU\0\2\0\0\300\4\0\0\0\3\0\0\0\0\0\0\0"..., 48, 848) = 48 pread64(3, "\4\0\0\0\24\0\0\0\3\0\0\0GNU\0\16\274\275\24T\23lD\202\301\6\336\nd\341&"..., 68, 896) = 68 newfstatat(3, "", {st_mode=S_IFREG|0755, st_size=1913536, ...}, AT_EMPTY_PATH) = 0 pread64(3, "\6\0\0\0\4\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0@\0\0\0\0\0\0\0"..., 784, 64) = 784 mmap(NULL, 1892824, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd88795f000 mprotect(0x7fd887985000, 1679360, PROT_NONE) = 0 mmap(0x7fd887985000, 1363968, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x26000) = 0x7fd887985000 mmap(0x7fd887ad2000, 311296, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x173000) = 0x7fd887ad2000 mmap(0x7fd887b1f000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bf000) = 0x7fd887b1f000 mmap(0x7fd887b25000, 33240, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd887b25000 close(3) = 0 openat(AT_FDCWD, "/lib64/libpcre2-8.so.0", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260$\0\0\0\0\0\0"..., 832) = 832 newfstatat(3, "", {st_mode=S_IFREG|0755, st_size=618856, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 614960, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd8878c8000 mprotect(0x7fd8878ca000, 602112, PROT_NONE) = 0 mmap(0x7fd8878ca000, 438272, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fd8878ca000 mmap(0x7fd887935000, 159744, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6d000) = 0x7fd887935000 mmap(0x7fd88795d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x94000) = 0x7fd88795d000 close(3) = 0 openat(AT_FDCWD, "/lib64/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\220\"\0\0\0\0\0\0"..., 832) = 832 newfstatat(3, "", {st_mode=S_IFREG|0755, st_size=25136, ...}, AT_EMPTY_PATH) = 0 mmap(NULL, 24688, PROT_READ, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fd8878c1000 mmap(0x7fd8878c3000, 8192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7fd8878c3000 mmap(0x7fd8878c5000, 4096, PROT_READ, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 0x7fd8878c5000 mmap(0x7fd8878c6000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4000) = 0x7fd8878c6000 mmap(0x7fd8878c7000, 112, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fd8878c7000 close(3) = 0 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fd8878bf000 arch_prctl(ARCH_SET_FS, 0x7fd8878bfc40) = 0 mprotect(0x7fd887b1f000, 12288, PROT_READ) = 0 mprotect(0x7fd8878c6000, 4096, PROT_READ) = 0 mprotect(0x7fd88795d000, 4096, PROT_READ) = 0 mprotect(0x7fd887b56000, 4096, PROT_READ) = 0 mprotect(0x403000, 4096, PROT_READ) = 0 mprotect(0x7fd887b9c000, 8192, PROT_READ) = 0 munmap(0x7fd887b5c000, 61688) = 0 statfs("/sys/fs/selinux", {f_type=SELINUX_MAGIC, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_NOSUID|ST_NOEXEC|ST_RELATIME}) = 0 statfs("/sys/fs/selinux", {f_type=SELINUX_MAGIC, f_bsize=4096, f_blocks=0, f_bfree=0, f_bavail=0, f_files=0, f_ffree=0, f_fsid={val=[0, 0]}, f_namelen=255, f_frsize=4096, f_flags=ST_VALID|ST_NOSUID|ST_NOEXEC|ST_RELATIME}) = 0 brk(NULL) = 0xee3000 brk(0xf04000) = 0xf04000 access("/etc/selinux/config", F_OK) = 0 access("/var/run/setrans/.setrans-unix", F_OK) = -1 ENOENT (No such file or directory) lsetxattr("/usr/bin/rngtest", "security.selinux", "system_u:object_r:bin_t:s0", 27, 0) = -1 EPERM (Operation not permitted) newfstatat(1, "", {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}, AT_EMPTY_PATH) = 0 write(1, "ret ffffffff\n", 13ret ffffffff ) = 13 dup(2) = 3 fcntl(3, F_GETFL) = 0x80002 (flags O_RDWR|O_CLOEXEC) newfstatat(3, "", {st_mode=S_IFCHR|0620, st_rdev=makedev(0x88, 0), ...}, AT_EMPTY_PATH) = 0 write(3, "\n", 1 ) = 1 write(3, ": Operation not permitted\n", 26: Operation not permitted ) = 26 close(3) = 0 exit_group(0) = ? +++ exited with 0 +++ [root@fedora ~]# [root@fedora ~]# dnf upgrade gnome-shell Last metadata expiration check: 0:58:14 ago on Wed 09 Jun 2021 03:45:09 PM CDT. Dependencies resolved. ================================================================================================================================================================================================================ Package Architecture Version Repository Size ================================================================================================================================================================================================================ Upgrading: gnome-shell x86_64 40.1-1.fc34 updates 1.6 M Transaction Summary ================================================================================================================================================================================================================ Upgrade 1 Package Total download size: 1.6 M Is this ok [y/N]: y Downloading Packages: gnome-shell-40.1-1.fc34.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 1.1 MB/s | 1.6 MB 00:01 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Upgrading : gnome-shell-40.1-1.fc34.x86_64 1/2 error: lsetfilecon: (/etc/xdg/autostart/gnome-shell-overrides-migration.desktop;60c1363c, system_u:object_r:etc_t:s0) Operation not permitted error: Plugin selinux: hook fsm_file_prepare failed Error unpacking rpm package gnome-shell-40.1-1.fc34.x86_64 Verifying : gnome-shell-40.1-1.fc34.x86_64 1/2 Verifying : gnome-shell-40.0-4.fc34.x86_64 2/2 Failed: gnome-shell-40.0-4.fc34.x86_64 gnome-shell-40.1-1.fc34.x86_64 Error: Transaction failed [root@fedora ~]# .autorelabel on boot fails on every file, including regular files (not links or directories) with "/sbin/setfiles: Could not set context for ..." On the host: root@noc1:/vmsystems/fedora_generic# ps axu | grep fedora_generic root 2161394 0.0 0.0 9808 3364 ? S 15:50 0:00 /bin/bash /vmsystems/custom_virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock root 2161396 0.0 0.0 80068 3120 ? Sl 15:50 0:00 /usr/lib/qemu/virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock libvirt+ 2161419 134 0.6 2827692 205884 ? Sl 15:50 71:09 /usr/bin/qemu-system-x86_64 -name guest=generic-fedora-virtiofs,debug-threads=on -S -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-8-generic-fedora-virti/master-key.aes -machine pc-q35-5.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off -cpu host,migratable=off -m 2048 -overcommit mem-lock=off -smp 3,sockets=3,cores=1,threads=1 -object memory-backend-file,id=ram-node0,mem-path=/dev/hugepages/libvirt/qemu/8-generic-fedora-virti,share=yes,prealloc=yes,size=2147483648 -numa node,nodeid=0,cpus=0-2,memdev=ram-node0 -uuid 594f3e75-3b9d-4ed3-9f99-4a6e37a1925a -no-user-config -nodefaults -chardev socket,id=charmonitor,fd=50,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global ICH9-LPC.disable_s3=1 -global ICH9-LPC.disable_s4=1 -boot menu=off,strict=on -kernel /vmsystems/fedora_generic/boot/vmlinuz -initrd /vmsystems/fedora_generic/boot/initrd.img -append root=virtiofs:myfs rd.shell rd.fstab -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 -device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 -device pcie-root-port,port=0x17,chassis=8,id=pci.8,bus=pcie.0,addr=0x2.0x7 -device pcie-pci-bridge,id=pci.9,bus=pci.4,addr=0x0 -device ich9-usb-ehci1,id=usb,bus=pcie.0,addr=0x1d.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pcie.0,multifunction=on,addr=0x1d -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pcie.0,addr=0x1d.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pcie.0,addr=0x1d.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-8-generic-fedora-virti/fs0-fs.sock -device vhost-user-fs-pci,chardev=chr-vu-fs0,queue-size=1024,tag=myfs,bus=pci.1,addr=0x0 -netdev tap,fd=52,id=hostnet0,vhost=on,vhostfd=53 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:f9:60:bf,bus=pci.2,addr=0x0 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,fd=54,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0,bus=usb.0,port=1 -spice port=5906,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 -device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device i6300esb,id=watchdog0,bus=pci.9,addr=0x1 -watchdog-action reset -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 -device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 -object rng-random,id=objrng0,filename=/dev/urandom -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.6,addr=0x0 -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny -msg timestamp=on root 2161434 6.3 0.1 3213388 62020 ? Sl 15:50 3:21 /usr/lib/qemu/virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock
P.S. thats with PERMISSIVE set on the guest. You'd think SELinux wouldn't screw up basic things with permissive set.
These labels indicate that the entire boot process is screwed up. You have lots of processes running as kernel_t, which means the proper boot process where the kernel launches systemd and it transitions to init_t, did not happen. I believe that systemd is supposed to handle this but something went wrong.
(In reply to Daniel Walsh from comment #24) > These labels indicate that the entire boot process is screwed up. You have > lots of processes running as kernel_t, which means the proper boot process > where the kernel launches systemd and it transitions to init_t, did not > happen. I believe that systemd is supposed to handle this but something > went wrong. Notice the directive given here from rh re virtiofs root: https://bugzilla.redhat.com/show_bug.cgi?id=1899703
I think the boot failure can simply be explained by the whole root fs being unlabeled. See: > [root@fedora ~]# ls -lZ /usr/bin/rngtest > -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 21176 May 24 12:27 /usr/bin/rngtest This still looks like something on the host preventing the virtiofs daemon from seeing and changing the labels on the host filesystem. BTW, can you show the output of `dmesg | grep SELinux` on the guest?
(In reply to Ondrej Mosnacek from comment #26) > I think the boot failure can simply be explained by the whole root fs being > unlabeled. See: > > > [root@fedora ~]# ls -lZ /usr/bin/rngtest > > -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 21176 May 24 12:27 /usr/bin/rngtest > > This still looks like something on the host preventing the virtiofs daemon > from seeing and changing the labels on the host filesystem. BTW, can you > show the output of `dmesg | grep SELinux` on the guest? Harry, You can enabling debugging in vitiofsd, with option "-d" to see if setxattr(security.selinux) request is coming to daemon or not. Also can put selinux on host in permissive mode just to rule out errors from host.
Collected answers for Ondrej and Vivek: Ondrej 1. You mentioned 'boot failure'. In the plain meaning there is no 'boot failure'. The guest boots to a normal gui, logins normal, etc. Guest Selinux booting in permissive mode, and with xattrs on in the host. It's all selinux issues crashing dnf so preventing upgrades/installs. Guest SELinux as enforcing I know is a long way off for virtiofs as root, but now it doesn't even work in permissive mode though it did a few months ago. Ondrej 2. In answer to your questions, on the guest: [root@fedora ~]# seinfo --genfs | grep virtiofs genfscon virtiofs / system_u:object_r:virtiofs_t:s0 [root@fedora ~]# seinfo --fs_use | grep virtiofs fs_use_xattr virtiofs system_u:object_r:fs_t:s0; [root@fedora ~]# ls -lZ /root total 84 -rw-------. 1 root root system_u:object_r:admin_home_t:s0 800 Oct 8 2020 anaconda-ks.cfg -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 18813 Dec 23 23:32 dec232020.pp -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 10763 Dec 23 23:32 dec232020.te -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 foo -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 foo2 -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 foo3 -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 24784 Jun 9 16:36 lsetfilecon -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 340 Jun 4 13:48 lsetfilecon.c -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 60 Nov 29 2020 virtiofs.cil -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 64 Oct 10 2020 virtiofs.conf -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 1394 Oct 11 2020 virtiofs_kernel_boot.pp -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 445 Oct 11 2020 virtiofs_kernel_boot.te [root@fedora ~]# And on the host: root@noc1:/vmsystems/fedora_generic# ls -lZ root total 84 -rw-------. 1 root root system_u:object_r:admin_home_t:s0 800 Oct 8 2020 anaconda-ks.cfg -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 18813 Dec 23 23:32 dec232020.pp -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 10763 Dec 23 23:32 dec232020.te -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 foo -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 foo2 -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 foo3 -rwxr-xr-x 1 root root ? 24784 Jun 9 16:36 lsetfilecon -rw-r--r-- 1 root root ? 340 Jun 4 13:48 lsetfilecon.c -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 60 Nov 29 2020 virtiofs.cil -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 64 Oct 10 2020 virtiofs.conf -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 1394 Oct 11 2020 virtiofs_kernel_boot.pp -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 445 Oct 11 2020 virtiofs_kernel_boot.te root@noc1:/vmsystems/fedora_generic# [root@fedora ~]# cat /etc/selinux/config # This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=permissive # SELINUXTYPE= can take one of these three values: # targeted - Targeted processes are protected, # minimum - Modification of targeted policy. Only selected processes are protected. # mls - Multi Level Security protection. SELINUXTYPE=targeted "lsetfilecon.c" is the tiny c 'reproducer' program from above. Fails 100% of the time on latest fc34, and that's with selinux in permissive mode on the guest. (In reply to Vivek Goyal from comment #27) 1) Notice as shown in #3 above the problem exists in permissive mode on the guest. 2) on the host, during the reproducer run on the guest (noteice the 'success' on the host, but it fails on the guest) Jun 10 09:39:04 noc1 virtiofsd[1]: [ID: 00000307] lo_lookup(parent=1, name=proc) Jun 10 09:39:04 noc1 virtiofsd[1]: [ID: 00000307] 1/proc -> 18 Jun 10 09:39:04 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206744, success, outsize: 144 Jun 10 09:39:04 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_msg: elem 3: with 2 in desc of length 144 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Queue 1 gave evalue: 1 available: in: 120 out: 56 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Waiting for Queue 1 event Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] fv_queue_worker: elem 3: with 2 out desc of length 56 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206746, opcode: GETATTR (3), nodeid: 724, insize: 56, pid: 2322 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206746, success, outsize: 120 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_msg: elem 3: with 2 in desc of length 120 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Queue 1 gave evalue: 1 available: in: 144 out: 52 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Waiting for Queue 1 event Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] fv_queue_worker: elem 3: with 2 out desc of length 52 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206748, opcode: LOOKUP (1), nodeid: 724, insize: 52, pid: 2322 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] lo_lookup(parent=724, name=lsetfilecon) Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] 724/lsetfilecon -> 8606 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206748, success, outsize: 144 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_msg: elem 3: with 2 in desc of length 144 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Queue 1 gave evalue: 1 available: in: 32 out: 48 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Waiting for Queue 1 event Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] fv_queue_worker: elem 3: with 2 out desc of length 48 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206750, opcode: OPEN (14), nodeid: 8606, insize: 48, pid: 2322 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] lo_open(ino=8606, flags=32800) Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206750, success, outsize: 32 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_msg: elem 3: with 2 in desc of length 32 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Queue 1 gave evalue: 1 available: in: 16400 out: 80 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Waiting for Queue 1 event Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] fv_queue_worker: elem 3: with 2 out desc of length 80 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206752, opcode: READ (15), nodeid: 8606, insize: 80, pid: 2322 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] lo_read(ino=8606, size=16384, off=0) Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_data_iov: count=1 len=16384 iov_len=16 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_data_iov: elem 3: with 5 in desc of length 16400 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_data_iov: after skip skip_size=0 in_sg_cpy_count=4 in_sg_left=16384 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] virtio_send_data_iov: preadv ret=16384 len=16384 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Got queue event on Queue 1 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Queue 1 gave evalue: 1 available: in: 12448 out: 126 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000004] fv_queue_thread: Waiting for Queue 1 event Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] fv_queue_worker: elem 3: with 2 out desc of length 80 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] unique: 1206754, opcode: READ (15), nodeid: 8606, insize: 80, pid: 2322 Jun 10 09:39:05 noc1 virtiofsd[1]: [ID: 00000307] lo_read(ino=8606, size=12288, off=16384)
(In reply to Harry Coin from comment #28) > Ondrej 1. You mentioned 'boot failure'. In the plain meaning there is no > 'boot failure'. The guest boots to a normal gui, logins normal, etc. Guest > Selinux booting in permissive mode, and with xattrs on in the host. It's all > selinux issues crashing dnf so preventing upgrades/installs. Guest SELinux > as enforcing I know is a long way off for virtiofs as root, but now it > doesn't even work in permissive mode though it did a few months ago. Yes, I was referring to Vivek's/Dan's comments, but I should've said "boot issues" instead... [...] > [root@fedora ~]# ls -lZ /root [...] > -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 > foo3 > -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 24784 Jun 9 16:36 > lsetfilecon > -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 340 Jun 4 13:48 > lsetfilecon.c ^^ This caught my attention, because it shows that newly created files don't get a label as they should. I was going to say that this proves that the xattr setting fails somewhere at VFS layer or below, but looking into the code I can see that virtiofs/fuse doesn't call security_inode_init_security() on newly created inodes. Without that xattr labeling will not work properly with virtiofs. The above probably still doesn't explain the -EPERM failures with direct lsetxattr(2), but still needs to be fixed. Vivek, that'll be a job for you, I guess :) Just grep the kernel for "security_inode_init_security" to find examples of how other filesystems do it. [...] > 2) on the host, during the reproducer run on the guest (noteice the > 'success' on the host, but it fails on the guest) <snip the debug output> I can't see any SETXATTR (or similiar) operation in the debug output, so my theory about the host denying the operations seems to be wrong and the operation is cut off already somewhere on the guest. Either way, -EPERM would suggest a denial by DAC, not by SELinux. (Or by capabilities, which can in turn be denied by SELinux...). One possibility would be the following check in selinux_inode_setxattr(): if (!inode_owner_or_capable(mnt_userns, inode)) return -EPERM; But in your case it fails even when root sets a label on a root-owned file, which shouldn't hit that condition (unless there is some user namespace weirdness going on). So I'm still baffled...
FWIW: [root@fedora ~]# uname -a Linux fedora.1.quietfountain.com 5.12.9-300.fc34.x86_64 #1 SMP Thu Jun 3 13:51:40 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux [root@fedora ~]# ls -lZ lsetfilecon -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 24784 Jun 9 16:36 lsetfilecon [root@fedora ~]# ls -lZ lsetfilecon.c -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 340 Jun 4 13:48 lsetfilecon.c [root@fedora ~]# restorecon lsetfilecon* restorecon: Could not set context for /root/lsetfilecon: Operation not permitted restorecon: Could not set context for /root/lsetfilecon.c: Operation not permitted [root@fedora ~]# mount | grep myfs myfs on / type virtiofs (rw,relatime,seclabel) [root@fedora ~]# seinfo --genfs Genfscon: 105 genfscon 9p / system_u:object_r:nfs_t:s0 ... genfscon nfs / system_u:object_r:nfs_t:s0 genfscon nfs4 / system_u:object_r:nfs_t:s0 ... genfscon rootfs / system_u:object_r:root_t:s0 ... genfscon virtiofs / system_u:object_r:virtiofs_t:s0 root@fedora ~]# seinfo --fs_use Fs_use: 33 fs_use_task eventpollfs system_u:object_r:fs_t:s0; fs_use_task pipefs system_u:object_r:fs_t:s0; fs_use_task sockfs system_u:object_r:fs_t:s0; fs_use_trans devpts system_u:object_r:devpts_t:s0; fs_use_trans devtmpfs system_u:object_r:device_t:s0; fs_use_trans hugetlbfs system_u:object_r:hugetlbfs_t:s0; fs_use_trans mqueue system_u:object_r:tmpfs_t:s0; fs_use_trans shm system_u:object_r:tmpfs_t:s0; fs_use_trans tmpfs system_u:object_r:tmpfs_t:s0; fs_use_xattr btrfs system_u:object_r:fs_t:s0; fs_use_xattr ceph system_u:object_r:fs_t:s0; fs_use_xattr encfs system_u:object_r:fs_t:s0; fs_use_xattr ext2 system_u:object_r:fs_t:s0; fs_use_xattr ext3 system_u:object_r:fs_t:s0; fs_use_xattr ext4 system_u:object_r:fs_t:s0; fs_use_xattr ext4dev system_u:object_r:fs_t:s0; fs_use_xattr f2fs system_u:object_r:fs_t:s0; fs_use_xattr gfs system_u:object_r:fs_t:s0; fs_use_xattr gfs2 system_u:object_r:fs_t:s0; fs_use_xattr gpfs system_u:object_r:fs_t:s0; fs_use_xattr jffs2 system_u:object_r:fs_t:s0; fs_use_xattr jfs system_u:object_r:fs_t:s0; fs_use_xattr lustre system_u:object_r:fs_t:s0; fs_use_xattr ocfs2 system_u:object_r:fs_t:s0; fs_use_xattr odms system_u:object_r:fs_t:s0; fs_use_xattr overlay system_u:object_r:fs_t:s0; fs_use_xattr shiftfs system_u:object_r:fs_t:s0; fs_use_xattr squashfs system_u:object_r:fs_t:s0; fs_use_xattr virtiofs system_u:object_r:fs_t:s0; fs_use_xattr vxclonefs system_u:object_r:fs_t:s0; fs_use_xattr vxfs system_u:object_r:fs_t:s0; fs_use_xattr xfs system_u:object_r:fs_t:s0; fs_use_xattr zfs system_u:object_r:fs_t:s0; [root@fedora ~]# [root@fedora ~]# getenforce Permissive [root@fedora ~]# ./lsetfilecon ret ffffffff : Operation not permitted [root@fedora ~]# cat lsetfilecon.c #include <selinux/selinux.h> #include <stdio.h> #include <errno.h> void perror(const char *s); int main(int argc,char *argv[]){ int i; i= lsetfilecon("/usr/bin/rngtest","system_u:object_r:bin_t:s0"); //i= lsetfilecon("/usr/bin/rngtest;60b9120b","system_u:object_r:bin_t:s0"); printf("ret %lx\n",i); perror("\n"); return 0; } [root@fedora ~]#
[root@fedora ~]# ls -lZ /usr/bin/rngtest -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 21176 May 24 12:27 /usr/bin/rngtest [root@fedora ~]# restorecon /usr/bin/rngtest restorecon: Could not set context for /usr/bin/rngtest: Operation not permitted [root@fedora ~]# On the host: root@noc1:/vmsystems/fedora_generic# ps axu | grep virtiofsd | grep fedora root 2629619 0.0 0.0 9808 3360 ? S 12:30 0:00 /bin/bash /vmsystems/custom_virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock root 2629621 0.0 0.0 80068 3160 ? Sl 12:30 0:00 /usr/lib/qemu/virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock root 2629761 1.4 0.0 3154540 9752 ? Sl 12:30 1:09 /usr/lib/qemu/virtiofsd --fd=50 -o source=/vmsystems/fedora_generic,xattr,flock,posix_lock root@noc1:/vmsystems/fedora_generic# /usr/lib/qemu/virtiofsd --version using FUSE kernel interface version 7.32
So I think we're down to two cases here: a) No xattrmap Here when the virtiofsd is run without cap_sys_admin, it can't set the security.* xattr's needed by selinux. You probably need to add -o modcaps=sys_admin to the virtiofsd to give the virtiofsd cap_sys_admin, then I think it should be able to set security.* on the host; that is as long as the host selinux allows you to - but I don't think you have it enabled. b) xattrmap to user.virtiofsd. I thought this would work, but the symlink problems above come down to something Vivek spotted; there's a kernel restriction that stops unpriviliged users setting user.* on symlinks. That's also a pain. I'm not seeing an easy work around for that in the existing kernel.
oops, that (a) should have been -o modcaps=+sys_admin - note the '+'
Do we know the reason for the restriction on user.* xattrs on symlinks?
(In reply to Daniel Walsh from comment #34) > Do we know the reason for the restriction on user.* xattrs on symlinks? Yeh, the xattr man page says: User extended attributes User extended attributes may be assigned to files and directories for storing arbitrary additional information such as the mime type, character set or encoding of a file. The access permissions for user attributes are defined by the file permission bits: read permission is required to retrieve the attribute value, and writer permission is required to change it. The file permission bits of regular files and directories are interpreted differently from the file permission bits of special files and symbolic links. For regular files and directories the file permission bits define access to the file's contents, while for device special files they de‐ fine access to the device described by the special file. The file permissions of symbolic links are not used in access checks. These differ‐ ences would allow users to consume filesystem resources in a way not controllable by disk quotas for group or world writable special files and directories. For this reason, user extended attributes are allowed only for regular files and directories, and access to user extended attributes is re‐ stricted to the owner and to users with appropriate capabilities for directories with the sticky bit set (see the chmod(1) manual page for an ex‐ planation of the sticky bit).
Assigning to Ondrej. Feel free to switch the component back to selinux-policy if the problem is isolated there.
(In reply to Dr. David Alan Gilbert from comment #32) > So I think we're down to two cases here: > > a) No xattrmap > Here when the virtiofsd is run without cap_sys_admin, it can't set the > security.* xattr's needed by selinux. > You probably need to add > -o modcaps=sys_admin > > to the virtiofsd to give the virtiofsd cap_sys_admin, then I think it > should be able to set security.* > on the host; that is as long as the host selinux allows you to - but I > don't think you have it enabled. I can change label on file without having CAP_SYS_ADMIN. IOW, I dropped a new file in shared directory and that showed as unlabeled_t. And then I called lsetfilecon program to set label to "system_u:object_r:bin_t:s0" and that works even without virtiofsd having CAP_SYS_ADMIN. I guess SELinux might have relaxed the rules here. But I can't create new security xattr say "security.virtiofsd". I suspect that if I had SELinux disabled on host, and there will not be any security.selinux xattr on the file to begin with. And probably that will require having CAP_SYS_ADMIN. If yes, I think that explains the different mine and Harry's setup. We both are running F34 guest. But I have SELinux enabled on host while Harry does not (I think so.). Harry, can you try running virtiofsd with "-o modcaps=+sys_admin" and see if it solves your lsetfilecon issue.
(In reply to Ondrej Mosnacek from comment #29) > (In reply to Harry Coin f > > [root@fedora ~]# ls -lZ /root > [...] > > -rw-r--r--. 1 root root system_u:object_r:admin_home_t:s0 0 Nov 29 2020 > > foo3 > > -rwxr-xr-x. 1 root root system_u:object_r:unlabeled_t:s0 24784 Jun 9 16:36 > > lsetfilecon > > -rw-r--r--. 1 root root system_u:object_r:unlabeled_t:s0 340 Jun 4 13:48 > > lsetfilecon.c > > ^^ This caught my attention, because it shows that newly created files don't > get a label as they should. I was going to say that this proves that the > xattr setting fails somewhere at VFS layer or below, but looking into the > code I can see that virtiofs/fuse doesn't call > security_inode_init_security() on newly created inodes. Without that xattr > labeling will not work properly with virtiofs. This reminded me of an old conversation we had. There were patches to make it work with virtiofs. Basically we had to determine what will be file security context and send it along with file creation request. Server will have to create file (and security context) atomically. Chirantan had posted few patches. There are not merged yet. It probably requires little more review and work. Not sure why Chirantan stopped following up with upstream. https://listman.redhat.com/archives/virtio-fs/2020-July/msg00014.html https://listman.redhat.com/archives/virtio-fs/2020-July/msg00015.html > > The above probably still doesn't explain the -EPERM failures with direct > lsetxattr(2), but still needs to be fixed. Vivek, that'll be a job for you, > I guess :) Just grep the kernel for "security_inode_init_security" to find > examples of how other filesystems do it. We probably will not call security_inode_init_security as we can set xattr atomically. While local filesystems can do that w.r.t file creation. So we will need something like what chirantan proposed and send security label with the file creation request to remote server.
(In reply to Vivek Goyal from comment #37) > I can change label on file without having CAP_SYS_ADMIN. IOW, I dropped a new > file in shared directory and that showed as unlabeled_t. And then I called > lsetfilecon program to set label to "system_u:object_r:bin_t:s0" and > that works even without virtiofsd having CAP_SYS_ADMIN. I guess SELinux > might have relaxed the rules here. But I can't create new security > xattr say "security.virtiofsd". > > I suspect that if I had SELinux disabled on host, and there will not > be any security.selinux xattr on the file to begin with. And probably > that will require having CAP_SYS_ADMIN. If yes, I think that explains > the different mine and Harry's setup. We both are running F34 guest. > But I have SELinux enabled on host while Harry does not (I think so.). > > Harry, can you try running virtiofsd with "-o modcaps=+sys_admin" and see if > it solves your lsetfilecon issue. Well, your virtiofsd logs suggest that setxattr() request did not even make it to server. So while "cap_sys_admin" is required but you are facing something before that. Now it looks like you will have to put printk() statement in kernel code to figure out who is returning -EPERM. I can't reproduce it.
This message is a reminder that Fedora Linux 34 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '34'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 34 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
Fedora Linux 34 entered end-of-life (EOL) status on 2022-06-07. Fedora Linux 34 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. Thank you for reporting this bug and we are sorry it could not be fixed.