Bug 2148352

Summary: [QEMU-7.2][virtiofs] mount virtiofs stuck and got error 'SELinux: (dev virtiofs, type virtiofs) getxattr errno 4' when force quite
Product: Red Hat Enterprise Linux 9 Reporter: Yanan Fu <yfu>
Component: qemu-kvmAssignee: German Maglione <gmaglione>
qemu-kvm sub component: virtio-fs QA Contact: xiagao
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: coli, gmaglione, jinzhao, juzhang, kwolf, mrezanin, vgoyal, virt-maint, xfu, xiagao, xuhan, yfu
Version: 9.2Keywords: Triaged, Upstream
Target Milestone: rcFlags: pm-rhel: mirror+
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-7.2.0-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-09 07:20:55 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Yanan Fu 2022-11-25 07:12:27 UTC
Description of problem:
mount virtiofs stuck and got error 'SELinux: (dev virtiofs, type virtiofs) getxattr errno 4' when force quite


In guest:
# getenforce 
Permissive


Version-Release number of selected component (if applicable):
qemu-kvm-core-7.2.0-0.rc1.el9.preview.x86_64
virtiofsd-1.4.0-1.el9.x86_64
kernel-5.14.0-197.el9.x86_64 (both host and guest)

How reproducible:
100%

Steps to Reproduce:
1.  start virtiofsd on rhel920 host(RHEL-9.2.0-20221122.2)

# mkdir -p /root/avocado/data/avocado-vt/virtio_fs_test/
# /usr/libexec/virtiofsd --socket-path=/var/tmp/avocado_9twgkzgh/avocado-vt-vm1-fs-virtiofsd.sock -o source=/root/avocado/data/avocado-vt/virtio_fs_test/,cache=always

2. Boot RHEL.9.2.0 guest with virtiofs device
     -m 4096 \
     -object memory-backend-file,size=4G,mem-path=/dev/shm,share=yes,id=mem-mem1  \
     -chardev socket,id=char_virtiofs_fs,path=/var/tmp/avocado_9twgkzgh/avocado-vt-vm1-fs-virtiofsd.sock \
     -device pcie-root-port,id=pcie-root-port-3,port=0x3,addr=0x1.0x3,bus=pcie.0,chassis=4 \
     -device vhost-user-fs-pci,id=vufs_virtiofs_fs,chardev=char_virtiofs_fs,tag=myfs,queue-size=1024,bus=pcie-root-port-3,addr=0x0 \


3. mount virtiofs in guest os
# mkdir -p /mnt/myfs
# mount -t virtiofs myfs /mnt/myfs

mount comand stuck, can not finish for long time.
Try to kill the mount process, dmesg show error:

SELinux: (dev virtiofs, type virtiofs) getxattr errno 4

And, the mount process can not be killed successfully.

Actual results:
mount virtiofs failed

Expected results:
mount virtiofs success

Additional info:
If disable the selinux in guest os, it can work well

Comment 3 German Maglione 2022-11-29 11:49:27 UTC
This is a qemu bug. Commit 02b61f38d3574900fb4cc4c450b17c75956a6a04 (in qemu) adds the backend features bits to vhost-user.c to vhost_user_set_features(). This exposed a bug on several devices, including vhost-user-fs.

```
static int vhost_user_set_features(...) {
    ...
    /*
     * We need to include any extra backend only feature bits that
     * might be needed by our device. Currently this includes the
     * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol
     * features.
     */
    return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES,
                              features | dev->backend_features,
                              log_enabled);
    ...
}
```

virtiofsd is a backend from a qemu POV, so the VHOST_USER_F_PROTOCOL_FEATURES is set by the rust version of virtiofs (The C version is not setting this feature, so is not affected). According to the vhost user spec, the master must send VHOST_USER_SET_VRING_ENABLE to the backend if VHOST_USER_F_PROTOCOL_FEATURES has been negotiated, otherwise the vring is initialized in a disable state. The vhost-user-fs is not doing that, so the guest blocks when trying to mount a virtiofs fs. I've also seen sometimes it blocks when you try to enter into the directory in the guest.

A patch is already sent to the upstream, but not approved yet: https://lists.gnu.org/archive/html/qemu-devel/2022-11/msg03814.html

Comment 4 German Maglione 2022-12-01 10:14:00 UTC
The patch that fixes this issue has been included in the Alex Benne's series that fixes also other vhost issues, he just sent a v4:
https://lore.kernel.org/qemu-devel/20221130112439.2527228-1-alex.bennee@linaro.org/

Comment 5 German Maglione 2022-12-05 10:31:30 UTC
The fix was merged in uptream, commit: 4daa5054c599c8aec70f1264a01341a15921c145

Comment 7 Yanan Fu 2022-12-20 09:19:11 UTC
QE bot(pre verify): Set 'Verified:Tested,SanityOnly' as gating/tier1 test pass.

Comment 9 xiagao 2022-12-28 06:52:02 UTC
Test on qemu-kvm-7.2.0-1.el9, the shared directory mounted works.

In guest:
# getenforce 
Permissive

pkg:
virtiofsd-1.5.0-1.el9.x86_64
qemu-kvm-7.2.0-1.el9
5.14.0-219.el9.x86_64(host)
5.14.0-226.el9.x86_64(guest)

Comment 12 xiagao 2023-01-11 01:37:54 UTC
Based on Comment9, I will set verified status for this bug.

Comment 14 errata-xmlrpc 2023-05-09 07:20:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: qemu-kvm security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2023:2162