Bug 2079582

Summary: [libvirt] virtiofsd: Allow option --thread-pool-size=X
Product: Red Hat Enterprise Linux 8 Reporter: Vivek Goyal <vgoyal>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Lili Zhu <lizhu>
Severity: medium Docs Contact:
Priority: medium    
Version: 8.4CC: dgilbert, dzheng, jdenemar, jsuchane, jtomko, lmen, nilal, virt-maint, xuzhang
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libvirt-8.0.0-9.module+el8.7.0+15830+85788ab7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-11-08 09:19:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version: 8.5.0
Embargoed:

Description Vivek Goyal 2022-04-27 18:59:10 UTC
Description of problem:

virtiofsd has an option --thread-pool-size=X which can setup a thread pool and use that thread pool to do I/O in parallel (instead of having single thread doing I/O).

There are more details in following bug.

https://bugzilla.redhat.com/show_bug.cgi?id=2072905

By default thread pool is disabled because thread pool seems to help only on faster storage with lot of parallel I/O going. Otherwise overhead of thread pool can be detrimental.

Providing this option will allow user to enable it if they know they are going to benefit from using a thread pool.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 2 Ján Tomko 2022-06-10 13:28:04 UTC
Upstream patches adding <thread_pool size='%i'/> element:
https://listman.redhat.com/archives/libvir-list/2022-June/232406.html

Comment 4 Ján Tomko 2022-06-16 13:05:46 UTC
Pushed upstream as:

commit 0df2e7df80452f81edbfeb0ee355235b533346a9
Author:     Ján Tomko <jtomko>
CommitDate: 2022-06-16 14:58:25 +0200

    conf: virtiofs: add thread_pool element
    
    Add an element to configure the thread pool size:
    
    ...
    <binary>
      <thread_pool size='16'/>
    </binary>
    ...
    
    https://bugzilla.redhat.com/show_bug.cgi?id=2072905
    
    Signed-off-by: Ján Tomko <jtomko>
    Reviewed-by: Michal Privoznik <mprivozn>

git describe: v8.4.0-256-g0df2e7df80
fedora ~/work/libvirt $ post HEAD^^ HEAD
commit 2753eba20ce76d3d8785b23a6e940574ca12fe3c
Author:     Ján Tomko <jtomko>
CommitDate: 2022-06-16 14:58:25 +0200

    qemu: virtiofs: format --thread-pool-size
    
    https://bugzilla.redhat.com/show_bug.cgi?id=2079582
    
    Signed-off-by: Ján Tomko <jtomko>
    Reviewed-by: Michal Privoznik <mprivozn>

commit 0df2e7df80452f81edbfeb0ee355235b533346a9
Author:     Ján Tomko <jtomko>
CommitDate: 2022-06-16 14:58:25 +0200

    conf: virtiofs: add thread_pool element
    
    Add an element to configure the thread pool size:
    
    ...
    <binary>
      <thread_pool size='16'/>
    </binary>
    ...
    
    https://bugzilla.redhat.com/show_bug.cgi?id=2072905
    
    Signed-off-by: Ján Tomko <jtomko>
    Reviewed-by: Michal Privoznik <mprivozn>

git describe: v8.4.0-257-g2753eba20c

Comment 6 Vivek Goyal 2022-06-16 13:28:01 UTC
(In reply to Ján Tomko from comment #4)
> Pushed upstream as:
> 

Thanks Jan Tomko for taking care of this.

Comment 15 Lili Zhu 2022-07-03 03:07:24 UTC
Verify this bug with:
libvirt-daemon-8.0.0-9.module+el8.7.0+15830+85788ab7.x86_64
qemu-kvm-6.2.0-16.module+el8.7.0+15743+c774064d.x86_64

1. prepare a guest with the following xml snippet
  <filesystem type='mount' accessmode='passthrough'>
      <driver type='virtiofs' queue='1024'/>
      <binary path='/usr/libexec/virtiofsd' xattr='on'>
        <thread_pool size='32'/>
      </binary>
      <source dir='/path1'/>
      <target dir='mount_tag1'/>
      <alias name='fs0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
    </filesystem>

2. start the guest
# virsh start avocado-vt-vm1 
Domain 'avocado-vt-vm1' started

3. check the virtiofsd process
# ps aux |grep virtiofs
root      235785  0.0  0.0 122808  4836 ?        Sl   22:53   0:00 /usr/libexec/virtiofsd --fd=44 -o source=/path1,xattr --thread-pool-size=32
root      235830  0.0  0.0 1318848 1900 ?        Sl   22:53   0:00 /usr/libexec/virtiofsd --fd=44 -o source=/path1,xattr --thread-pool-size=32

4. check the qemu cmdline
...
-chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-14-avocado-vt-vm1/fs0-fs.sock -device vhost-user-fs-pci,id=fs0,chardev=chr-vu-fs0,queue-size=1024,tag=mount_tag1,bus=pcie.0,addr=0x9
...

5.login the guest then mount the virtiofs:
[guest os]# mkdir /root/mount1; mount -t virtiofs mount_tag1 /root/mount1

6. Generate a file on the mountpoint in the guest:
(guest os)# dd if=/dev/random of=testfile bs=1M count=1024

7. check the md5sum value
[guest os]# md5sum /mount1/testfile
24007156c378fb3a7ec0d937c03775b8  testfile

[host os]# md5sum testfile 
24007156c378fb3a7ec0d937c03775b8  testfile

After added the option --thread-pool-size=X, functions work well. Mark the bug as verified.

Comment 17 errata-xmlrpc 2022-11-08 09:19:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: virt:rhel and virt-devel:rhel security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7472