Bug 1726528

Summary: qemu-img convert/map much slower when using qemu-nbd compared with direct access to image
Product: Red Hat Enterprise Linux Advanced Virtualization Reporter: Tingting Mao <timao>
Component: qemu-kvmAssignee: Maxim Levitsky <mlevitsk>
Status: CLOSED ERRATA QA Contact: Tingting Mao <timao>
Severity: unspecified Docs Contact:
Priority: medium    
Version: 8.1CC: chayang, coli, ddepaula, eblake, juzhang, mlevitsk, nsoffer, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-4.1.0-1.module+el8.1.0+3966+4a23dca1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1647104 Environment:
Last Closed: 2019-11-06 07:17:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1647104    
Bug Blocks:    

Description Tingting Mao 2019-07-03 06:17:10 UTC 
+++ This bug was initially created as a clone of Bug #1647104 +++

Description of problem:
qemu-img convert and map are much slower when accessing an image using qemu-nbd 
compared with accessing an image directly.


Version-Release number of selected component (if applicable):
qemu-kvm-4.0.0-4.module+el8.1.0+3523+b348b848
kernel-4.18.0-107.el8


How reproducible:
Always


Steps to reproduce:
Local:
# time qemu-img map --output=json /dev/nvme0n1p1 > test.json

real	0m0.014s
user	0m0.006s
sys	0m0.007s

# time qemu-img convert -f qcow2 -O raw /dev/nvme0n1p1 /dev/shm/tgt.img 

real	0m0.013s
user	0m0.007s
sys	0m0.006s


Over NBD:
# time qemu-img map -f raw --output=json nbd:unix:/home/test/my.socket:exportname=export > test_nbd.json 

real	0m0.732s
user	0m0.295s
sys	0m0.242s
# time qemu-img convert -f raw -O raw nbd:unix:/home/test/my.socket:exportname=export /dev/shm/tgt.img 

real	0m5.023s
user	0m0.981s
sys	0m2.808s


Additional info:
Source image info:
1. Partition info:
nvme0n1                          259:0    0 745.2G  0 disk 
└─nvme0n1p1                      259:1    0     5G  0 part 
2. Mount the partition and write 1.5G data to it
# mount /dev/nvme0n1p1 /mnt/
# cd /mnt/
# dd if=/dev/urandom of=f1 bs=1M count=1024
# dd if=/dev/urandom of=f2 bs=1M count=1024
# umount /mnt
# qemu-img create -f qcow2 /dev/nvme0n1p1 5G

Export image via below CML:
# qemu-nbd -k /home/test/my.socket -v -f raw -x export --cache=none --aio=threads --persistent /dev/nvme0n1p1
Comment 2 John Ferlan 2019-08-01 11:42:40 UTC 
Patch is accepted upstream:

https://git.qemu.org/?p=qemu.git;a=commit;h=867eccfed84f96b54f4a432c510a02c2ce03b430
Comment 4 Tingting Mao 2019-08-23 07:55:20 UTC 
Tried to verify this bug as below:


Tested with:
qemu-kvm-4.1.0-4.module+el8.1.0+4020+16089f93
kernel-4.18.0-134.el8


Steps:
Local:
# time qemu-img map --output=json /dev/nvme0n1p1 > tt.json

real	0m0.012s
user	0m0.008s
sys	0m0.004s

# time qemu-img convert -f qcow2 -O raw /dev/nvme0n1p1 /dev/shm/tgt.img 

real	0m0.014s
user	0m0.008s
sys	0m0.006s


Over NBD:
# time qemu-img map -f raw --output=json nbd:unix:/home/test/my.socket:exportname=export > test_nbd.json

real	0m0.019s
user	0m0.009s
sys	0m0.007s

# time qemu-img convert -f raw -O raw nbd:unix:/home/test/my.socket:exportname=export /dev/shm/tgt.img 

real	0m3.276s
user	0m0.334s
sys	0m2.203s


Additional info:
Source image info:
1. Partition info:
nvme0n1                            259:0    0 745.2G  0 disk 
└─nvme0n1p1                        259:1    0     5G  0 part 

2. Mount the partition and write 1.5G data to it
# mkfs.xfs /dev/nvme0n1p1
# mount /dev/nvme0n1p1 /mnt/
# cd /mnt/
# dd if=/dev/urandom of=f1 bs=1M count=1024
# dd if=/dev/urandom of=f2 bs=1M count=1024
# umount /mnt
# qemu-img create -f qcow2 /dev/nvme0n1p1 5G

Export image via below CML:
# qemu-nbd -k /home/test/my.socket -v -f raw -x export --cache=none --aio=threads --persistent /dev/nvme0n1p1
Comment 8 errata-xmlrpc 2019-11-06 07:17:18 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:3723