Bug 1482537 - [RFE] qemu-img copy-offloading (convert command)
Summary: [RFE] qemu-img copy-offloading (convert command)
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.6
Hardware: Unspecified
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Fam Zheng
QA Contact: Tingting Mao
URL:
Whiteboard:
Depends On:
Blocks: 912761 RHEV_clone_disk_with_xcopy 1558125 1594820
TreeView+ depends on / blocked
 
Reported: 2017-08-17 13:38 UTC by Yaniv Kaul
Modified: 2018-11-01 11:01 UTC (History)
19 users (show)

Fixed In Version: qemu-kvm-rhev-2.12.0-7.el7
Doc Type: Enhancement
Doc Text:
Clone Of:
: 1594820 (view as bug list)
Environment:
Last Closed: 2018-11-01 11:01:10 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Yaniv Kaul 2017-08-17 13:38:55 UTC
Description of problem:

qemu-img should support copying data, if the target and source support XCOPY, using the XCOPY SCSI command. 
It will significantly reduce CPU and bandwidth consumption on the host and will accelerate the copy.
It may be using ibiscsi, which already has the support for it[1]. Another interesting factor is that the libiscsi maintainer recently joined Red Hat (as kernel CIFS maintainer). 

This feature will benefit both RHV and OpenStack.

[1] https://github.com/sahlberg/libiscsi/commit/4bc5f962e2f0655734d0cc9cfe327d2800b7ac79

Comment 2 Yaniv Kaul 2018-05-16 07:43:00 UTC
My original request has been somewhat generic 'qemu-img'. That means that any copy command should benefit from it - such as commit, 'dd' (who's using it?!), rebase(?).

Should we open more specific RFEs, especially on commit?

Comment 3 Fam Zheng 2018-05-16 08:13:12 UTC
Listing the sub-commands is enough IMO. I count 3 of them: commit, rebase and convert.

Another BZ could be opened for block jobs (block-commit, block-stream, drive-backup, drive-mirror and all their blockdev-* variants.

Comment 4 Yaniv Kaul 2018-05-16 08:19:57 UTC
(In reply to Fam Zheng from comment #3)
> Listing the sub-commands is enough IMO. I count 3 of them: commit, rebase
> and convert.
> 
> Another BZ could be opened for block jobs (block-commit, block-stream,
> drive-backup, drive-mirror and all their blockdev-* variants.

Of course - this was actually my main motivation - faster live storage migration...  opened https://bugzilla.redhat.com/show_bug.cgi?id=1578699

Comment 7 Miroslav Rezanina 2018-07-04 08:12:29 UTC
Fix included in qemu-kvm-rhev-2.12.0-7.el7

Comment 10 Tingting Mao 2018-07-17 09:32:45 UTC
Hi Fam,

According the result in comment 9, it seems only the usage of libiscsi(usage: iscsi://) improved the performance. However, for nfs and iscsi(usage: login target and create lvs) backend, the convert time is longer with Qemu-kvm-rhev-2.12.0-7.el7 than it with Qemu-kvm-rhev-2.10.0-21.el7_5.1. So could you please help to check where is wrong.

Thanks.

Comment 11 Fam Zheng 2018-07-17 09:47:22 UTC
Can you do

strace -e copy_file_range -f qemu-img convert ...

for the nfs case and see if the syscall is invoked?

We can ignore the lv+iscsi for now, it seems host kernel is not optimized.

Comment 12 Tingting Mao 2018-07-17 10:06:46 UTC
(In reply to Fam Zheng from comment #11)
> Can you do
> 
> strace -e copy_file_range -f qemu-img convert ...
> 
> for the nfs case and see if the syscall is invoked?

Yes, it does.

# time strace -e copy_file_range -f qemu-img convert -f qcow2 -O qcow2 base.qcow2 convert.qcow2 
strace: Process 58483 attached
[pid 58483] copy_file_range(10, [327680], 12, [327680], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [2424832], 12, [2424832], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [4521984], 12, [4521984], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [6619136], 12, [6619136], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [8716288], 12, [8716288], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [10813440], 12, [10813440], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [12910592], 12, [12910592], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [15007744], 12, [15007744], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [17104896], 12, [17104896], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [19202048], 12, [19202048], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [21299200], 12, [21299200], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [23396352], 12, [23396352], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [25493504], 12, [25493504], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [27590656], 12, [27590656], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [29687808], 12, [29687808], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [31784960], 12, [31784960], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [33882112], 12, [33882112], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [35979264], 12, [35979264], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [38076416], 12, [38076416], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [40173568], 12, [40173568], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [42270720], 12, [42270720], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [44367872], 12, [44367872], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [46465024], 12, [46465024], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [48562176], 12, [48562176], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [50659328], 12, [50659328], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [52756480], 12, [52756480], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [54853632], 12, [54853632], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [56950784], 12, [56950784], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [59047936], 12, [59047936], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [61145088], 12, [61145088], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [63242240], 12, [63242240], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [65339392], 12, [65339392], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [67436544], 12, [67436544], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [69533696], 12, [69533696], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [71630848], 12, [71630848], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [73728000], 12, [73728000], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [75825152], 12, [75825152], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [77922304], 12, [77922304], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [80019456], 12, [80019456], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [82116608], 12, [82116608], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [84213760], 12, [84213760], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [86310912], 12, [86310912], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [88408064], 12, [88408064], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [90505216], 12, [90505216], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [92602368], 12, [92602368], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [94699520], 12, [94699520], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [96796672], 12, [96796672], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [98893824], 12, [98893824], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [100990976], 12, [100990976], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [103088128], 12, [103088128], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [105185280], 12, [105185280], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [107282432], 12, [107282432], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [109379584], 12, [109379584], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [111476736], 12, [111476736], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [113573888], 12, [113573888], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [115671040], 12, [115671040], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [117768192], 12, [117768192], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [119865344], 12, [119865344], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [121962496], 12, [121962496], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [124059648], 12, [124059648], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [126156800], 12, [126156800], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [128253952], 12, [128253952], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [130351104], 12, [130351104], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [132448256], 12, [132448256], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [134545408], 12, [134545408], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [136642560], 12, [136642560], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [138739712], 12, [138739712], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [140836864], 12, [140836864], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [142934016], 12, [142934016], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [145031168], 12, [145031168], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [147128320], 12, [147128320], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [149225472], 12, [149225472], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [151322624], 12, [151322624], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [153419776], 12, [153419776], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [155516928], 12, [155516928], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [157614080], 12, [157614080], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [159711232], 12, [159711232], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [161808384], 12, [161808384], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [163905536], 12, [163905536], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [166002688], 12, [166002688], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [168099840], 12, [168099840], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [170196992], 12, [170196992], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [172294144], 12, [172294144], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [174391296], 12, [174391296], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [176488448], 12, [176488448], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [178585600], 12, [178585600], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [180682752], 12, [180682752], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [182779904], 12, [182779904], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [184877056], 12, [184877056], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [186974208], 12, [186974208], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [189071360], 12, [189071360], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [191168512], 12, [191168512], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [193265664], 12, [193265664], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [195362816], 12, [195362816], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [197459968], 12, [197459968], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [199557120], 12, [199557120], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [201654272], 12, [201654272], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [203751424], 12, [203751424], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [205848576], 12, [205848576], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [207945728], 12, [207945728], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [210042880], 12, [210042880], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [212140032], 12, [212140032], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [214237184], 12, [214237184], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [216334336], 12, [216334336], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [218431488], 12, [218431488], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [220528640], 12, [220528640], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [222625792], 12, [222625792], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [224722944], 12, [224722944], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [226820096], 12, [226820096], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [228917248], 12, [228917248], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [231014400], 12, [231014400], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [233111552], 12, [233111552], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [235208704], 12, [235208704], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [237305856], 12, [237305856], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [239403008], 12, [239403008], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [241500160], 12, [241500160], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [243597312], 12, [243597312], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [245694464], 12, [245694464], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [247791616], 12, [247791616], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [249888768], 12, [249888768], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [251985920], 12, [251985920], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [254083072], 12, [254083072], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [256180224], 12, [256180224], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [258277376], 12, [258277376], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [260374528], 12, [260374528], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [262471680], 12, [262471680], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [264568832], 12, [264568832], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [266665984], 12, [266665984], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [268763136], 12, [268763136], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [270860288], 12, [270860288], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [272957440], 12, [272957440], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [275054592], 12, [275054592], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [277151744], 12, [277151744], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [279248896], 12, [279248896], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [281346048], 12, [281346048], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [283443200], 12, [283443200], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [285540352], 12, [285540352], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [287637504], 12, [287637504], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [289734656], 12, [289734656], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [291831808], 12, [291831808], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [293928960], 12, [293928960], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [296026112], 12, [296026112], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [298123264], 12, [298123264], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [300220416], 12, [300220416], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [302317568], 12, [302317568], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [304414720], 12, [304414720], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [306511872], 12, [306511872], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [308609024], 12, [308609024], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [310706176], 12, [310706176], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [312803328], 12, [312803328], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [314900480], 12, [314900480], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [316997632], 12, [316997632], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [319094784], 12, [319094784], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [321191936], 12, [321191936], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [323289088], 12, [323289088], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [325386240], 12, [325386240], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [327483392], 12, [327483392], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [329580544], 12, [329580544], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [331677696], 12, [331677696], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [333774848], 12, [333774848], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [335872000], 12, [335872000], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [337969152], 12, [337969152], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [340066304], 12, [340066304], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [342163456], 12, [342163456], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [344260608], 12, [344260608], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [346357760], 12, [346357760], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [348454912], 12, [348454912], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [350552064], 12, [350552064], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [352649216], 12, [352649216], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [354746368], 12, [354746368], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [356843520], 12, [356843520], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [358940672], 12, [358940672], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [361037824], 12, [361037824], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [363134976], 12, [363134976], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [365232128], 12, [365232128], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [367329280], 12, [367329280], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [369426432], 12, [369426432], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [371523584], 12, [371523584], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [373620736], 12, [373620736], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [375717888], 12, [375717888], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [377815040], 12, [377815040], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [379912192], 12, [379912192], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [382009344], 12, [382009344], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [384106496], 12, [384106496], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [386203648], 12, [386203648], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [388300800], 12, [388300800], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [390397952], 12, [390397952], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [392495104], 12, [392495104], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [394592256], 12, [394592256], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [396689408], 12, [396689408], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [398786560], 12, [398786560], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [400883712], 12, [400883712], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [402980864], 12, [402980864], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [405078016], 12, [405078016], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [407175168], 12, [407175168], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [409272320], 12, [409272320], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [411369472], 12, [411369472], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [413466624], 12, [413466624], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [415563776], 12, [415563776], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [417660928], 12, [417660928], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [419758080], 12, [419758080], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [421855232], 12, [421855232], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [423952384], 12, [423952384], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [426049536], 12, [426049536], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [428146688], 12, [428146688], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [430243840], 12, [430243840], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [432340992], 12, [432340992], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [434438144], 12, [434438144], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [436535296], 12, [436535296], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [438632448], 12, [438632448], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [440729600], 12, [440729600], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [442826752], 12, [442826752], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [444923904], 12, [444923904], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [447021056], 12, [447021056], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [449118208], 12, [449118208], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [451215360], 12, [451215360], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [453312512], 12, [453312512], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [455409664], 12, [455409664], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [457506816], 12, [457506816], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [459603968], 12, [459603968], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [461701120], 12, [461701120], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [463798272], 12, [463798272], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [465895424], 12, [465895424], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [467992576], 12, [467992576], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [470089728], 12, [470089728], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [472186880], 12, [472186880], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [474284032], 12, [474284032], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [476381184], 12, [476381184], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [478478336], 12, [478478336], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [480575488], 12, [480575488], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [482672640], 12, [482672640], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [484769792], 12, [484769792], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [486866944], 12, [486866944], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [488964096], 12, [488964096], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [491061248], 12, [491061248], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [493158400], 12, [493158400], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [495255552], 12, [495255552], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [497352704], 12, [497352704], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [499449856], 12, [499449856], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [501547008], 12, [501547008], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [503644160], 12, [503644160], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [505741312], 12, [505741312], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [507838464], 12, [507838464], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [509935616], 12, [509935616], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [512032768], 12, [512032768], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [514129920], 12, [514129920], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [516227072], 12, [516227072], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [518324224], 12, [518324224], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [520421376], 12, [520421376], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [522518528], 12, [522518528], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [524615680], 12, [524615680], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [526712832], 12, [526712832], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [528809984], 12, [528809984], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [530907136], 12, [530907136], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [533004288], 12, [533004288], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [535101440], 12, [535101440], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [537264128], 12, [537264128], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [539361280], 12, [539361280], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [541458432], 12, [541458432], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [543555584], 12, [543555584], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [545652736], 12, [545652736], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [547749888], 12, [547749888], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [549847040], 12, [549847040], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [551944192], 12, [551944192], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [554041344], 12, [554041344], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [556138496], 12, [556138496], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [558235648], 12, [558235648], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [560332800], 12, [560332800], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [562429952], 12, [562429952], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [564527104], 12, [564527104], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [566624256], 12, [566624256], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [568721408], 12, [568721408], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [570818560], 12, [570818560], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [572915712], 12, [572915712], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [575012864], 12, [575012864], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [577110016], 12, [577110016], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [579207168], 12, [579207168], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [581304320], 12, [581304320], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [583401472], 12, [583401472], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [585498624], 12, [585498624], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [587595776], 12, [587595776], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [589692928], 12, [589692928], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [591790080], 12, [591790080], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [593887232], 12, [593887232], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [595984384], 12, [595984384], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [598081536], 12, [598081536], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [600178688], 12, [600178688], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [602275840], 12, [602275840], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [604372992], 12, [604372992], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [606470144], 12, [606470144], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [608567296], 12, [608567296], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [610664448], 12, [610664448], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [612761600], 12, [612761600], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [614858752], 12, [614858752], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [616955904], 12, [616955904], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [619053056], 12, [619053056], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [621150208], 12, [621150208], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [623247360], 12, [623247360], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [625344512], 12, [625344512], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [627441664], 12, [627441664], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [629538816], 12, [629538816], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [631635968], 12, [631635968], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [633733120], 12, [633733120], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [635830272], 12, [635830272], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [637927424], 12, [637927424], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [640024576], 12, [640024576], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [642121728], 12, [642121728], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [644218880], 12, [644218880], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [646316032], 12, [646316032], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [648413184], 12, [648413184], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [650510336], 12, [650510336], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [652607488], 12, [652607488], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [654704640], 12, [654704640], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [656801792], 12, [656801792], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [658898944], 12, [658898944], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [660996096], 12, [660996096], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [663093248], 12, [663093248], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [665190400], 12, [665190400], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [667287552], 12, [667287552], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [669384704], 12, [669384704], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [671481856], 12, [671481856], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [673579008], 12, [673579008], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [675676160], 12, [675676160], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [677773312], 12, [677773312], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [679870464], 12, [679870464], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [681967616], 12, [681967616], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [684064768], 12, [684064768], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [686161920], 12, [686161920], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [688259072], 12, [688259072], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [690356224], 12, [690356224], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [692453376], 12, [692453376], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [694550528], 12, [694550528], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [696647680], 12, [696647680], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [698744832], 12, [698744832], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [700841984], 12, [700841984], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [702939136], 12, [702939136], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [705036288], 12, [705036288], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [707133440], 12, [707133440], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [709230592], 12, [709230592], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [711327744], 12, [711327744], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [713424896], 12, [713424896], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [715522048], 12, [715522048], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [717619200], 12, [717619200], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [719716352], 12, [719716352], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [721813504], 12, [721813504], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [723910656], 12, [723910656], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [726007808], 12, [726007808], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [728104960], 12, [728104960], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [730202112], 12, [730202112], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [732299264], 12, [732299264], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [734396416], 12, [734396416], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [736493568], 12, [736493568], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [738590720], 12, [738590720], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [740687872], 12, [740687872], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [742785024], 12, [742785024], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [744882176], 12, [744882176], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [746979328], 12, [746979328], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [749076480], 12, [749076480], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [751173632], 12, [751173632], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [753270784], 12, [753270784], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [755367936], 12, [755367936], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [757465088], 12, [757465088], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [759562240], 12, [759562240], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [761659392], 12, [761659392], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [763756544], 12, [763756544], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [765853696], 12, [765853696], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [767950848], 12, [767950848], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [770048000], 12, [770048000], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [772145152], 12, [772145152], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [774242304], 12, [774242304], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [776339456], 12, [776339456], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [778436608], 12, [778436608], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [780533760], 12, [780533760], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [782630912], 12, [782630912], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [784728064], 12, [784728064], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [786825216], 12, [786825216], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [788922368], 12, [788922368], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [791019520], 12, [791019520], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [793116672], 12, [793116672], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [795213824], 12, [795213824], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [797310976], 12, [797310976], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [799408128], 12, [799408128], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [801505280], 12, [801505280], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [803602432], 12, [803602432], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [805699584], 12, [805699584], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [807796736], 12, [807796736], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [809893888], 12, [809893888], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [811991040], 12, [811991040], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [814088192], 12, [814088192], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [816185344], 12, [816185344], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [818282496], 12, [818282496], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [820379648], 12, [820379648], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [822476800], 12, [822476800], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [824573952], 12, [824573952], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [826671104], 12, [826671104], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [828768256], 12, [828768256], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [830865408], 12, [830865408], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [832962560], 12, [832962560], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [835059712], 12, [835059712], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [837156864], 12, [837156864], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [839254016], 12, [839254016], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [841351168], 12, [841351168], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [843448320], 12, [843448320], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [845545472], 12, [845545472], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [847642624], 12, [847642624], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [849739776], 12, [849739776], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [851836928], 12, [851836928], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [853934080], 12, [853934080], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [856031232], 12, [856031232], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [858128384], 12, [858128384], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [860225536], 12, [860225536], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [862322688], 12, [862322688], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [864419840], 12, [864419840], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [866516992], 12, [866516992], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [868614144], 12, [868614144], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [870711296], 12, [870711296], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [872808448], 12, [872808448], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [874905600], 12, [874905600], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [877002752], 12, [877002752], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [879099904], 12, [879099904], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [881197056], 12, [881197056], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [883294208], 12, [883294208], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [885391360], 12, [885391360], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [887488512], 12, [887488512], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [889585664], 12, [889585664], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [891682816], 12, [891682816], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [893779968], 12, [893779968], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [895877120], 12, [895877120], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [897974272], 12, [897974272], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [900071424], 12, [900071424], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [902168576], 12, [902168576], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [904265728], 12, [904265728], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [906362880], 12, [906362880], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [908460032], 12, [908460032], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [910557184], 12, [910557184], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [912654336], 12, [912654336], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [914751488], 12, [914751488], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [916848640], 12, [916848640], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [918945792], 12, [918945792], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [921042944], 12, [921042944], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [923140096], 12, [923140096], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [925237248], 12, [925237248], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [927334400], 12, [927334400], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [929431552], 12, [929431552], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [931528704], 12, [931528704], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [933625856], 12, [933625856], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [935723008], 12, [935723008], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [937820160], 12, [937820160], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [939917312], 12, [939917312], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [942014464], 12, [942014464], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [944111616], 12, [944111616], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [946208768], 12, [946208768], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [948305920], 12, [948305920], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [950403072], 12, [950403072], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [952500224], 12, [952500224], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [954597376], 12, [954597376], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [956694528], 12, [956694528], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [958791680], 12, [958791680], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [960888832], 12, [960888832], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [962985984], 12, [962985984], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [965083136], 12, [965083136], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [967180288], 12, [967180288], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [969277440], 12, [969277440], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [971374592], 12, [971374592], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [973471744], 12, [973471744], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [975568896], 12, [975568896], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [977666048], 12, [977666048], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [979763200], 12, [979763200], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [981860352], 12, [981860352], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [983957504], 12, [983957504], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [986054656], 12, [986054656], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [988151808], 12, [988151808], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [990248960], 12, [990248960], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [992346112], 12, [992346112], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [994443264], 12, [994443264], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [996540416], 12, [996540416], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [998637568], 12, [998637568], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1000734720], 12, [1000734720], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1002831872], 12, [1002831872], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1004929024], 12, [1004929024], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1007026176], 12, [1007026176], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1009123328], 12, [1009123328], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1011220480], 12, [1011220480], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1013317632], 12, [1013317632], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1015414784], 12, [1015414784], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1017511936], 12, [1017511936], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1019609088], 12, [1019609088], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1021706240], 12, [1021706240], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1023803392], 12, [1023803392], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1025900544], 12, [1025900544], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1027997696], 12, [1027997696], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1030094848], 12, [1030094848], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1032192000], 12, [1032192000], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1034289152], 12, [1034289152], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1036386304], 12, [1036386304], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1038483456], 12, [1038483456], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1040580608], 12, [1040580608], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1042677760], 12, [1042677760], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1044774912], 12, [1044774912], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1046872064], 12, [1046872064], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1048969216], 12, [1048969216], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1051066368], 12, [1051066368], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1053163520], 12, [1053163520], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1055260672], 12, [1055260672], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1057357824], 12, [1057357824], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1059454976], 12, [1059454976], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1061552128], 12, [1061552128], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1063649280], 12, [1063649280], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1065746432], 12, [1065746432], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1067843584], 12, [1067843584], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1069940736], 12, [1069940736], 2097152, 0) = 2097152
[pid 58483] copy_file_range(10, [1072037888], 12, [1072037888], 2097152, 0) = 2097152
[pid 58483] +++ exited with 0 +++
+++ exited with 0 +++

real	0m48.354s
user	0m0.108s
sys	0m1.626s

Comment 13 Fam Zheng 2018-07-18 01:21:01 UTC
I think it is doing what we want. The time may be a result of disk bottleneck.

Can you collect the same for "iscsi host initiator+lv" case?

Comment 14 Tingting Mao 2018-07-18 02:08:04 UTC
(In reply to Fam Zheng from comment #13)
> I think it is doing what we want. The time may be a result of disk
> bottleneck.

It is still a little hard to understand why the time is unexpectedly longer than the result with qemu-kvm-rhel-2.10 (For the tested images, I did the same operation for them.)

> Can you collect the same for "iscsi host initiator+lv" case?

The result is the same as NFS.

# time strace -e trace=copy_file_range -f qemu-img convert -f raw -O qcow2 /dev/vgtest/source /dev/vgtest/target 
strace: Process 41060 attached
[pid 41060] copy_file_range(10, [0], 12, [327680], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [2097152], 12, [2424832], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [4194304], 12, [4521984], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [6291456], 12, [6619136], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [8388608], 12, [8716288], 2097152, 0) = 2097152
......
......
......
[pid 41060] copy_file_range(10, [2134900736], 12, [2135425024], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [2136997888], 12, [2137522176], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [2139095040], 12, [2139619328], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [2141192192], 12, [2141716480], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [2143289344], 12, [2143813632], 2097152, 0) = 2097152
[pid 41060] copy_file_range(10, [2145386496], 12, [2145910784], 2096640, 0) = 2096640
[pid 41060] copy_file_range(10, [2147483136], 12, [2148007424], 512, 0) = 512
[pid 41060] +++ exited with 0 +++
+++ exited with 0 +++

real	1m37.363s
user	0m0.072s
sys	0m3.137s

Comment 15 Fam Zheng 2018-07-18 02:49:58 UTC
(In reply to timao from comment #14)
> (In reply to Fam Zheng from comment #13)
> > I think it is doing what we want. The time may be a result of disk
> > bottleneck.
> 
> It is still a little hard to understand why the time is unexpectedly longer
> than the result with qemu-kvm-rhel-2.10 (For the tested images, I did the
> same operation for them.)

Your disk is slow, but your network is fast. The old QEMU uses read/write and cache, and the new uses copy offloading, which cannot use host page cache, that's why the time is longer. How does it compare if you add "-t none" to the command line?

Comment 16 Tingting Mao 2018-07-24 10:35:03 UTC
Result of regression test for 'convert':

For raw
new bug:
Bug 1607774 - Target files for 'qemu-img convert' do not support thin_provisoning with iscsi backend

For qcow2
new bugs(2):
Bug 1601871 - Target image needs larger size when executing convert with iscsi backend 
Bug 1605061 - preallocation mode(full and falloc) is not supported when creating qcow2 file with iscsi backend

Comment 17 Tingting Mao 2018-07-24 10:44:09 UTC
Hi Fam,

According to the comment 9, comment 12, comment 14 and comment 16, is it enough to set the bug as verified? Thanks in advance.

Best,
Tingting Mao

Comment 18 Fam Zheng 2018-07-26 01:21:53 UTC
Please run qemu-img regression tests as well then this can be verified.

Comment 19 Tingting Mao 2018-08-13 06:16:35 UTC
Hi Fam,

After run qcow2+nfs(v4, soft), qcow2+libiscsi(iscsi://), qcow2+iscsi(local+lvs), raw+nfs(v4, soft), raw+libsicsi(iscsi://) and raw+iscsi(local+lvs), besides the new reported bugs in comment 16, there is two new bugs listed below. So is the test enough to set the bug as verified now? Thanks in advance. 

Bug 1613215 - qemu-img shouldn't truncate image with the invalid option value
Bug 1601310 - qemu-img map 'Aborted (core dumped)' when specifying a plain file

Comment 20 Fam Zheng 2018-08-14 07:20:30 UTC
Yes. Thanks.

Comment 21 Tingting Mao 2018-08-14 07:23:55 UTC
According comment 9, comment 12, comment 14, comment 16 and comment 19, set the bug as verified.

Comment 22 Yaniv Kaul 2018-08-29 13:06:16 UTC
(In reply to Tingting Mao from comment #21)
> According comment 9, comment 12, comment 14, comment 16 and comment 19, set
> the bug as verified.

Is the performance better now than before of the qemu-img command?

Comment 23 Tingting Mao 2018-08-29 15:01:12 UTC
(In reply to Yaniv Kaul from comment #22)
> (In reply to Tingting Mao from comment #21)
> > According comment 9, comment 12, comment 14, comment 16 and comment 19, set
> > the bug as verified.
> 
> Is the performance better now than before of the qemu-img command?

For libiscsi, it indeed improved. While for iscsi(local + lvs) and nfs backend, it seems not. For more details, you could refer to comment 9 and comment 13.

Comment 24 Yaniv Kaul 2018-08-29 15:32:15 UTC
(In reply to Tingting Mao from comment #23)
> (In reply to Yaniv Kaul from comment #22)
> > (In reply to Tingting Mao from comment #21)
> > > According comment 9, comment 12, comment 14, comment 16 and comment 19, set
> > > the bug as verified.
> > 
> > Is the performance better now than before of the qemu-img command?
> 
> For libiscsi, it indeed improved. While for iscsi(local + lvs) and nfs
> backend, it seems not. For more details, you could refer to comment 9 and
> comment 13.

So we need specific bugs tracking those performance regressions - either we allow to disable this feature, or we fix the performance regression?

Comment 25 Fam Zheng 2018-08-30 01:55:04 UTC
In bug 1607774 we've disabled it by default. It's done because of behavioral change on sparsifying.

Comment 26 errata-xmlrpc 2018-11-01 11:01:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:3443


Note You need to log in before you can comment on or make changes to this bug.