Bug 2011709
| Summary: | Rebase to nbdkit 1.28 before RHEL 9 GA | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 9 | Reporter: | Richard W.M. Jones <rjones> |
| Component: | nbdkit | Assignee: | Richard W.M. Jones <rjones> |
| Status: | CLOSED ERRATA | QA Contact: | mxie <mxie> |
| Severity: | high | Docs Contact: | |
| Priority: | medium | ||
| Version: | 9.0 | CC: | chhu, eblake, juzhou, kkiwi, lersek, mxie, rjones, tyan, tzheng, virt-maint, xiaodwan |
| Target Milestone: | rc | Keywords: | Triaged |
| Target Release: | --- | Flags: | pm-rhel:
mirror+
|
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | nbdkit-1.28.5-1.el9 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2022-05-17 12:50:41 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1999559, 2018463, 2029751 | ||
|
Description
Richard W.M. Jones
2021-10-07 07:35:12 UTC
New version is nbdkit-1.28.1-1.el9 Note this is a rebase along the upstream stable branch and so contains only bug fixes (except for the new feature added in bug 2018463). Setting Verified Tested to move the process along. There is a build which passed the internal tests: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=1776921 nbdkit-1.28.3-1.el9 This is a simple rebase along the stable branch, containing only bug fixes: https://gitlab.com/nbdkit/nbdkit/-/commits/stable-1.28/ nbdkit-1.28.4-1.el9 This is another simple rebase along the stable branch, bringing only bug fixes: https://gitlab.com/nbdkit/nbdkit/-/commits/stable-1.28/ As before this is an update along the stable branch bringing in a few more upstream fixes before RHEL 9.0. Verify the bug with nbdkit-1.28.5-1.el9.x86_64
Steps:
1.Rebuild nbdkit rpm packages from src package
$ rpmbuild --rebuild nbdkit-1.28.5-1.el9.src.rpm
....
libtool: warning: remember to run 'libtool --finish /usr/lib64/nbdkit/filters'
....
+ exit 0
2.Check if '-D cow.verbose=1' can restore the old debug output
2.1 # nbdkit -fv --filter=cow memory 10k --run 'nbdsh -u $uri -c "h.pwrite(bytearray(1024), 0)"' |& grep 'debug: cow: blk'
2.2 # nbdkit -fv --filter=cow memory 10k --run 'nbdsh -u $uri -c "h.pwrite(bytearray(1024), 0)"' -D cow.verbose=1 |& grep 'debug: cow: blk'
nbdkit: memory.0: debug: cow: blk_read_multiple block 0 (offset 0) run of length 1 is not allocated
nbdkit: memory.0: debug: cow: blk_write block 0 (offset 0)
3.Check default block size for nbdkit-cache-filter and nbdkit-cow-filter
# man nbdkit-cache-filter |grep 64K -B 3
cache-min-block-size=SIZE
Set the minimum block size used by the cache. This must be a power of 2 and ≥ 4096.
The default is 64K, or the block size of the filesystem which contains the temporary file storing the cache (whichever
# man nbdkit-cow-filter |grep 64K -B 4
cow-block-size=N
Set the block size used by the filter. This has to be a power of two and the minimum block size is 4K. The maximum
block size depends on the plugin, but a block size larger than a few megabytes is not usually a good idea.
The default is 64K.
4.Convert a guest with vddk.stats=1 by nbdkit
# # nbdkit --exit-with-parent --threads 16 -D nbdkit.backend.datapath=0 -D vddk.stats=1 -D vddk.datapath=0 --verbose --filter multi-conn --filter cow --filter cacheextents --filter retry vddk server=10.73.198.169 vm=moref=vm-6306 file='[esx7.0-matrix] esx7.0-rhel8.5-x86_64_1/esx7.0-rhel8.5-x86_64.vmdk' user=root libdir=/home/vddk7.0.2 thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78 multi-conn-mode=disable cow-block-size=4096 cow-on-read=/tmp/v2v.6l3Qwq/convert password=+/home/passwd --run 'qemu-img convert $nbd /home/esx7.0-rhel8.5-x86_64'
.....
nbdkit: debug: VDDK function stats (-D vddk.stats=1):
nbdkit: debug: VixDiskLib_... µs calls bytes
nbdkit: debug: Exit 1002565 1
nbdkit: debug: ConnectEx 707986 1
nbdkit: debug: ReadAsync 385188 22530 17209233408
nbdkit: debug: Open 286491 1
nbdkit: debug: InitEx 129163 1
nbdkit: debug: QueryAllocatedBlocks 35957 19
nbdkit: debug: Disconnect 4481 1
nbdkit: debug: GetInfo 3411 1
nbdkit: debug: Close 2472 1
nbdkit: debug: FreeBlockList 324 19
nbdkit: debug: Wait 37 1
nbdkit: debug: FreeInfo 28 1
nbdkit: debug: FreeConnectParams 23 1
nbdkit: debug: GetTransportMode 18 1
nbdkit: debug: AllocateConnectParams 9 1
nbdkit: debug: retry: unload filter
nbdkit: debug: cacheextents: unload filter
nbdkit: debug: cow: unload filter
nbdkit: debug: multi-conn: unload filter
Hi Richard,
nbdkit 1.28 release note said 'nbdkit-cow-filter...In addition the default block size for both filters is now 64K. Both changes greatly improve performance', my understanding is 64k should be the best size for performance for cow filter，but I found v2v use 'cow-block-size=4096' in conversion, is it expected?
# cat virt-v2v-1.45.97-2.log |grep 'cow-block-size=4096'
Jan 26 11:16:41 LANG=C 'nbdkit' '--exit-with-parent' '--foreground' '--pidfile' '/tmp/v2vnbdkit.P1jGNB/nbdkit1.pid' '--unix' '/tmp/v2v.UgyuQD/in0' '--threads' '16' '--selinux-label' 'system_u:object_r:svirt_socket_t:s0' '-D' 'nbdkit.backend.datapath=0' '-D' 'vddk.stats=1' '-D' 'vddk.datapath=0' '--verbose' '--filter' 'multi-conn' '--filter' 'cow' '--filter' 'cacheextents' '--filter' 'retry' 'vddk' 'server=10.73.198.169' 'vm=moref=vm-6306' 'file=[esx7.0-matrix] esx7.0-rhel8.5-x86_64_1/esx7.0-rhel8.5-x86_64.vmdk' 'user=vsphere.local\Administrator' 'libdir=/home/vddk7.0.2' 'thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78' 'multi-conn-mode=disable' 'cow-block-size=4096' 'cow-on-read=/tmp/v2v.UgyuQD/convert' 'password=+/home/passwd'
> nbdkit 1.28 release note said 'nbdkit-cow-filter...In addition the default block size for both filters is now 64K. Both changes greatly improve performance', my understanding is 64k should be the best size for performance for cow filter，but I found v2v use 'cow-block-size=4096' in conversion, is it expected? The release notes for 1.28 were wrong. The optimal block size for nbdkit-cow-filter depends on how the filter is used, but for our use case in virt-v2v using a smaller block size gave better performance by avoiding read-modify-write cycles in the fstrim step: https://github.com/libguestfs/virt-v2v/commit/351d61f768c51287917a04b9fbedf24d79f5deb4 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (new packages: nbdkit), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2022:2408 |