Bug 1906049

We might rebase to nbdkit 1.24, to be released soon.

I'm not clear when the rebase deadline is, since I can only see
"Rebase Deadline - libvirt 7.0" in the schedule.

Release note for 1.24 showing what has changed since 1.22:

https://libguestfs.org/nbdkit-release-notes-1.24.1.html

Verify bug with below builds:
nbdkit-ssh-plugin-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-basic-filters-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-curl-plugin-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-server-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-vddk-plugin-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-basic-plugins-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64
nbdkit-python-plugin-1.24.0-1.module+el8.4.0+9341+96cf2672.x86_64


Steps:
1.Test if nbdkit is a vulnerable version. Result: nbdkit package has no vulnerable
# nbdkit -fv null --run 'sleep 1 >/dev/tcp/localhost/10809' 2>&1 | grep -q 'open readonly' && echo vulnerable || echo patched
patched


2.Build nbdkit packages from src package.Result: nbdkit rpm packages can be built from src package successfully

2.1 Download nbdkit src package and create .rpmmacros file 
$ cat .rpmmacros
%_topdir  %(echo $HOME)/rpmbuild
%_smp_mflags -j5

2.2 Resolve the dependence problem before rebuilding,then build nbdkit rpm packages from src package
# rpmbuild --rebuild rpmbuild --rebuild nbdkit-1.24.0-1.module+el8.4.0+9341+96cf2672.src.rpm
.....
+ exit 0


3.Test option '-D nbdkit.backend.controlpath=0 and -D nbdkit.backend.datapath=0 in nbdkit ssh plugin. Result: these two options can suppress some debug message successfully
# nbdkit -f -v -U - ssh host=10.73.224.33 /var/lib/xen/images/xen-hvm-rhel7.8-x86_64.img --run 'qemu-img convert -p -f raw $nbd /var/tmp/xen-hvm-rhel7.8.img' --filter=retry   -D nbdkit.backend.controlpath=0  -D nbdkit.backend.datapath=0
nbdkit: debug: nbdkit 1.24.0 (nbdkit-1.24.0-1.module+el8.4.0+9341+96cf2672)
nbdkit: debug: TLS disabled: could not load TLS certificates
nbdkit: debug: registering plugin /usr/lib64/nbdkit/plugins/nbdkit-ssh-plugin.so
nbdkit: debug: registered plugin /usr/lib64/nbdkit/plugins/nbdkit-ssh-plugin.so (name ssh)
nbdkit: debug: ssh: load
nbdkit: debug: registering filter /usr/lib64/nbdkit/filters/nbdkit-retry-filter.so
nbdkit: debug: registered filter /usr/lib64/nbdkit/filters/nbdkit-retry-filter.so (name retry)
nbdkit: debug: retry: load
nbdkit: debug: retry: config key=host, value=10.73.224.33
nbdkit: debug: ssh: config key=host, value=10.73.224.33
nbdkit: debug: retry: config key=path, value=/var/lib/xen/images/xen-hvm-rhel7.8-x86_64.img
nbdkit: debug: ssh: config key=path, value=/var/lib/xen/images/xen-hvm-rhel7.8-x86_64.img
nbdkit: debug: retry: config_complete
nbdkit: debug: ssh: config_complete
nbdkit: debug: using thread model: serialize_requests
nbdkit: debug: retry: get_ready thread_model=2
nbdkit: debug: ssh: get_ready
nbdkit: debug: bound to unix socket /tmp/nbdkitk6lH0E/socket
nbdkit: debug: forked into background (new pid = 31747)
nbdkit: debug: retry: after_fork
nbdkit: debug: ssh: after_fork
nbdkit: debug: accepted connection
nbdkit: ssh[1]: debug: retry: preconnect
nbdkit: ssh[1]: debug: ssh: preconnect
nbdkit: ssh[1]: debug: newstyle negotiation: flags: global 0x3
nbdkit: ssh[1]: debug: newstyle negotiation: client flags: 0x3
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_STRUCTURED_REPLY: client requested structured replies
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: client requested export ''
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set count: 1
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: set base:allocation
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: replying with base:allocation id 1
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_SET_META_CONTEXT: reply complete
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_GO: client requested export ''
nbdkit: ssh[1]: debug: authentication methods offered by the server: 0x26 password publickey gssapi-with-mic
nbdkit: ssh[1]: debug: opened libssh handle
nbdkit: ssh[1]: debug: newstyle negotiation: flags: export 0x8c9
nbdkit: ssh[1]: debug: newstyle negotiation: NBD_OPT_GO: ignoring NBD_INFO_* request 3 (NBD_INFO_BLOCK_SIZE)
nbdkit: ssh[1]: debug: handshake complete, processing requests serially
    (100.00/100%)
nbdkit: ssh[1]: debug: client sent NBD_CMD_DISC, closing connection
nbdkit: ssh[1]: debug: reopens needed: 0


4.Testing nbdkit with annobin. Result: there is no FAIL result in below output

# annocheck -v --skip-cf-protection --skip-glibcxx-assertions --skip-glibcxx-assertions --skip-stack-realign --section-size=.gnu.build.attributes --ignore-gaps nbdkit-server-1.24.0-1.el8.x86_64.rpm --debug-rpm=nbdkit-server-debuginfo-1.24.0-1.el8.x86_64.rpm
annocheck: Version 9.35.
Hardened: ./usr/sbin/nbdkit: info: Set binary producer to gcc version 8.
Section Size: ./usr/sbin/nbdkit: .gnu.build.attributes: 0x5dc
Hardened: ./usr/sbin/nbdkit: WARN: DW_FORM_GNU_strp_alt not yet handled.
Hardened: ./usr/sbin/nbdkit: WARN: DW_FORM_GNU_strp_alt not yet handled.
Hardened: ./usr/sbin/nbdkit: WARN: DW_FORM_GNU_strp_alt not yet handled.
Hardened: ./usr/sbin/nbdkit: WARN: DW_FORM_GNU_strp_alt not yet handled.
Hardened: ./usr/sbin/nbdkit: WARN: DW_FORM_GNU_strp_alt not yet handled.
Hardened: ./usr/sbin/nbdkit: WARN: DW_FORM_GNU_strp_alt not yet handled.
Hardened: ./usr/sbin/nbdkit: skip: Test for -Wl,-z-now.  (Dynamic segment present, but no dynamic relocations found).
Hardened: ./usr/sbin/nbdkit: skip: Branch protection.  (Not an AArch64 binary).
Hardened: ./usr/sbin/nbdkit: skip: Compiled with -fcf-protection=all (x86 only, gcc 8+ only).
Hardened: ./usr/sbin/nbdkit: PASS: One dynamic section/segment found.
Hardened: ./usr/sbin/nbdkit: skip: Test of dynamic tags.  (AArch64 specific).
Hardened: ./usr/sbin/nbdkit: PASS: Entry point instruction is ENDBR.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled with -D_FORTIFY_SOURCE=2.
Hardened: ./usr/sbin/nbdkit: skip: Compiled with -D_GLIBCXX_ASSERTIONS.
Hardened: ./usr/sbin/nbdkit: skip: Test for -Wl,-z,relro.  (No dynamic relocations).
Hardened: ./usr/sbin/nbdkit: PASS: Stack not executable.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled with sufficient optimization.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled with PIC/PIE.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled as a position independent binary.
Hardened: ./usr/sbin/nbdkit: PASS: Good GNU Property note.
Hardened: ./usr/sbin/nbdkit: PASS: DT_RPATH/DT_RUNPATH absent or rooted at /usr.
Hardened: ./usr/sbin/nbdkit: PASS: No RWX segments found.
Hardened: ./usr/sbin/nbdkit: PASS: Consistent use of the -fshort-enum option.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled with -fstack-clash-protection.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled with sufficient stack protection.
Hardened: ./usr/sbin/nbdkit: skip: Compiled with -mstackrealign (i686 only).
Hardened: ./usr/sbin/nbdkit: PASS: No text relocations found.
Hardened: ./usr/sbin/nbdkit: PASS: No thread cancellation problems.
Hardened: ./usr/sbin/nbdkit: PASS: Compiled with either -Wall and/or -Wformat-security.
Hardened: ./usr/sbin/nbdkit: PASS: GOT/PLT relocations are read only.
Hardened: ./usr/sbin/nbdkit: PASS.
Section Size: Section '.gnu.build.attributes' found in 1 files, total size: 0x5dc


5.Use nbdkit to access VMware to convert guest, test new option compression. Result: conversions can finish without error

5.1 #nbdkit -rfv -U - --exportname --filter=cacheextents --filter=retry vddk server=10.73.198.169 user=root password=+/home/passwd vm=moref=vm-3069       file='[esx7.0-matrix] esx7.0-sles15sp2-x86_64/esx7.0-sles15sp2-x86_64.vmdk' libdir=/home/vddk6.7  thumbprint='B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78' transports=nbd  compression=skipz --run 'qemu-img convert $nbd /var/tmp/esx7.0-sles15sp2-skipz'

5.1.1 # qemu-img info /var/tmp/esx7.0-sles15sp2-skipz
image: /var/tmp/esx7.0-sles15sp2-skipz
file format: raw
virtual size: 8 GiB (8589934592 bytes)
disk size: 6.04 GiB

5.2 # nbdkit -rfv -U - --exportname --filter=cacheextents --filter=retry vddk server=10.73.198.169 user=root password=+/home/passwd vm=moref=vm-2490        file='[esx7.0-function] esx7.0-rhel8.3-x86_64-efi-secure/esx7.0-rhel8.3-x86_64-efi-secure.vmdk' libdir=/home/vddk6.7  thumbprint='B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78' transports=nbd  compression=fastlz --run 'qemu-img convert $nbd /var/tmp/esx7.0-rhel8.3-efi-fastlz'

5.2.1# qemu-img info /var/tmp/esx7.0-rhel8.3-efi-fastlz 
image: /var/tmp/esx7.0-rhel8.3-efi-fastlz
file format: raw
virtual size: 20 GiB (21474836480 bytes)
disk size: 9.66 GiB

5.3 # nbdkit -rfv -U - --exportname --filter=cacheextents --filter=retry vddk server=10.73.198.169 user=root password=+/home/passwd vm=moref=vm-2408        file='[esx7.0-matrix] esx7.0-win2019-x86_64/esx7.0-win2019-x86_64.vmdk' libdir=/home/vddk7.0  thumbprint='B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78' transports=nbd  compression=zlib --run 'qemu-img convert $nbd /var/tmp/esx7.0-win2019-zlib'

5.3.1 # qemu-img info /var/tmp/esx7.0-win2019-zlib 
image: /var/tmp/esx7.0-win2019-zlib
file format: raw
virtual size: 20 GiB (21474836480 bytes)
disk size: 19.1 GiB

5.4 # nbdkit -rfv -U - --exportname --filter=cacheextents --filter=retry vddk server=10.73.198.169 user=root password=+/home/passwd vm=moref=vm-2408        file='[esx7.0-matrix] esx7.0-win2019-x86_64/esx7.0-win2019-x86_64.vmdk' libdir=/home/vddk6.5  thumbprint='B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78' transports=nbd  compression=none --run 'qemu-img convert $nbd /var/tmp/esx7.0-win2019-none'

5.4.1 # qemu-img info /var/tmp/esx7.0-win2019-none
image: /var/tmp/esx7.0-win2019-none
file format: raw
virtual size: 20 GiB (21474836480 bytes)
disk size: 19.1 GiB

6. Convert guest from VMware to rhv by v2v, the conversion can finish without error
# virt-v2v -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 -it vddk -io vddk-libdir=/home/vddk7.0 -io  vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78  -o rhv-upload -of qcow2 -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -ip /home/passwd -op /home/rhvpasswd  -os nfs_data_8_4 -n ovirtmgmt  esx7.0-win2016-x86_64-vmware-tool -oo rhv-cluster=rhel8.4
[   0.4] Opening the source -i libvirt -ic vpx://root.198.169/data/10.73.199.217/?no_verify=1 esx7.0-win2016-x86_64-vmware-tool -it vddk  -io vddk-libdir=/home/vddk7.0 -io vddk-thumbprint=B5:52:1F:B4:21:09:45:24:51:32:56:F6:63:6A:93:5D:54:08:2D:78
[   2.1] Creating an overlay to protect the source from being modified
[   2.8] Opening the overlay
libvirt needs authentication to connect to libvirt URI qemu:///system
(see also: http://libvirt.org/auth.html http://libvirt.org/uri.html)
Please enter your authentication name: test
Please enter your password: 
[  11.9] Inspecting the overlay
[  15.8] Checking for sufficient free disk space in the guest
[  15.8] Estimating space required on target for each disk
[  15.8] Converting Windows Server 2016 Standard to run on KVM
virt-v2v: warning: /usr/share/virt-tools/pnp_wait.exe is missing.  
Firstboot scripts may conflict with PnP.
libvirt needs authentication to connect to libvirt URI qemu:///system
(see also: http://libvirt.org/auth.html http://libvirt.org/uri.html)
Please enter your authentication name: test
Please enter your password: 
libvirt needs authentication to connect to libvirt URI qemu:///system
(see also: http://libvirt.org/auth.html http://libvirt.org/uri.html)
Please enter your authentication name: test
Please enter your password: 
virt-v2v: warning: there is no QXL driver for this version of Windows (10.0 
x86_64).  virt-v2v looks for this driver in 
/usr/share/virtio-win/virtio-win.iso

The guest will be configured to use a basic VGA display driver.
virt-v2v: This guest has virtio drivers installed.
[  91.9] Mapping filesystem data to avoid copying unused and blank areas
[  92.6] Closing the overlay
[  93.5] Assigning disks to buses
[  93.5] Checking if the guest needs BIOS or UEFI to boot
[  93.5] Initializing the target -o rhv-upload -oc https://dell-per740-22.lab.eng.pek2.redhat.com/ovirt-engine/api -op /home/rhvpasswd -os nfs_data_8_4
[  94.7] Copying disk 1/1 to qemu URI json:{ "file.driver": "nbd", "file.path": "/tmp/v2vnbdkit.L3avp4/nbdkit4.sock", "file.export": "/" } (qcow2)
    (100.00/100%)
[1014.8] Creating output metadata
[1016.8] Finishing off


Result:
    No problem was found，so move the bug from ON_QA to VERIFIED

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (virt:av bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:2098