Bug 2057490

Summary:	nbdkit crashes in VDDK
Product:	Red Hat Enterprise Linux 9	Reporter:	Xiaodai Wang <xiaodwan>
Component:	gnutls	Assignee:	Daiki Ueno <dueno>
Status:	CLOSED ERRATA	QA Contact:	Alexander Sosedkin <asosedki>
Severity:	high	Docs Contact:
Priority:	high
Version:	9.0	CC:	asosedki, dueno, eblake, juzhou, lersek, mxie, rjones, tyan, tzheng, virt-maint, vwu
Target Milestone:	rc	Keywords:	Automation, Regression, TestBlocker, Triaged
Target Release:	---	Flags:	pm-rhel: mirror+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	gnutls-3.7.3-9.el9	Doc Type:	No Doc Update
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2022-05-17 15:52:13 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Xiaodai Wang 2022-02-23 13:40:30 UTC

Description of problem:
nbdkit crashes in VDDK

Version-Release number of selected component (if applicable):
virt-v2v-1.45.99-1.el9.x86_64
nbdkit-server-1.28.5-1.el9.x86_64

How reproducible:
100%

Steps to Reproduce:
1. A minimal reproducer from rjones:

# cd /tmp
# qemu-img create -f vmdk test.vmdk 1G
# nbdkit -fv vddk libdir=/root/vddk_libdir/latest /tmp/test.vmdk

It crashes in VDDK.  The output from VDDK is quite lengthy, I'm not
sure how much is that interesting.

The stack trace with debug symbols is:

#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x00007f9175bc874f in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#2  0x00007f9175bc8a89 in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#3  0x00007f9175ba1273 in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#4  0x00007f9175a8a0bc in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#5  0x00007f9175a8b14a in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDis--Type <RET> for more, q to quit, c to continue without paging--
kLib.so.7
#6  0x00007f9175a997cf in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#7  0x00007f9175a58183 in ?? () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#8  0x00007f9175a5a261 in VixDiskLib_InitEx () from /root/vddk_libdir/vddklib_1/lib64/libvixDiskLib.so.7
#9  0x00007f9175cfb6bc in vddk_after_fork () at /usr/src/debug/nbdkit-1.28.5-1.el9.x86_64/plugins/vddk/vddk.c:547
#10 0x000055879e4e229f in plugin_after_fork (b=0x5587a784fdc0) at /usr/src/debug/nbdkit-1.28.5-1.el9.x86_64/server/plugins.c:273
#11 0x000055879e4dc40f in start_serving () at /usr/src/debug/nbdkit-1.28.5-1.el9.x86_64/server/main.c:984
#12 main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/nbdkit-1.28.5-1.el9.x86_64/server/main.c:745

This is VDDK 7.0.2.

I uploaded VDDK 7.0.3 to this machine (/root/vddk-7.0.3) and
unfortunately that also crashes.

2. For QE, just run virt-v2v with -it vddk to reproduce it.
# virt-v2v -ic vpx://root.x.x/data/x.x.x.x/?no_verify=1 -o null esx6.7-rhel8.5-x86_64 -it vddk -io vddk-libdir=/root/vddk_libdir/latest -io vddk-thumbprint=xxx -ip /root/vpx_pwd

Actual results:
nbdkit crashes in vddk.

Expected results:
v2v should run successfully.

Additional info:

Comment 3 Richard W.M. Jones 2022-02-23 14:02:07 UTC

It's caused by the update to gnutls (bug 2033220).  This causes a
new chain of dependencies to appear which pulls in libcrypto.so.3:

  nbdkit -> libgnutls.so.30 -> libtss2-esys.so.0 -> libcrypto.so.3

Since VDDK bundles its own incompatible libcrypto.so.1 this causes
symbols to conflict and general chaos.

libtss2 is something to do with the TPM (package name: tpm2-tss)

Comment 4 Richard W.M. Jones 2022-02-24 09:11:14 UTC

Reassigning to gnutls since the proposed solution requires a change
to gnutls: https://gitlab.com/gnutls/gnutls/-/merge_requests/1544

Comment 9 Xiaodai Wang 2022-02-25 01:15:53 UTC

The fix also works for me:

# virt-v2v -ic vpx://root.x.x/data/x.x.x.x/?no_verify=1 -o null esx6.7-rhel8.5-x86_64 -it vddk -io vddk-libdir=/root/vddk_libdir/latest -io vddk-thumbprint=1F:97:34:5F:B6:C2:BA:66:46:CB:1A:71:76:7D:6B:50:1E:03:00:EA -ip /root/vpx_pwd 
[   0.0] Setting up the source: -i libvirt -ic vpx://root.x.x/data/x.x.x.x/?no_verify=1 -it vddk esx6.7-rhel8.5-x86_64
[   1.9] Opening the source
[  11.1] Inspecting the source
[  22.9] Checking for sufficient free disk space in the guest
[  22.9] Converting Red Hat Enterprise Linux 8.5 (Ootpa) to run on KVM
virt-v2v: This guest has virtio drivers installed.
[ 147.4] Mapping filesystem data to avoid copying unused and blank areas
[ 150.0] Closing the overlay
[ 150.3] Assigning disks to buses
[ 150.3] Checking if the guest needs BIOS or UEFI to boot
[ 150.3] Setting up the destination: -o null
[ 151.9] Copying disk 1/1
█ 100% [****************************************]
[ 360.5] Creating output metadata
[ 360.5] Finishing off

Comment 15 errata-xmlrpc 2022-05-17 15:52:13 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (new packages: gnutls), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:3937