2059289 – RFE: Rebase nbdkit to 1.30 in RHEL 9.1

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2059289 - RFE: Rebase nbdkit to 1.30 in RHEL 9.1

Summary: RFE: Rebase nbdkit to 1.30 in RHEL 9.1

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 9
Classification:	Red Hat
Component:	nbdkit
Sub Component:
Version:	9.1
Hardware:	Unspecified
OS:	Unspecified
Priority:	unspecified
Severity:	unspecified
Target Milestone:	rc
Target Release:	---
Assignee:	Richard W.M. Jones
QA Contact:	mxie@redhat.com
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+	depends on / blocked

Reported:	2022-02-28 16:30 UTC by Richard W.M. Jones
Modified:	2022-11-15 10:07 UTC (History)
CC List:	10 users (show)
Fixed In Version:	nbdkit-1.30.8-1.el9
Doc Type:	If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:	2022-11-15 09:50:17 UTC
Type:	Feature Request
Target Upstream Version:
Embargoed:
Dependent Products:
Flags:	pm-rhel: mirror+

Attachments	(Terms of Use)

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Issue Tracker	RHELPLAN-114034	0	None	None	None	2022-02-28 16:37:42 UTC
Red Hat Product Errata	RHBA-2022:7945	0	None	None	None	2022-11-15 09:50:40 UTC

Description Richard W.M. Jones 2022-02-28 16:30:05 UTC

Description of problem:

This is a request to rebase nbdkit to the latest
upstream stable release in RHEL 9.1.

The upstream release is 1.30.

Release notes are available outlining the main changes:
https://libguestfs.org/nbdkit-release-notes-1.30.1.html

Comment 5 mxie@redhat.com 2022-03-29 16:00:49 UTC

Test the bug with nbdkit-1.30.1-2.el9.x86_64

Steps:
1.Rebuild nbdkit rpm packages from src package when vhost_vsock module is not loaded(bug2069558)
$ rpmbuild --rebuild nbdkit-1.30.1-2.el9.src.rpm 
....
+ exit 0

2.Check if there are newly added or removed filters and plugins after comparing with nbdkit-1.28.5
Newly added: nbdkit-retry-request-filter, nbdkit-protect-filter, nbdkit-blocksize-policy-filter

3.Check new options of nbdkit-vddk-plugin
3.1 Check new create=true option and create-type option
3.1.1 # man nbdkit-vddk-plugin  |grep create
.....
       create=true
           Create a new, local VMDK file.  Instead of opening an existing VMDK file, a new VMDK file is created and
           not possible to create a remote file using nbdkit.
           If this is used, the "create-size" parameter is required to specify the virtual size of the disk.  Other
           "create-*" parameters (see below) can be used to control the VMDK sub-format.
       create-adapter-type=ide
       create-adapter-type=scsi-buslogic
       create-adapter-type=scsi-lsilogic
       create-hwversion=workstation4
       create-hwversion=workstation5
       create-hwversion=workstation6
       create-hwversion=esx30
       create-hwversion=esx4x
       create-hwversion=esx50
       create-hwversion=esx51
       create-hwversion=esx55
       create-hwversion=esx60
       create-hwversion=esx65
       create-hwversion=N
       create-size=SIZE
           Specify the virtual size of the created disk.  The "SIZE" can use modifiers like "100M" etc.  It must be a
           If you use "create=true" then this parameter is required.
       create-type=monolithic-sparse
       create-type=monolithic-flat
       create-type=split-sparse
       create-type=split-flat
       create-type=vmfs-flat
       create-type=stream-optimized
       create-type=vmfs-thin
       create-type=vmfs-sparse

3.1.2 
# nbdkit vddk file='vddk-create-vmdk-vmfs-flat' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=1000M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-vmfs-flat'

# qemu-img info /home/vddk-create-vmdk-vmfs-flat
image: /home/vddk-create-vmdk-vmfs-flat
file format: raw
virtual size: 0.977 GiB (1048576000 bytes)
disk size: 1 MiB

3.1.3 
#nbdkit vddk file='vddk-create-vmdk-vmfs-sparse' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=vmfs-sparse libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-vmfs-sparse'
nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-sparse: The specified feature is not supported by this version
qemu-img: Could not open 'nbd://localhost:10809': Requested export not available

3.1.4 
#  nbdkit vddk file='vddk-create-vmdk-vmfs-thin' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=vmfs-thin libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-vmfs-thin'
nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-thin: One of the parameters was invalid
qemu-img: Could not open 'nbd://localhost:10809': Requested export not available

3.1.5
#  nbdkit vddk file='vddk-create-vmdk-monolithic-sparse' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=monolithic-sparse libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-monolithic-sparse'

# qemu-img info /home/vddk-create-vmdk-monolithic-sparse
image: /home/vddk-create-vmdk-monolithic-sparse
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.1.6 
#nbdkit vddk file='vddk-create-vmdk-monolithic-flat' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=monolithic-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-monolithic-flat'

# qemu-img info /home/vddk-create-vmdk-monolithic-flat
image: /home/vddk-create-vmdk-monolithic-flat
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.1.7
# nbdkit vddk file='vddk-create-vmdk-split-sparse' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=split-sparse libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-split-sparse'

# qemu-img info /home/vddk-create-vmdk-split-sparse
image: /home/vddk-create-vmdk-split-sparse
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB


3.1.8
#nbdkit vddk file='vddk-create-vmdk-msplit-flat' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=split-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-split-flat'

# qemu-img info /home/vddk-create-vmdk-split-flat
image: /home/vddk-create-vmdk-split-flat
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.1.9
#nbdkit vddk file='vddk-create-vmdk-stream-optimized' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=stream-optimized libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-stream-optimized'

# qemu-img info /home/vddk-create-stream-optimized
image: /home/vddk-create-stream-optimized
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.2 Check new create=true option and create-adapter-type option
3.2.1 create-adapter-type=scsi-buslogic has been tested in step3.1.2 
3.2.2 
# nbdkit vddk file='vddk-create-vmdk-scsi-lsilogic' create=true create-adapter-type=scsi-lsilogic create-hwversion=esx65 create-size=1000M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-scsi-lsilogic'

# qemu-img info /home/vddk-create-vmdk-scsi-lsilogic
image: /home/vddk-create-vmdk-scsi-lsilogic
file format: raw
virtual size: 0.977 GiB (1048576000 bytes)
disk size: 1 MiB

3.2.3 
# nbdkit vddk file='vddk-create-vmdk-ide' create=true create-adapter-type=ide create-hwversion=esx65 create-size=1000M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-ide'

# qemu-img info /home/vddk-create-vmdk-ide
image: /home/vddk-create-vmdk-ide
file format: raw
virtual size: 0.977 GiB (1048576000 bytes)
disk size: 1 MiB


3.3 Test create-hwversion option randomly
3.3.1 create-hwversion=esx65 has been tested in  step3.1.2 
3.3.2 
# nbdkit vddk file='vddk-create-vmdk' create=true create-adapter-type=scsi-lsilogic create-hwversion=esx60 create-size=100M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk'

# qemu-img info /home/vddk-create-vmdk
image: /home/vddk-create-vmdk
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.3.3 
# nbdkit vddk file='vddk-create-vmdk-esx30' create=true create-adapter-type=scsi-lsilogic create-hwversion=esx30 create-size=100M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-esx30'

# qemu-img info /home/vddk-create-vmdk-esx30
image: /home/vddk-create-vmdk-esx30
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.3.4
# nbdkit vddk file='vddk-create-vmdk-workstation4' create=true create-adapter-type=scsi-lsilogic create-hwversion=workstation4 create-size=100M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-workstation4'

# qemu-img info /home/vddk-create-vmdk-workstation4
image: /home/vddk-create-vmdk-workstation4
file format: raw
virtual size: 100 MiB (104857600 bytes)
disk size: 1 MiB

3.3.5
#nbdkit vddk file='vddk-create-vmdk-esx70' create=true create-adapter-type=scsi-lsilogic create-hwversion=esx70 create-size=100M create-type=vmfs-flat libdir=/home/vddk7.0.3 --run 'qemu-img convert $nbd /home/vddk-create-vmdk-esx70'
nbdkit: error: create-hwversion: empty string where we expected a number
nbdkit: error: unknown create-hwversion: esx70


4 Check new info of nbdkit-info-plugin --dump-plugin
# nbdkit info --dump-plugin  |grep address
info_address=yes

5.Check basic function of new nbdkit-retry-request-filter
5.1 Convert a guest from Xen with retry-request filter, disconnect ssh process of nbdkit on Xen server during nbdkit conversion, but the connection will be disconnected after retrying several times
# nbdkit -f -v -U - ssh host=10.73.224.33 /var/lib/xen/images/xen-hvm-rhel7.9-x86_64.img --run 'qemu-img convert -p -f raw $nbd /tmp/out.img' --filter=retry-request 
.....
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 1: waiting 2 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=150994944
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 2: waiting 2 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=150994944
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
.....
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 1: waiting 2 seconds before retrying
qemu-img: error while reading at byte 163577856: Input/output error
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=165675008
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 2: waiting 2 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=165675008
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: sending error reply: Input/output error
qemu-img: error while reading at byte 165675008: Input/output error
....
nbdkit: ssh[1]: debug: client sent NBD_CMD_DISC, closing connection
nbdkit: ssh[1]: debug: retry-request: finalize
nbdkit: ssh[1]: debug: ssh: finalize
nbdkit: ssh[1]: debug: retry-request: close
nbdkit: ssh[1]: debug: ssh: close
nbdkit: ssh[1]: error: cannot close file: Socket error: disconnected
nbdkit: debug: retry-request: cleanup
nbdkit: debug: ssh: cleanup
nbdkit: debug: ssh: unload plugin
nbdkit: debug: retry-request: unload filter

5.2 Convert a guest from Xen with retry-request filter and set 'retry-request-retries=4 retry-request-delay=5 retry-request-open=true' in command line. Disconnect ssh process of nbdkit on Xen server during nbdkit conversion, but the connection won't be disconnected after retrying specified times, the problem has been tracked in bug1819240
# nbdkit -f -v -U - ssh host=10.73.224.33 /var/lib/xen/images/xen-hvm-rhel7.9-x86_64.img --run 'qemu-img convert -p -f raw $nbd /tmp/out.img' --filter=retry-request retry-request-retries=4 retry-request-delay=5 retry-request-open=true
.....
nbdkit: ssh[1]: debug: retry-request: pread count=2097152 offset=297795584
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=297795584
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 1: waiting 5 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=297795584
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 2: waiting 5 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=297795584
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 3: waiting 5 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=297795584
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 4: waiting 5 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=297795584
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: sending error reply: Input/output error
nbdkit: ssh[1]: debug: retry-request: pread count=2097152 offset=299892736
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=299892736
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 1: waiting 5 seconds before retrying
qemu-img: error while reading at byte 297795584: Input/output error
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=299892736
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 2: waiting 5 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=299892736
......

6. Check basic function of new nbdkit-protect-filter
6.1 # fdisk -l /var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda
Disk /var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda: 16 GiB, 17179869184 bytes, 33554432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0d655377

Device                                               Boot   Start      End  Sectors Size Id Type
/var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda1 *       2048  2099199  2097152   1G 83 Linux
/var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda2      2099200 33554431 31455232  15G 8e Linux LVM

6.2 #start=$((2048*512))
6.3 #end=$((2099199*512+511))
6.4 #nbdkit --filter=protect file /var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda protect=~$start-$end
6.5 # guestfish --rw -a /var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda

Welcome to guestfish, the guest filesystem shell for
editing virtual machine filesystems and disk images.

Type: ‘help’ for help on commands
      ‘man’ to read the manual
      ‘quit’ to quit the shell

><fs> run
><fs> list-filesystems
/dev/sda1: xfs
/dev/rhel_bootp-73-199-156/root: xfs
/dev/rhel_bootp-73-199-156/swap: swap
><fs> mount /dev/rhel_bootp-73-199-156/root /
><fs> edit /etc/sysconfig/network-scripts/ifcfg-enp1s0 
><fs> exit


7.Check basic function of new nbdkit-blocksize-policy-filter

7.1 #nbdkit --filter=blocksize-policy memory 1G blocksize-preferred=32K

7.2 # nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 1073741824 (1G)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
	block_size_minimum: 1
	block_size_preferred: 32768
	block_size_maximum: 4294967295

7.3 nbdkit --filter=blocksize-policy --filter=blocksize file /var/lib/libvirt/images/esx7.0-rhel9.0-snapshot-sda blocksize-error-policy=allow blocksize-minimum=64K minblock=64K

7.4 # nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 17179869184 (16G)
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
	block_size_minimum: 65536
	block_size_preferred: 65536
	block_size_maximum: 4294967295


Hi Richard,

   (1) Please help to check the result of step3.1.3, step3.1.4, there is something wrong with create-type=vmfs-sparse and create-type=vmfs-thin option
   (2) Please help to check the result of step3.3.5,  why latest hwversion esx70 is not supported but old hwversion 'esx30, esx4x, esx51,esx55,esx60 are supported?
   (3) Please help to check the result of step5.1, my understanding is retry-request should retry one time, something like below, but retry-request filter retried two times in step5.1, is it expected?

nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 1: waiting 2 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=150994944
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)
nbdkit: ssh[1]: debug: retry 1: waiting 2 seconds before retrying
nbdkit: ssh[1]: debug: ssh: pread count=2097152 offset=150994944
nbdkit: ssh[1]: error: read failed: Socket error: disconnected (-1)

   (4)Please help to check the result of step6.5, I'm not sure if the step6.5 correctly checks step6.4, please give me some suggestions

Comment 6 Richard W.M. Jones 2022-04-04 08:54:59 UTC

>    (1) Please help to check the result of step3.1.3, step3.1.4, there is something wrong with create-type=vmfs-sparse and create-type=vmfs-thin option

There's a problem with the command you used.  If you use [simplified]:

  nbdkit vddk file=foo create=true --run 'qemu-img convert $nbd foo'

then you're setting up a sort of loop copying from the server to its own output file:

  nbdkit  ---> qemu-img convert
    |                 |
    V                 |
  foo <---------------/

If you just want to create the file, use something like “--run 'nbdinfo $uri'” which
will run an external command that connects (which is needed to create the output) but
does not copy over the output file:

$ nbdkit vddk file='vddk-create-vmdk-vmfs-sparse' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=vmfs-sparse libdir=~/tmp/vddk/vmware-vix-disklib-distrib --run 'nbdinfo $uri'
 
That command fails for me with:

nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-sparse: The specified feature is not supported by this version

because the features (both vmfs-thin and vmfs-sparse) really don't seem to be supported
by VDDK, although they are both shown as possible values for the disk type.
https://vdc-repo.vmware.com/vmwb-repository/dcr-public/ae1a858c-2f59-4a75-b333-f2ddfa631fbd/4ae542fa-c7ec-4713-a3a0-6fef938b03d1/doc/vddkDataStruct.5.3.html

So I guess just ignore vmfs-thin and vmfs-sparse.  Maybe they are for some future version
of VDDK or some specialised case that we don't know about.

>    (2) Please help to check the result of step3.3.5,  why latest hwversion esx70 is not supported but old hwversion 'esx30, esx4x, esx51,esx55,esx60 are supported?

There's no hwversion esx70.  It just doesn't exist in current versions of VDDK (even 7.x).

>    (3) Please help to check the result of step5.1, my understanding is retry-request should retry one time, something like below, but retry-request filter retried two times in step5.1, is it expected?

Retry-request filter will only retry a single command (eg. NBD_CMD_READ).  The problem with
ssh is that if the server goes away you have to completely reconnect to the server.  Just
retrying the command will not recover the connection.  We have a different filter (nbdkit-retry-filter)
which does this.

To test nbdkit-retry-request-filter you'll need a plugin where a failing single command can
be recovered by retrying.  As an example, I made a custom plugin where commands fail every
odd second:

$ nbdkit -f eval get_size=' echo 1G ' \
                 pread=' sleep 1; if test $((`date +%s` % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1 ; fi ' 

Copying fails:

$ nbdcopy nbd://localhost /tmp/out
read at offset 262144 failed: Input/output error

If I add the retry-request filter:

$ nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s` % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1 ; fi ' --filter=retry-request retry-request-retries=10 

The same copy command will work (very very slowly), even though you can still see errors printed in
the output of nbdkit.

(more to follow ...)

Comment 7 Richard W.M. Jones 2022-04-04 09:04:28 UTC

>   (4)Please help to check the result of step6.5, I'm not sure if the step6.5 correctly checks step6.4, please give me some suggestions

The test protects /dev/sda1 (the boot partition) of the disk, and then verifies
you can write to another part of the disk.

It might be easier to test this filter more directly, for example like below.

Create a 64K disk containing a test pattern, and protect the first 1K:

$ nbdkit -f --filter=protect data '"AB" * 32768' protect=0-1023

In an nbdsh, try reading and writing.  It should be OK to read any part of
the disk, and write to the region 1K-64K.  It should also be OK to write
the same data back to the protected region.  However trying to change the
protected region should fail.

$ nbdsh -u nbd://localhost

nbd> print(h.pread(512,0))
b'ABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABAB'

nbd> print(h.pread(512,32768))
b'ABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABAB'

nbd> buf=b'01'*256
nbd> h.pwrite(buf,32768)
nbd> print(h.pread(512,32768))
b'01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101'

nbd> h.pwrite(buf,0)
Traceback (most recent call last):
  File "/usr/lib64/python3.10/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/usr/lib64/python3.10/site-packages/nbd.py", line 1719, in pwrite
    return libnbdmod.pwrite(self._o, buf, offset, flags)
nbd.Error: nbd_pwrite: write: command failed: Operation not permitted (EPERM)

## Reading and writing the same data back to the protected region is allowed:

nbd> buf=h.pread(512,0)
nbd> h.pwrite(buf,0)

Comment 8 mxie@redhat.com 2022-07-15 02:06:38 UTC

Verify the bug with nbdkit-1.30.7-1.el9.x86_64

Steps:
1.Rebuild nbdkit rpm packages from src package

$ rpmbuild --rebuild nbdkit-1.30.7-1.el9.src.rpm 
....
+ exit 0


2.Check new options of nbdkit-vddk-plugin

2.1 Check new create=true option and the various create-* options in man page

# man nbdkit-vddk-plugin  |grep create
.....
       create=true
           Create a new, local VMDK file.  Instead of opening an existing VMDK file, a new VMDK file is created and opened.  The filename is
           given by the "file" parameter (see below).  The file must not exist already.  It is not possible to create a remote file using
           If this is used, the "create-size" parameter is required to specify the virtual size of the disk.  Other "create-*" parameters
       create-adapter-type=ide
       create-adapter-type=scsi-buslogic
       create-adapter-type=scsi-lsilogic
       create-hwversion=workstation4
       create-hwversion=workstation5
       create-hwversion=workstation6
       create-hwversion=esx30
       create-hwversion=esx4x
       create-hwversion=esx50
       create-hwversion=esx51
       create-hwversion=esx55
       create-hwversion=esx60
       create-hwversion=esx65
       create-hwversion=N
       create-size=SIZE
           Specify the virtual size of the created disk.  The "SIZE" can use modifiers like "100M" etc.  It must be a multiple of 512 bytes
           If you use "create=true" then this parameter is required.
       create-type=monolithic-sparse
       create-type=monolithic-flat
       create-type=split-sparse
       create-type=split-flat
       create-type=vmfs-flat
       create-type=stream-optimized
       create-type=vmfs-thin
       create-type=vmfs-sparse

2.2 Check create-type=vmfs-sparse
# nbdkit vddk file='vddk-create-vmdk-vmfs-sparse' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=vmfs-sparse libdir=/home/vddk7.0.3 --run 'nbdinfo $uri'
nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-sparse: The specified feature is not supported by this version
nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-sparse: The specified feature is not supported by this version
nbdinfo: : nbd_opt_go: server replied with error to opt_go request: No such file or directory
protocol: newstyle-fixed without TLS

2.3  Check create-type=vmfs-thin
# nbdkit vddk file='vddk-create-vmdk-vmfs-thin' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=vmfs-thin libdir=/home/vddk7.0.3 --run 'nbdinfo $uri'
nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-thin: One of the parameters was invalid
nbdkit: vddk[1]: error: VixDiskLib_Create: vddk-create-vmdk-vmfs-thin: One of the parameters was invalid
nbdinfo: : nbd_opt_go: server replied with error to opt_go request: No such file or directory
protocol: newstyle-fixed without TLS

2.4 Check create-type=stream-optimized, create-adapter-type=scsi-buslogic create-hwversion=esx65
#  nbdkit vddk file='vddk-create-vmdk-stream-optimized' create=true create-adapter-type=scsi-buslogic create-hwversion=esx65 create-size=100M create-type=stream-optimized libdir=/home/vddk7.0.3 --run 'nbdinfo $uri'
protocol: newstyle-fixed without TLS
export="":
	export-size: 104857600 (100M)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: false
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: true
	block_size_minimum: 512
	block_size_preferred: 4096
	block_size_maximum: 4294967295

# ls vddk-create-vmdk-stream-optimized 
vddk-create-vmdk-stream-optimized

2.5 Check create-type=vmfs-flat, create-adapter-type=scsi-lsilogic create-hwversion=esx60
# nbdkit vddk file='vddk-create-vmdk-vmfs-flat' create=true create-adapter-type=scsi-lsilogic create-hwversion=esx60 create-size=100M create-type=vmfs-flat libdir=/home/vddk6.7 --run 'nbdinfo $uri'
protocol: newstyle-fixed without TLS
export="":
	export-size: 104857600 (100M)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: false
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: true
	block_size_minimum: 512
	block_size_preferred: 4096
         block_size_maximum: 4294967295

# ls vddk-create-vmdk-vmfs-flat
vddk-create-vmdk-vmfs-flat       vddk-create-vmdk-vmfs-flat-flat  

2.6 Check create-type=split-flat, create-adapter-type=ide create-hwversion=esx55
#nbdkit vddk file='vddk-create-vmdk-split-flat' create=true create-adapter-type=ide create-hwversion=esx55 create-size=100M create-type=split-flat libdir=/home/vddk6.7 --run 'nbdinfo $uri'
protocol: newstyle-fixed without TLS
export="":
	export-size: 104857600 (100M)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: false
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: true
	block_size_minimum: 512
	block_size_preferred: 4096
	block_size_maximum: 4294967295

# ls vddk-create-vmdk-split-flat
vddk-create-vmdk-split-flat       vddk-create-vmdk-split-flat-f001  

2.7 Check create-type=split-sparse and create-hwversion=esx30
#  nbdkit vddk file='vddk-create-vmdk-split-sparse' create=true create-adapter-type=scsi-lsilogic create-hwversion=esx30 create-size=10M create-type=split-sparse  libdir=/home/vddk7.0.3 --run 'nbdinfo $uri'
protocol: newstyle-fixed without TLS
export="":
	export-size: 10485760 (10M)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: false
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: true
	block_size_minimum: 512
	block_size_preferred: 4096
	block_size_maximum: 4294967295


# ls vddk-create-vmdk-split-sparse
vddk-create-vmdk-split-sparse       vddk-create-vmdk-split-sparse-s001  

2.8 Check create-type=monolithic-flat and create-hwversion=workstation6
#nbdkit vddk file='vddk-create-vmdk-monolithic-flat' create=true create-adapter-type=scsi-lsilogic create-hwversion=workstation6 create-size=10M create-type=monolithic-flat libdir=/home/vddk7.0.3 --run 'nbdinfo $uri'
protocol: newstyle-fixed without TLS
export="":
	export-size: 10485760 (10M)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: false
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: true
	block_size_minimum: 512
	block_size_preferred: 4096
	block_size_maximum: 4294967295

# ls vddk-create-vmdk-monolithic-flat
vddk-create-vmdk-monolithic-flat       vddk-create-vmdk-monolithic-flat-flat  

2.9 Check create-type=monolithic-sparse and create-hwversion=workstation4
# nbdkit vddk file='vddk-create-vmdk-monolithic-sparse' create=true create-adapter-type=scsi-lsilogic create-hwversion=workstation4 create-size=10M create-type=monolithic-sparse libdir=/home/vddk6.5 --run 'nbdinfo $uri'
protocol: newstyle-fixed without TLS
export="":
	export-size: 10485760 (10M)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: false
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: true
	block_size_minimum: 512
	block_size_preferred: 4096
	block_size_maximum: 4294967295

#ls vddk-create-vmdk-monolithic-sparse 
vddk-create-vmdk-monolithic-sparse


3.Check nbdkit-vddk-plugin about async

3.1 # man nbdkit-vddk-plugin |grep async -B 2 -A 1
       "ReadAsync"
       "WriteAsync"
           Same as above, but for asynchronous read and write calls introduced in nbdkit 1.30.  Unfortunately at the moment the amount of
           time spent in these calls is not accounted for correctly.

3.2 Found ReadAsync info in v2v vddk debug log
# cat v2v-vddk.log |grep ReadAsync
nbdkit: debug:   ReadAsync                       120272  5779     10177990656


4.Check scripts/vddk-open.sh 

4.1 Just found vddk-open.sh after rebuilding virt-v2v src package
# find / |grep vddk-open.sh 
/home/mxie/rpmbuild/BUILD/nbdkit-1.30.1/scripts/vddk-open.sh.in
/home/mxie/rpmbuild/BUILD/nbdkit-1.30.1/scripts/vddk-open.sh

4.2 Not found vddk-open.sh in v2v vddk debug log
# cat v2v-vddk.log |grep vddk-open.sh
nothing


5. Check new info of nbdkit-info-plugin --dump-plugin
# nbdkit info --dump-plugin  |grep address
info_address=yes


6.Check nbdkit-curl-plugin man page about new TLS/SSL options
# man nbdkit-curl-plugin |grep ssl-version -A 2
       ssl-version=tlsv1
       ssl-version=sslv2
       ssl-version=sslv3
       ssl-version=tlsv1.0
       ssl-version=tlsv1.1
       ssl-version=tlsv1.2
       ssl-version=tlsv1.3
           Set the SSL ciphers and TLS version.  For further information see CURLOPT_SSL_CIPHER_LIST(3) and CURLOPT_SSLVERSION(3).


7.Check basic function for newly added nbdkit-retry-request-filter

7.1 Not add the retry-request filter
#nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s` % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1 ; fi '

# nbdcopy nbd://localhost /tmp/out
nbdcopy: read at offset 262144 failed: Input/output error

# nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s` % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1 ; fi '
nbdkit: eval[1]: error: /tmp/nbdkitciaeOx/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkitciaeOx/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: write error reply: Broken pipe

7.2 Add retry-request filter

$ nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s` % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1 ; fi ' --filter=retry-request

# nbdcopy nbd://localhost /tmp/out
nbdcopy: read at offset 262144 failed: Input/output error

nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error, but did not print an error message on stderr
.......


8. Check basic function of newly added nbdkit-protect-filter

8.1 Create a 64K disk containing a test pattern, and protect the first 1K:

# nbdkit -f --filter=protect data '"AB" * 32768' protect=0-1023

8.2 open nbdsh, try reading and writing.
$ nbdsh -u nbd://localhost

nbd> print(h.pread(512,0))
b'ABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABAB'

nbd> print(h.pread(512,32768))
b'ABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABABAB'

nbd> buf=b'01'*256
nbd> h.pwrite(buf,32768)
nbd> print(h.pread(512,32768))
b'01010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101010101'

nbd> h.pwrite(buf,0)
Traceback (most recent call last):
  File "/usr/lib64/python3.10/code.py", line 90, in runcode
    exec(code, self.locals)
  File "<console>", line 1, in <module>
  File "/usr/lib64/python3.10/site-packages/nbd.py", line 1719, in pwrite
    return libnbdmod.pwrite(self._o, buf, offset, flags)
nbd.Error: nbd_pwrite: write: command failed: Operation not permitted (EPERM)


9.Check basic function of newly added nbdkit-blocksize-policy-filter

9.1 #nbdkit --filter=blocksize-policy memory 1G blocksize-preferred=32K blocksize-maximum=40K blocksize-minimum=4K blocksize-error-policy=allow

9.2 # nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 1073741824 (1G)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
	block_size_minimum: 4096
	block_size_preferred: 32768
	block_size_maximum: 40960



Hi Richard,

    Thanks for giving me many examples to verify the bug, I have some questions:

  (1)please help to check step4, will vddk-open.sh be used in v2v vddk conversion?

  (2)please help to check step7.2, I don't understand why the connection can't stop after adding retry-request option, is it normal?

Comment 9 Richard W.M. Jones 2022-07-15 10:05:13 UTC

(In reply to mxie from comment #8)
> 4.Check scripts/vddk-open.sh 
> 
> 4.1 Just found vddk-open.sh after rebuilding virt-v2v src package
> # find / |grep vddk-open.sh 
> /home/mxie/rpmbuild/BUILD/nbdkit-1.30.1/scripts/vddk-open.sh.in
> /home/mxie/rpmbuild/BUILD/nbdkit-1.30.1/scripts/vddk-open.sh
> 
> 4.2 Not found vddk-open.sh in v2v vddk debug log
> # cat v2v-vddk.log |grep vddk-open.sh
> nothing

>   (1)please help to check step4, will vddk-open.sh be used in v2v vddk conversion?

This script is not used by virt-v2v.  In fact the script has been deleted
from upstream nbdkit and replaced by a new project:

https://gitlab.com/nbdkit/vddk-remote

(and this is also not used by virt-v2v).

Basically don't worry about this script, it's just for upstream use and
it's going away.

> 7.Check basic function for newly added nbdkit-retry-request-filter
> 
> 7.1 Not add the retry-request filter
> #nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s`
> % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1
> ; fi '
> 
> # nbdcopy nbd://localhost /tmp/out
> nbdcopy: read at offset 262144 failed: Input/output error
> 
> # nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s`
> % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1
> ; fi '
> nbdkit: eval[1]: error: /tmp/nbdkitciaeOx/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkitciaeOx/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: write error reply: Broken pipe
> 
> 7.2 Add retry-request filter
> 
> $ nbdkit -f eval get_size=' echo 1G ' pread=' sleep 1; if test $((`date +%s`
> % 2)) -eq 0 ; then dd if=/dev/zero count=$3 iflag=count_bytes ; else exit 1
> ; fi ' --filter=retry-request
> 
> # nbdcopy nbd://localhost /tmp/out
> nbdcopy: read at offset 262144 failed: Input/output error
> 
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> nbdkit: eval[1]: error: /tmp/nbdkit5NSL1Y/pread: script exited with error,
> but did not print an error message on stderr
> .......

> (2)please help to check step7.2, I don't understand why the connection can't stop after adding retry-request option, is it normal?

I think this is working as expected?  The filter is transparently retrying the
request, and eventually it succeeds (because the request fails stochastically).

By the way, you can fix the "but did not print an error message on stderr"
by adding:

  ... else echo EIO Error >&2; exit 1 ...

(https://libguestfs.org/nbdkit-sh-plugin.3.html#Exit-codes)

I think it's a good test of the filter.

> 8. Check basic function of newly added nbdkit-protect-filter
> 
...
> nbd> h.pwrite(buf,0)
> Traceback (most recent call last):
>   File "/usr/lib64/python3.10/code.py", line 90, in runcode
>     exec(code, self.locals)
>   File "<console>", line 1, in <module>
>   File "/usr/lib64/python3.10/site-packages/nbd.py", line 1719, in pwrite
>     return libnbdmod.pwrite(self._o, buf, offset, flags)
> nbd.Error: nbd_pwrite: write: command failed: Operation not permitted (EPERM)

You can also try writing the *same* data back to the protected area, which
should be permitted (because it wouldn't change the protected data).

Rich.

Comment 10 mxie@redhat.com 2022-07-18 04:21:29 UTC

Thanks rjones, move the bug from ON_QA to VERIFIED according to comment8 and comment9

Comment 12 errata-xmlrpc 2022-11-15 09:50:17 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nbdkit bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:7945

Note You need to log in before you can comment on or make changes to this bug.