Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 2135764

Summary: RHEL 9.2: Rebase libnbd to 1.14.2
Product: Red Hat Enterprise Linux 9 Reporter: Richard W.M. Jones <rjones>
Component: libnbdAssignee: Richard W.M. Jones <rjones>
Status: CLOSED ERRATA QA Contact: Vera <vwu>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 9.2CC: chhu, jsuchane, juzhou, lersek, mxie, rjones, tyan, tzheng, virt-maint, vwu, xiaodwan
Target Milestone: rcKeywords: Rebase, Triaged
Target Release: ---Flags: pm-rhel: mirror+
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libnbd-1.14.2-1.el9 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-05-09 07:48:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Deadline: 2023-01-16   

Description Richard W.M. Jones 2022-10-18 11:30:59 UTC 
Description of problem:

This is a general rebase bug to rebase libnbd to the latest stable
version in RHEL 9.2.

The plan for RHEL 9.2 is here:
https://lists.corp.redhat.com/archives/v2v-devel/2022-August/000028.html
Comment 1 John Ferlan 2022-10-18 11:59:01 UTC 
Setting ITR=9.2.0 and DTM=15 just to give a target. 

This needs to be completed before 15-Jan in order to be ready for Comprehensive Test Cycle 2
Comment 2 Richard W.M. Jones 2023-01-03 15:07:20 UTC 
Upstream, there were not sufficient changes to justify a whole
new stable release.  Also there were many recently introduced new
APIs, and I think these need to stew upstream for a while before
we make them permanent.

What I am doing downstream instead is a large stable branch
update (version 1.14.2, 57 commits):

https://gitlab.com/nbdkit/libnbd/-/commits/stable-1.14/
Comment 3 Richard W.M. Jones 2023-01-03 15:18:49 UTC 
I notice we're upgrading from 1.12.6, and the release notes cover
the major changes:

https://libguestfs.org/libnbd-release-notes-1.14.1.html
Comment 4 Vera 2023-01-17 08:29:48 UTC 
Verified with 
libnbd-1.14.2-1.el9.x86_64
qemu-img-7.2.0-4.el9.x86_64

New features Testing

S1: nbddump (new in libnbd1.14)
# nbdkit -p 10820 random size=1024 --run 'nbddump $uri'
0000000000: 4a c6 fb 35 93 f7 5d e6  ad 19 7b 04 1d dc 19 d3 │J··5··]···{·····│
0000000010: 2e a3 45 2e 30 14 69 e5  4b 84 a0 73 12 7c 34 73 │.·E.0·i·K··s·|4s│
.....
00000003d0: 11 23 8f d2 22 e3 59 1a  11 b4 44 e1 69 4b ce e7 │·#··"·Y···D·iK··│
00000003e0: 62 4a 9a d9 20 32 89 fc  a8 a7 e3 d2 58 04 ed 15 │bJ·· 2······X···│
00000003f0: e1 a0 c8 d6 9a ce 1f ad  3e c4 c7 a7 b2 61 43 c1 │········>····aC·│

# nbdkit -p 10820 pattern size=512 --run 'nbddump $uri'
0000000000: 00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 08 │················│
0000000010: 00 00 00 00 00 00 00 10  00 00 00 00 00 00 00 18 │················│
0000000020: 00 00 00 00 00 00 00 20  00 00 00 00 00 00 00 28 │······· ·······(│
....
00000001d0: 00 00 00 00 00 00 01 d0  00 00 00 00 00 00 01 d8 │················│
00000001e0: 00 00 00 00 00 00 01 e0  00 00 00 00 00 00 01 e8 │················│
00000001f0: 00 00 00 00 00 00 01 f0  00 00 00 00 00 00 01 f8 │················│


S2: New tests for nbd+vsock:// URI support.(new in libnbd1.14)
# ./iperf3 --vsock -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------

Accepted connection from 1, port 1982481010
[  5] local 1 port 5201 connected to 1 port 1982481011
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  7.99 GBytes  68.7 Gbits/sec                  
[  5]   1.00-2.00   sec  8.08 GBytes  69.4 Gbits/sec                  
[  5]   2.00-3.00   sec  8.08 GBytes  69.4 Gbits/sec                  
[  5]   3.00-4.00   sec  8.09 GBytes  69.5 Gbits/sec                  
[  5]   4.00-5.00   sec  8.11 GBytes  69.6 Gbits/sec                  
[  5]   5.00-6.00   sec  8.12 GBytes  69.7 Gbits/sec                  
[  5]   6.00-7.00   sec  8.11 GBytes  69.6 Gbits/sec                  
[  5]   7.00-8.00   sec  8.11 GBytes  69.7 Gbits/sec                  
[  5]   8.00-9.00   sec  8.10 GBytes  69.6 Gbits/sec                  
[  5]   9.00-10.00  sec  8.10 GBytes  69.6 Gbits/sec                  
[  5]  10.00-10.00  sec   256 KBytes  51.2 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  80.9 GBytes  69.5 Gbits/sec                  receiver

# nbdkit --vsock --port 18022 pattern size=512 --run 'nbdinfo "$uri"'
protocol: newstyle-fixed without TLS
export="":
	export-size: 512
	content: data
	uri: nbd+vsock://1:18022/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: true
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: false
	can_fua: false
	can_multi_conn: true
	can_trim: false
	can_zero: false

S3: rare TLS deadlock (new in libnbd1.14)
1. Set up a qemu-nbd + TLS server using steps 1-4 in bug2111524
5. On the second machine run the second attached test program.
# tc qdisc add dev eno1 root netem loss 30%
# ./test-read.py 
.........................................................................
(As expected dotting)

# nbdsh -c 'h.set_tls_certificates("/var/tmp/pki")' -c 'h.connect_uri("nbds://10.73.116.44")' -c 'h.shutdown()'
# 
# nbdcopy nbds://10.73.116.44?tls-certificates=/var/tmp/pki null:


Regression Testing

Scenario1:nbdinfo
1.1 IPv6 numeric URIs are supported.
# nbdinfo nbd://[2620:52:0:4290:2d8:61ff:fe3d:da54]:10809
protocol: newstyle-fixed without TLS, using structured packets
export="":
	export-size: 10737418240 (10G)
	content: data
	uri: nbd://[2620:52:0:4290:2d8:61ff:fe3d:da54]:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true

1.2 <nbdinfo(1)> <--list> (which lists all exports of a server) no longer exists early if one of the exports is inaccessible or there is a recoverable error
 # ls
1  好
 # 
 # nbdkit -U - file dir=/tmp/test --run 'nbdinfo --list "$uri"'
protocol: newstyle-fixed without TLS, using structured packets
export="1":
	export-size: 0
	uri: nbd+unix:///1?socket=/tmp/nbdkitPQkUta/socket
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
export="好":
	export-size: 0
	uri: nbd+unix:///%E5%A5%BD?socket=/tmp/nbdkitPQkUta/socket
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
 # 
1.3 Use nbdinfo to display information and metadata about an NBD server.

# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS, using structured packets
export="":
	export-size: 10737418240 (10G)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
# nbdinfo --json nbd://localhost | jq
{
  "protocol": "newstyle-fixed",
  "TLS": false,
  "exports": [
    {
      "export-name": "",
      "content": "QEMU QCOW2 Image (v3), 10737418240 bytes",
      "uri": "nbd://localhost:10809/",
      "contexts": [
        "base:allocation"
      ],
      "is_rotational": false,
      "is_read_only": false,
      "can_cache": true,
      "can_df": true,
      "can_fast_zero": true,
      "can_flush": true,
      "can_fua": true,
      "can_multi_conn": true,
      "can_trim": true,
      "can_zero": true,
      "export-size": 10737418240,
      "export-size-str": "10G"
    }
  ]
}

Scenario2: nbdcopy
2.1 nbdcopy new <--request-size> option to allow controlling the size of requests made to NBD servers.  Adjusting this can affect performance

#  nbdcopy --request-size=1048576 -p  RHEL-9.2.0-20230108.37-x86_64.qcow2 nbd://localhost
█ 100% [****************************************]
# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS, using structured packets
export="":
	export-size: 10737418240 (10G)
	content: QEMU QCOW2 Image (v3), 10737418240 bytes
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true


2.2 nbdcopy new I<-v> flag to enable libnbd and other debugging.

# qemu-img info RHEL9.2.qcow2 
image: RHEL9.2.qcow2
file format: qcow2
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.01 GiB
cluster_size: 65536
Format specific information:
    compat: 1.1
    compression type: zlib
    lazy refcounts: false
    refcount bits: 16
    corrupt: false
    extended l2: false
# qemu-img convert -f qcow2 -O raw  RHEL9.2.qcow2 RHEL9.2.img
# qemu-img info RHEL9.2.img
image: RHEL9.2.img
file format: raw
virtual size: 10 GiB (10737418240 bytes)
disk size: 1.49 GiB
# nbdcopy -p nbd://localhost RHEL9.2.img -v |& tee >  nbdcopy-test.log
[root@dhcp-66-146-143 /]# grep RHEL9.2 nbdcopy-test.log 
nbdcopy: dst: file_ops "RHEL9.2.img"


2.3  Copy between a local file and an NBD server (in both directions)

# nbdcopy nbd://localhost verify.img -p
█ 100% [****************************************]
[root@dhcp-66-146-143 /]# ls -al |grep verify
-rw-r--r--.   1 root   root   10737418240 Jan 10 15:03 verify.img
[root@dhcp-66-146-143 /]# 
[root@dhcp-66-146-143 /]# ls -lh |grep verify
-rw-r--r--.   1 root   root    10G Jan 10 15:03 verify.img


2.4 Use nbdcopy to copy the file between multiple NBD servers
# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS, using structured packets
export="":
	export-size: 10737418240 (10G)
	content: QEMU QCOW2 Image (v3), 10737418240 bytes
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true

# nbdinfo nbd://localhost:10810
protocol: newstyle-fixed without TLS, using structured packets
export="":
	export-size: 10737418240 (10G)
	content: data
	uri: nbd://localhost:10810/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true

# nbdcopy -p nbd://localhost nbd://localhost:10810
█ 100% [****************************************]
[root@dhcp-66-146-143 ~]# 
[root@dhcp-66-146-143 ~]# nbdinfo nbd://localhost:10810
protocol: newstyle-fixed without TLS, using structured packets
export="":
	export-size: 10737418240 (10G)
	content: QEMU QCOW2 Image (v3), 10737418240 bytes
	uri: nbd://localhost:10810/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true

2.5 
# nbdkit -U - eval get_size=' echo 100M ' pread=' if [ $4 -lt 10000000 ]; then dd if=/dev/zero count=$3 iflag=count_bytes ; else echo EIO Failed >&2 ; exit 1; fi ' --run 'nbdcopy $uri null: -p'
nbdkit: eval[1]: error: /tmp/nbdkitG6XwBZ/pread: Failed
nbdcopy: read at offset 10223616 failed: Input/output error
nbdkit: eval[1]: error: /tmp/nbdkitG6XwBZ/pread: Failed
nbdkit: eval[1]: error: /tmp/nbdkitG6XwBZ/pread: Failed
nbdkit: eval[1]: error: write error reply: Broken pipe
# echo $?
1

Scenario3: nbdfuse
3.1 nbdfuse new I<-v> flag to enable libnbd and other debugging
# nbdfuse /dir/ramdsk [ nbdkit memory 1G ] &
[1] 249454

# nbdfuse nbdfuse-v/ramdisk --command nbdkit -s memory 2G -v |& tee > /nbdfuse-v.log
[root@dhcp-66-146-143 /]# 
[root@dhcp-66-146-143 /]# ls -lh /dir
total 0
-rw-rw-rw-. 1 root root 1.0G Jan 10 19:02 ramdsk

3.2 Turn a qcow2 file into raw file
# nbdfuse /var/tmp/mp/RHEL9.2.raw [ qemu-nbd -f qcow2 RHEL9.2.qcow2  ] &
[1] 249500
[root@dhcp-66-146-143 /]# 
[root@dhcp-66-146-143 /]# ll /var/tmp/mp
total 0
-rw-rw-rw-. 1 root root 10737418240 Jan 10 19:10 RHEL9.2.raw

Scenario4:nbdsh
4.1 Use nbdsh to test simple nbd API calls
# nbdsh

Welcome to nbdsh, the shell for interacting with
Network Block Device (NBD) servers.

The ‘nbd’ module has already been imported and there
is an open NBD handle called ‘h’.

h.connect_tcp("remote", "10809")   # Connect to a remote server.
h.get_size()                       # Get size of the remote disk.
buf = h.pread(512, 0)              # Read the first sector.
exit() or Ctrl-D                   # Quit the shell
help(nbd)                          # Display documentation

nbd> h.connect_command (["nbdkit", "-s", "memory", "size=2G"])
nbd> h.get_size()
2147483648
nbd> buf = b"hello"
nbd> h.pwrite (buf, 0, 0)
nbd> buf2 = h.pread (100, 0)
nbd> print (buf2)
b'hello\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
nbd> print(buf.is_zero(offset=0))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
AttributeError: 'bytes' object has no attribute 'is_zero'
nbd> buf=nbd.Buffer.from_bytearray(bytearray(b"\0a\02\0"))
nbd> print(buf.is_zero(offset=4))
True
nbd> print(buf.is_zero(offset=0))
False
nbd> print(buf.is_zero(offset=1))
False
nbd> buf=nbd.Buffer.from_bytearray(bytearray(b""))
nbd> print(buf.is_zero(offset=0))
True
nbd> print(buf.is_zero(offset=1))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib64/python3.9/site-packages/nbd.py", line 164, in is_zero
    return libnbdmod.aio_buffer_is_zero(self._o, offset, size)
IndexError: offset out of range
nbd> 

Marking as Verified:Tested.
Comment 7 Vera 2023-02-06 08:06:59 UTC 
Marking as Verified.
Comment 9 errata-xmlrpc 2023-05-09 07:48:58 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (libnbd bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2365