RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2059288 - RFE: Rebase libnbd to 1.12 in RHEL 9.1
Summary: RFE: Rebase libnbd to 1.12 in RHEL 9.1
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: libnbd
Version: 9.1
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: Vera
URL:
Whiteboard:
Depends On:
Blocks: 2047660
TreeView+ depends on / blocked
 
Reported: 2022-02-28 16:30 UTC by Richard W.M. Jones
Modified: 2022-11-15 10:07 UTC (History)
9 users (show)

Fixed In Version: libnbd-1.12.6-1.el9
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-11-15 09:50:16 UTC
Type: Feature Request
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-114033 0 None None None 2022-02-28 16:37:38 UTC
Red Hat Product Errata RHBA-2022:7944 0 None None None 2022-11-15 09:50:21 UTC

Description Richard W.M. Jones 2022-02-28 16:30:03 UTC 
Description of problem:

This is a request to rebase libnbd to the latest
upstream stable release in RHEL 9.1.

The upstream release is 1.12.

Release notes are available outlining the main changes:
https://libguestfs.org/libnbd-release-notes-1.12.1.html
Comment 5 Vera 2022-03-24 05:29:23 UTC 
Verified with the Versions:
libnbd-1.12.2-1.el9.x86_64
nbdkit-server-1.30.1-1.el9.x86_64
nbdkit-nbd-plugin-1.30.1-1.el9.x86_64
nbdfuse-1.12.2-1.el9.x86_64

Steps:
Scenario1:nbdinfo
1.1 IPv6 numeric URIs are supported.
# nbdinfo 'nbd://[2620:52:0:49b2:2eea:7fff:feee:53c0]:10809'
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: data
	uri: nbd://[2620:52:0:49b2:2eea:7fff:feee:53c0]:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true


1.2 <nbdinfo(1)> <--list> (which lists all exports of a server) no longer exists early if one of the exports is inaccessible or there is a recoverable error
# nbdkit -U - file dir=/tmp/test --run 'nbdinfo --list "$uri"'
protocol: newstyle-fixed without TLS
export="1":
	export-size: 0
	uri: nbd+unix:///1?socket=/tmp/nbdkitMgXfww/socket
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
export="2":
	export-size: 0
	uri: nbd+unix:///2?socket=/tmp/nbdkitMgXfww/socket
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true


1.3 Use nbdinfo to display information and metadata about an NBD server.
# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: data
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true


# nbdinfo --json nbd://localhost | jq
{
  "protocol": "newstyle-fixed",
  "TLS": false,
  "exports": [
    {
      "export-name": "",
      "content": "data",
      "uri": "nbd://localhost:10809/",
      "contexts": [
        "base:allocation"
      ],
      "is_rotational": false,
      "is_read_only": false,
      "can_cache": true,
      "can_df": true,
      "can_fast_zero": true,
      "can_flush": true,
      "can_fua": true,
      "can_multi_conn": true,
      "can_trim": true,
      "can_zero": true,
      "export-size": 10737418240,
      "export-size-str": "10G"
    }
  ]
}

Scenario2: nbdcopy
2.1 nbdcopy new <--request-size> option to allow controlling the size of requests made to NBD servers.  Adjusting this can affect performance

# nbdcopy --request-size=1048576 -p fedora-32.qcow2 nbd://localhost
█ 100% [****************************************]
# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: QEMU QCOW2 Image (v3), 6442450944 bytes
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true


2.2 nbdcopy new I<-v> flag to enable libnbd and other debugging.
# nbdcopy -p nbd://localhost test.img -v |& tee >  nbdcopy.log
█ 100% [****************************************]
# grep test nbdcopy.log
nbdcopy: dst: file_ops "test.img"


2.3  Copy between a local file and an NBD server (in both directions)

# nbdcopy --request-size=1048576 -p fedora-32.qcow2 nbd://localhost
█ 100% [****************************************]
# nbdcopy nbd://localhost verfiy.img -p
█ 100% [****************************************]
# ls -al |grep verfiy
-rw-r--r--.  1 root root 10737418240 Mar 24 00:06 verfiy.img


2.4 Use nbdcopy to copy the file between multiple NBD servers

# nbdinfo nbd://localhost:10810
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: data
	uri: nbd://localhost:10810/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true
# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: QEMU QCOW2 Image (v3), 6442450944 bytes
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true


# nbdcopy -p nbd://localhost nbd://localhost:10810
█ 100% [****************************************]
# nbdinfo nbd://localhost:10810
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: QEMU QCOW2 Image (v3), 6442450944 bytes
	uri: nbd://localhost:10810/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true



2.5 nbdfuse now supports nbdcopy-style C<[ CMD ]>, eg: nbdfuse dir/ramdisk [ nbdkit memory 1G ]

# nbdfuse dir/ramdsk [ nbdkit memory 1G ] &
[2] 84357
# ls -lh dir
total 0
-rw-rw-rw-. 1 root root 1.0G Mar 24 00:39 ramdsk



Scenario3: nbdfuse

3.1 nbdfuse new I<-v> flag to enable libnbd and other debugging

# nbdfuse nbdfuse-v/ramdisk --command nbdkit -s memory 2G -v |& tee > nbdfuse-v.log
# ls -lh dir
total 0
-rw-rw-rw-. 1 root root 1.0G Mar 24 00:39 ramdsk


3.2 Turn a qcow2 file into raw file
# nbdfuse /var/tmp/mp/fedora-32.raw [ qemu-nbd -f qcow2 fedora-32.qcow2 ] &
[3] 84388
# ll /var/tmp/mp
total 0
-rw-rw-rw-. 1 root root 6442450944 Mar 24 00:44 fedora-32.raw




Scenario4:nbdsh
4.1 Use nbdsh to test simple nbd API calls
# nbdsh

Welcome to nbdsh, the shell for interacting with
Network Block Device (NBD) servers.

The ‘nbd’ module has already been imported and there
is an open NBD handle called ‘h’.

h.connect_tcp("remote", "10809")   # Connect to a remote server.
h.get_size()                       # Get size of the remote disk.
buf = h.pread(512, 0)              # Read the first sector.
exit() or Ctrl-D                   # Quit the shell
help(nbd)                          # Display documentation


nbd> help(nbd)

nbd> h.connect_command (["nbdkit", "-s", "memory", "size=2G"])
nbd> h.get_size()
2147483648
nbd> buf = b"hello"
nbd> h.pwrite (buf, 0, 0)
nbd> buf2 = h.pread (100, 0)
nbd> print (buf2)
b'hello\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
nbd> 
nbd> buf=nbd.Buffer.from_bytearray(bytearray(b"abc"))
nbd> print(buf.is_zero(offset=0))
False
nbd> buf=nbd.Buffer.from_bytearray(bytearray(b"\0a\02\0"))
nbd> print(buf.is_zero(offset=4))
True
nbd> print(buf.is_zero(offset=0))
False
nbd> print(buf.is_zero(offset=1))
False
nbd> buf=nbd.Buffer.from_bytearray(bytearray(b""))
nbd> print(buf.is_zero(offset=0))
True
nbd> 
nbd> print(buf.is_zero(offset=1))
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/lib64/python3.9/site-packages/nbd.py", line 157, in is_zero
    return libnbdmod.aio_buffer_is_zero(self._o, offset, size)
IndexError: offset out of range
nbd> 

Scenario 5: rebuilding src rpm
Steps:
1. Download src package ;
-rw-r--r--.  1 root root     1302858 Mar 15 13:42 libnbd-1.12.2-1.el9.src.rpm


2.  Resolve the dependence problem before rebuilding,then build libnbd rpm packages from src package with non-root user;

# rpmbuild --rebuild libnbd-1.12.2-1.el9.src.rpm 
Installing libnbd-1.12.2-1.el9.src.rpm
warning: user mockbuild does not exist - using root
warning: group mock does not exist - using root
warning: user mockbuild does not exist - using root
warning: group mock does not exist - using root
warning: user mockbuild does not exist - using root
warning: group mock does not exist - using root
warning: user mockbuild does not exist - using root
warning: group mock does not exist - using root
warning: user mockbuild does not exist - using root
warning: group mock does not exist - using root
setting SOURCE_DATE_EPOCH=1647302400
Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.uTvyqX
+ umask 022
+ cd /root/rpmbuild/BUILD
+ /usr/lib/rpm/redhat/gpgverify --keyring=/root/rpmbuild/SOURCES/libguestfs.keyring --signature=/root/rpmbuild/SOURCES/libnbd-1.12.2.tar.gz.sig --data=/root/rpmbuild/SOURCES/libnbd-1.12.2.tar.gz
gpgv: Signature made Tue Mar 15 12:40:17 2022 EDT
gpgv:                using RSA key F7774FB1AD074A7E8C8767EA91738F73E1B768A0
gpgv:                issuer "rich"
gpgv: Good signature from "Richard W.M. Jones <rjones>"
gpgv:                 aka "Richard W.M. Jones <rich>"
+ cd /root/rpmbuild/BUILD
+ rm -rf libnbd-1.12.2
...........
..........
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.c17Pc5
+ umask 022
+ cd /root/rpmbuild/BUILD
+ cd libnbd-1.12.2
+ /usr/bin/rm -rf /root/rpmbuild/BUILDROOT/libnbd-1.12.2-1.el9.x86_64
+ RPM_EC=0
++ jobs -p
+ exit 0
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.C3ZU8Z
+ umask 022
+ cd /root/rpmbuild/BUILD
+ rm -rf libnbd-1.12.2
+ RPM_EC=0
++ jobs -p
+ exit 0

3. check the rpms under the ~/RPMS/x86_64 and compare with the rpms on the brewweb.

# ls
libnbd-1.12.2-1.el9.x86_64.rpm              ocaml-libnbd-1.12.2-1.el9.x86_64.rpm
libnbd-debuginfo-1.12.2-1.el9.x86_64.rpm    ocaml-libnbd-debuginfo-1.12.2-1.el9.x86_64.rpm
libnbd-debugsource-1.12.2-1.el9.x86_64.rpm  ocaml-libnbd-devel-1.12.2-1.el9.x86_64.rpm
libnbd-devel-1.12.2-1.el9.x86_64.rpm        python3-libnbd-1.12.2-1.el9.x86_64.rpm
nbdfuse-1.12.2-1.el9.x86_64.rpm             python3-libnbd-debuginfo-1.12.2-1.el9.x86_64.rpm
nbdfuse-debuginfo-1.12.2-1.el9.x86_64.rpm


Mark Verified:Tested and moving to Verified.
Comment 6 Vera 2022-07-07 04:30:42 UTC 
Tried more testing on new features with the versions:
nbdkit-server-1.30.6-2.el9.x86_64
libnbd-1.12.4-3.el9.x86_64
nbdkit-nbd-plugin-1.30.6-2.el9.x86_64
nbdfuse-1.12.4-3.el9.x86_64


1. Security

# nbdcopy -p -- [ nbdkit --filter=error pattern 5M error-pread-rate=1 ] null:
nbdkit: pattern.0: error: injecting EIO error into pread
nbdkit: pattern.1: error: injecting EIO error into pread
nbdkit: pattern.2: error: injecting EIO error into pread
nbdkit: pattern.3: error: injecting EIO error into pread
nbdkit: pattern.4: error: injecting EIO error into pread
nbdkit: pattern.5: error: injecting EIO error into pread
nbdkit: pattern.6: error: injecting EIO error into pread
nbdkit: pattern.7: error: injecting EIO error into pread
nbdkit: pattern.8: error: injecting EIO error into pread
nbdkit: pattern.9: error: injecting EIO error into pread
nbdkit: pattern.10: error: injecting EIO error into pread
nbdkit: pattern.11: error: injecting EIO error into pread
nbdkit: pattern.12: error: injecting EIO error into pread
nbdkit: pattern.13: error: injecting EIO error into pread
nbdkit: pattern.14: error: injecting EIO error into pread
nbdkit: pattern.14: error: injecting EIO error into pread
nbdcopy: read at offset 0 failed: Input/output error
nbdkit: pattern.15: error: injecting EIO error into pread
nbdkit: pattern.15: error: injecting EIO error into pread
nbdkit: pattern.15: error: injecting EIO error into pread
nbdkit: pattern.0: error: injecting EIO error into pread
nbdkit: pattern.1: error: read request: Connection reset by peer
# echo $?
1

2. Enhancement to existing API:
 Error messages about incorrect URIs in nbd_connect_uri(3) have been improved to make common mistakes clearer for the user.
with libnbd-1.6.0-5.module+el8.6.0+14156+faad4bd2.x86_64
# nbdinfo --list "$uri"
nbdinfo: nbd_connect_uri: unknown URI scheme: NULL: Invalid argument


# nbdinfo --list "$uri"
nbdinfo: nbd_connect_uri: NBD URI does not have a scheme: valid NBD URIs should start with a scheme like nbd://, nbds:// or nbd+unix://: Invalid argument


3. Tools: 
 3.1 New syntax: nbdinfo [ CMD ... ] allowing you to query the properties of an NBD server directly.

# nbdinfo -- [ qemu-nbd -r -f qcow2 fedora-30.qcow2 ]
protocol: newstyle-fixed without TLS
export="":
	export-size: 6442450944 (6G)
	content: DOS/MBR boot sector
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: true
	can_cache: true
	can_df: true
	can_fast_zero: false
	can_flush: true
	can_fua: true
	can_multi_conn: false
	can_trim: false
	can_zero: false
	block_size_minimum: 1
	block_size_preferred: 4096
	block_size_maximum: 33554432


 3.2 nbdcopy(1) new --queue-size option to control the maximum size of the internal buffer (Nir Soffer).
--queue-size=N
Set the maximum number of bytes to queue for in flight requests. The default value is 16 MiB, allowing up to 64 256k requests per NBD connection. If you use larger --request-size you may want to increase this value.

[root@dhcp-66-146-36 ~]# nbdcopy --queue-size=1048576 -p fedora-30.img nbd://localhost
█ 100% [****************************************]
[root@dhcp-66-146-36 ~]# 
[root@dhcp-66-146-36 ~]# 
[root@dhcp-66-146-36 ~]# nbdinfo nbd://localhost
protocol: newstyle-fixed without TLS
export="":
	export-size: 10737418240 (10G)
	content: DOS/MBR boot sector
	uri: nbd://localhost:10809/
	contexts:
		base:allocation
	is_rotational: false
	is_read_only: false
	can_cache: true
	can_df: true
	can_fast_zero: true
	can_flush: true
	can_fua: true
	can_multi_conn: true
	can_trim: true
	can_zero: true

 3.3 nbdcopy(1) now names the source and destination handles to make it easier to understand debug output.
# nbdcopy -p -v nbd://localhost nbd://localhost:10810
…..
libnbd: debug: dst3: nbd_aio_get_fd: enter:
libnbd: debug: dst3: nbd_aio_get_fd: leave: ret=10
libnbd: debug: src2: nbd_aio_pread: transition: ISSUE_COMMAND.SEND_REQUEST -> ISSUE_COMMAND.PREPARE_WRITE_PAYLOAD
libnbd: debug: src2: nbd_aio_pread: transition: ISSUE_COMMAND.PREPARE_WRITE_PAYLOAD -> ISSUE_COMMAND.FINISH
libnbd: debug: src2: nbd_aio_pread: transition: ISSUE_COMMAND.FINISH -> READY
libnbd: debug: src2: nbd_aio_pread: leave: ret=1159
libnbd: debug: dst1: nbd_aio_pwrite: enter: buf="
Comment 8 errata-xmlrpc 2022-11-15 09:50:16 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (libnbd bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2022:7944
Note You need to log in before you can comment on or make changes to this bug.