Bug 1820275 - [FTBFS] nbdkit test-nozero.sh fails: w: invalid option -- 'u'
Summary: [FTBFS] nbdkit test-nozero.sh fails: w: invalid option -- 'u'
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: nbdkit
Version: 7.8
Hardware: All
OS: Linux
medium
unspecified
Target Milestone: rc
: ---
Assignee: Richard W.M. Jones
QA Contact: mxie@redhat.com
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-04-02 16:19 UTC by Pat Riehecky
Modified: 2020-09-29 20:05 UTC (History)
8 users (show)

Fixed In Version: nbdkit-1.8.0-4.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-09-29 20:05:22 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2020:3954 0 None None None 2020-09-29 20:05:25 UTC

Description Pat Riehecky 2020-04-02 16:19:37 UTC
Description of problem:
test-nozero.sh uses an option for 'write' from util linux.

Version-Release number of selected component (if applicable): nbdkit-1.8.0-3.el7


How reproducible:100%


Steps to Reproduce:
1.Attempt to rebuild on RHEL7.8
2.
3.

Actual results:
FAIL: test-nozero.sh
====================
w: invalid option -- 'u'
write [-bcCpqz] [-P pattern ] off len -- writes a number of bytes at a specified offset

Expected results:
Tests should run as expected.

Additional info:

Comment 2 Richard W.M. Jones 2020-04-02 16:35:23 UTC
I think it's actually running a qemu-io subcommand:

https://github.com/libguestfs/nbdkit/blob/611f931d75feaf516da8097e6a69ff084b4ca38c/tests/test-nozero.sh#L105

Which version of qemu is installed?  When this package was originally
built we used qemu 10:1.5.3-167.el7_7.1  Since this ancient version
of qemu didn't even have qemu-io it skipped the test.

Comment 3 Pat Riehecky 2020-04-02 17:05:27 UTC
I'm running with qemu-kvm-1.5.3-173.el7 in the build root, and the test doesn't seem to be skipping....

Comment 4 Richard W.M. Jones 2020-04-02 17:32:02 UTC
I can reproduce this with qemu-img-1.5.3-173.el7_8.1.x86_64

Comment 5 Eric Blake 2020-04-02 17:34:56 UTC
(In reply to Pat Riehecky from comment #0)
> Description of problem:
> test-nozero.sh uses an option for 'write' from util linux.
> 

(In reply to Richard W.M. Jones from comment #2)
> I think it's actually running a qemu-io subcommand:
> 
> https://github.com/libguestfs/nbdkit/blob/
> 611f931d75feaf516da8097e6a69ff084b4ca38c/tests/test-nozero.sh#L105

Correct, it is qemu-io that is too old, and nothing to do with 'write' from util-linux.  qemu-io added the -u option to its write sub-command in commit c2e001cc, v2.7.0, but RHEL is using an older qemu.

> 
> Which version of qemu is installed?  When this package was originally
> built we used qemu 10:1.5.3-167.el7_7.1  Since this ancient version
> of qemu didn't even have qemu-io it skipped the test.

Did qemu-img change between 1.5.3-167.el7_7 and 1.5.3-173.el7_8?  At any rate, the best fix upstream is to rewrite the test to use nbdsh rather than qemu-io (as that is a lot more flexible at providing the needed functionality under finer-grain control), at which point backporting the upstream change to downstream nbdkit 1.8 would cause the test to be skipped unless we also start shipping libnbd in RHEL 7.

Comment 6 Richard W.M. Jones 2020-04-02 17:41:13 UTC
The test actually works with nbdkit upstream compiling on RHEL 7 (which
I semi-regularly test):

  if ! qemu-io -f raw -d unmap -c 'w -z -u 0 1M' nozero1.img; then
      echo "$0: missing or broken qemu-io"
      rm nozero?.img
      exit 77
  fi

causes the test to skip because:

  qemu-io: invalid option -- 'f'

This actual bug happens because of a downstream-only change we have made in
RHEL 7's spec file:

  %prep
  ...
  # Ancient qemu-io in RHEL 7 doesn't support -f FORMAT option.  However
  # we can just omit it and the tests still work fine.
  for f in tests/*.sh; do
    sed -i -e 's/qemu-io -f raw/qemu-io/g' \
           -e 's/qemu-io -r -f raw/qemu-io -r/g' $f
  done

I guess this worked fine until somewhere in qemu > 167 .. qemu <= 173.

Comment 7 Richard W.M. Jones 2020-04-02 17:50:01 UTC
Removing the downstream hunk from %prep actually fixes everything in the
RHEL 7 build, so that is my recommendation.  I checked it with both
qemu-img-1.5.3-173.el7_8.1.x86_64 and qemu-img-rhev-2.12.0-45.el7.x86_64

(And I agree that upstream the best fix is to use nbdsh.)

Comment 13 mxie@redhat.com 2020-04-08 09:06:05 UTC
I can reproduce the bug with below builds:
qemu-kvm-1.5.3-173.el7_8.1.x86_64
libvirt-4.5.0-33.el7_8.1.x86_64
nbdkit-1.8.0-3.el7.src.rpm

Steps to reproduce:
1.Download nbdkit-1.8.0-3.el7.src.rpm package to /home

2.log into server with regular user and create .rpmmacros file containing:
$ cat .rpmmacros 
%_topdir  %(echo $HOME)/rpmbuild
%_smp_mflags -j5

3.Resolve the dependence problem before rebuilding,then build nbdkit rpm packages from 1.8.0-3 src package
$rpmbuild --rebuild nbdkit-1.8.0-3.el7.src.rpm 
....
PASS: test-help.sh
PASS: test-version.sh
PASS: test-dump-config.sh
PASS: test-help-plugin.sh
PASS: test-version-plugin.sh
PASS: test-version-filter.sh
PASS: test-dump-plugin.sh
PASS: test-start.sh
SKIP: test-single.sh
PASS: test-captive.sh
PASS: test-random-sock.sh
SKIP: test-tls.sh
SKIP: test-tls-psk.sh
SKIP: test-ip.sh
PASS: test-socket-activation
PASS: test-foreground.sh
PASS: test-debug-flags.sh
PASS: test-exit-with-parent
PASS: test-parallel-file.sh
PASS: test-parallel-nbd.sh
PASS: test-data-7E.sh
SKIP: test-data-base64.sh
PASS: test-data-file.sh
PASS: test-data-raw.sh
PASS: test-floppy.sh
PASS: test-iso.sh
SKIP: test-memory-largest.sh
PASS: test-memory-largest-for-qemu.sh
SKIP: test-partitioning1.sh
SKIP: test-partitioning4.sh
PASS: test-partitioning2.sh
PASS: test-partitioning3.sh
PASS: test-pattern.sh
SKIP: test-pattern-largest.sh
PASS: test-pattern-largest-for-qemu.sh
PASS: test-vddk.sh
SKIP: test-zero.sh
PASS: test-python-exception.sh
PASS: test-shebang-python.sh
PASS: test-layers.sh
PASS: test-layers
SKIP: test-blocksize.sh
PASS: test-cache.sh
PASS: test-cow.sh
PASS: test-fua.sh
PASS: test-log.sh
FAIL: test-nozero.sh
PASS: test-offset2.sh
PASS: test-truncate1.sh
PASS: test-truncate2.sh
SKIP: test-truncate3.sh
make[4]: Entering directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
============================================================================
Testsuite summary for nbdkit 1.8.0
============================================================================
# TOTAL: 51
# PASS:  38
# SKIP:  12
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
....
FAIL: test-nozero.sh
====================
w: invalid option -- 'u'
....

Additional info
1.Can't reproduce the bug with below builds and result of test-nozero.sh is PASS during building
qemu-kvm-rhev-2.12.0-44.el7_8.1.x86_64
libvirt-4.5.0-33.el7_8.1.x86_64
nbdkit-1.8.0-3.el7.src.rpm 


Verify the bug with builds:
qemu-kvm-1.5.3-173.el7_8.1.x86_64
libvirt-4.5.0-33.el7_8.1.x86_64
nbdkit-1.8.0-4.el7.src.rpm

1.Download nbdkit-1.8.0-4.el7.src.rpm package to /home

2.log into server with regular user to build nbdkit rpm packages from 1.8.0-4 src package
$rpmbuild --rebuild nbdkit-1.8.0-4.el7.src.rpm 
...
PASS: test-help.sh
PASS: test-version.sh
PASS: test-dump-config.sh
PASS: test-help-plugin.sh
PASS: test-version-plugin.sh
PASS: test-version-filter.sh
PASS: test-dump-plugin.sh
PASS: test-start.sh
SKIP: test-single.sh
PASS: test-captive.sh
PASS: test-random-sock.sh
SKIP: test-tls.sh
SKIP: test-tls-psk.sh
SKIP: test-ip.sh
PASS: test-socket-activation
PASS: test-foreground.sh
PASS: test-debug-flags.sh
PASS: test-exit-with-parent
PASS: test-parallel-file.sh
PASS: test-parallel-nbd.sh
PASS: test-data-7E.sh
SKIP: test-data-base64.sh
PASS: test-data-file.sh
PASS: test-data-raw.sh
PASS: test-floppy.sh
PASS: test-iso.sh
SKIP: test-memory-largest.sh
PASS: test-memory-largest-for-qemu.sh
SKIP: test-partitioning1.sh
SKIP: test-partitioning4.sh
PASS: test-partitioning2.sh
PASS: test-partitioning3.sh
PASS: test-pattern.sh
SKIP: test-pattern-largest.sh
PASS: test-pattern-largest-for-qemu.sh
PASS: test-vddk.sh
SKIP: test-zero.sh
PASS: test-python-exception.sh
PASS: test-shebang-python.sh
PASS: test-layers.sh
PASS: test-layers
SKIP: test-blocksize.sh
PASS: test-cache.sh
PASS: test-cow.sh
PASS: test-fua.sh
PASS: test-log.sh
SKIP: test-nozero.sh
PASS: test-offset2.sh
PASS: test-truncate1.sh
PASS: test-truncate2.sh
SKIP: test-truncate3.sh
make[4]: Entering directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
make[4]: Nothing to be done for `all'.
make[4]: Leaving directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
============================================================================
Testsuite summary for nbdkit 1.8.0
============================================================================
# TOTAL: 51
# PASS:  38
# SKIP:  13
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
make[2]: Leaving directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
make[1]: Leaving directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0/tests'
make[1]: Entering directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0'
make[1]: Leaving directory `/home/mxie/rpmbuild/BUILD/nbdkit-1.8.0'
+ exit 0
....

2.Try to rebuild nbdkit-1.8.0-4.el7.src.rpm with below builds, building can finish successfully and the result of test-nozero.sh is PASS during building
qemu-kvm-rhev-2.12.0-44.el7_8.1.x86_64
libvirt-4.5.0-33.el7_8.1.x86_64


Result:
   Can build nbdkit rpm packages from 1.8.0-4 src package with qemu-kvm-1.5.3-173.el7_8.1.x86_64 and qemu-kvm-rhev-2.12.0-44.el7_8.1.x86_64 successfully, move the bug from ON_QA to VERIFIED

Comment 15 errata-xmlrpc 2020-09-29 20:05:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nbdkit bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:3954


Note You need to log in before you can comment on or make changes to this bug.