Bug 984704 - RFE: appliance-tools should use --block-size to prepare seekable xz images
RFE: appliance-tools should use --block-size to prepare seekable xz images
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: appliance-tools (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Neal Gompa
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-15 14:28 EDT by Richard W.M. Jones
Modified: 2017-04-01 12:58 EDT (History)
5 users (show)

See Also:
Fixed In Version: appliance-tools-008.0-4.fc25 appliance-tools-008.0-4.fc26
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2017-03-28 21:29:50 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Richard W.M. Jones 2013-07-15 14:28:54 EDT
Description of problem:

Recent versions of xz (note: not all versions) have a new
--block-size parameter which allows xz files to be created
with a predictable block size.  The block size is specified
in bytes, for example:

 $ xz --block-size=16777216 --best test1.img
 $ xz --list test1.img.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1       7     18.9 KiB    100.0 MiB  0.000  CRC64   test1.img.xz
           ^^^^

The advantage of using a large, finite block size is that
these xz-images become seekable:

 https://rwmj.wordpress.com/2013/06/24/xz-plugin-for-nbdkit/#content

nbdkit, an NBD server, has support for seeking in xz files,
provided they have been prepared with --block-size:

 https://github.com/libguestfs/nbdkit/tree/master/plugins/xz

There is some overhead to using a block size, but with a 16MB
block size the overhead is only around 1%.  Note there is a trade
off between the effectiveness of compression (larger block sizes
are better) and the amount of time & memory needed when seeking
(larger block sizes cause more data to be uncompressed and a
larger amount of memory to be allocated).  16 MB seems like a
good compromise for current hardware.

BTW if I'm reading the appliance-tools code correctly, I think
you should also allow a compression level to be specified, and/or
default to --best.

Version-Release number of selected component (if applicable):

appliance-tools 007

Additional info:

https://lists.fedoraproject.org/pipermail/devel/2013-July/thread.html#185369
Comment 1 Fedora Admin XMLRPC Client 2014-01-12 14:49:56 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 2 Fedora Admin XMLRPC Client 2016-12-21 16:14:37 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 3 Fedora Admin XMLRPC Client 2016-12-21 16:16:26 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 4 Neal Gompa 2017-03-12 17:28:43 EDT
While I see the value in adding --block-size, I'm not sure what value I should set here. Do you have any particular recommendations?
Comment 5 Fedora Update System 2017-03-12 18:44:13 EDT
appliance-tools-008.0-4.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-fdc8ea1d5d
Comment 6 Fedora Update System 2017-03-12 18:46:21 EDT
appliance-tools-008.0-4.fc25 livecd-tools-24.2-1.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-7799a1cc7c
Comment 7 Fedora Update System 2017-03-12 18:46:29 EDT
appliance-tools-008.0-4.fc25 livecd-tools-24.2-1.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-7799a1cc7c
Comment 8 Fedora Update System 2017-03-12 21:51:52 EDT
appliance-tools-008.0-4.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-fdc8ea1d5d
Comment 9 Fedora Update System 2017-03-13 21:22:50 EDT
appliance-tools-008.0-4.fc25, livecd-tools-24.2-1.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-7799a1cc7c
Comment 10 Fedora Update System 2017-03-28 21:29:50 EDT
appliance-tools-008.0-4.fc25, livecd-tools-24.2-1.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.
Comment 11 Fedora Update System 2017-04-01 12:58:09 EDT
appliance-tools-008.0-4.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.