Bug 984704

Summary: RFE: appliance-tools should use --block-size to prepare seekable xz images
Product: [Fedora] Fedora Reporter: Richard W.M. Jones <rjones>
Component: appliance-toolsAssignee: Dennis Gilmore <dennis>
Status: NEW --- QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: agrimm, apevec, dhuff, mattdm, mgoldman, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:

Description Richard W.M. Jones 2013-07-15 14:28:54 EDT
Description of problem:

Recent versions of xz (note: not all versions) have a new
--block-size parameter which allows xz files to be created
with a predictable block size.  The block size is specified
in bytes, for example:

 $ xz --block-size=16777216 --best test1.img
 $ xz --list test1.img.xz
  Strms  Blocks   Compressed Uncompressed  Ratio  Check   Filename
      1       7     18.9 KiB    100.0 MiB  0.000  CRC64   test1.img.xz
           ^^^^

The advantage of using a large, finite block size is that
these xz-images become seekable:

 https://rwmj.wordpress.com/2013/06/24/xz-plugin-for-nbdkit/#content

nbdkit, an NBD server, has support for seeking in xz files,
provided they have been prepared with --block-size:

 https://github.com/libguestfs/nbdkit/tree/master/plugins/xz

There is some overhead to using a block size, but with a 16MB
block size the overhead is only around 1%.  Note there is a trade
off between the effectiveness of compression (larger block sizes
are better) and the amount of time & memory needed when seeking
(larger block sizes cause more data to be uncompressed and a
larger amount of memory to be allocated).  16 MB seems like a
good compromise for current hardware.

BTW if I'm reading the appliance-tools code correctly, I think
you should also allow a compression level to be specified, and/or
default to --best.

Version-Release number of selected component (if applicable):

appliance-tools 007

Additional info:

https://lists.fedoraproject.org/pipermail/devel/2013-July/thread.html#185369
Comment 1 Fedora Admin XMLRPC Client 2014-01-12 14:49:56 EST
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.