Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 590743 - Add .xz support for, at least, .sqlite MD ... mostly solves the N versions of kernel with 9k provides problem
Summary: Add .xz support for, at least, .sqlite MD ... mostly solves the N versions of...
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: yum
Version: 6.0
Hardware: All
OS: Linux
Target Milestone: rc
: ---
Assignee: James Antill
QA Contact: BaseOS QE Security Team
Depends On:
TreeView+ depends on / blocked
Reported: 2010-05-10 15:27 UTC by James Antill
Modified: 2014-01-21 06:17 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2012-06-26 19:33:37 UTC
Target Upstream Version:

Attachments (Terms of Use)

Description James Antill 2010-05-10 15:27:19 UTC
Description of problem:

 This is kind of an RFE, however with kernel doing 9k provides this could be regarded as more of a bug. See SuSE's Bug about the problem:


 Here is the data I tested with:

% l /tmp/kerns-repo -1
total 168M
 21M kernel-2.6.32-19.el6.i686.rpm
 23M kernel-2.6.32-19.el6.x86_64.rpm
 20M kernel-2.6.32-22.el6.i686.rpm
 22M kernel-2.6.32-22.el6.x86_64.rpm
 21M kernel-2.6.32-23.el6.i686.rpm
 22M kernel-2.6.32-23.el6.x86_64.rpm
 21M kernel-2.6.32-24.el6.i686.rpm
 22M kernel-2.6.32-24.el6.x86_64.rpm

...and an example of the data:

% rpm -qp --provides /tmp/kerns-repo/kernel-2.6.32-19.el6.i686.rpm | wc -l

...then the createrepo data:

% l /tmp/kerns-repo/repodata -1
total 17M
 88K filelists.sqlite.bz2
108K filelists.xml.gz
 36K other.sqlite.bz2
100K other.xml.gz
1.7M primary.sqlite.bz2
960K primary.xml.gz

...and finally the primary.sqlite file both uncompressed and then re-compressed using "xz":

7.9M primary.sqlite
1.1M primary.sqlite.xz

...next I did:

% mkdir /tmp/kerns-repo/1
% mkdir /tmp/kerns-repo/2
% mkdir /tmp/kerns-repo/3
% mv /tmp/kerns-repo/*.rpm /tmp/kerns-repo/1
% ln /tmp/kerns-repo/1/* /tmp/kerns-repo/2
% ln /tmp/kerns-repo/1/* /tmp/kerns-repo/3

...which is a hacky way to get 24 versions of the kernel, instead of 8. Primary sizes again:

 24M primary.sqlite
4.7M primary.sqlite.bz2
2.0M primary.sqlite.xz

...and "time" results:

% time bzip2 -dc /tmp/kerns-repo/repodata/primary.sqlite.bz2 > /tmp/kerns-repo/repodata/primary.sqlite
bzip2 -dc /tmp/kerns-repo/repodata/primary.sqlite.bz2 >   1.19s user 0.13s system 97% cpu 1.352 total
% time xz -9 /tmp/kerns-repo/repodata/primary.sqlite
xz -9 /tmp/kerns-repo/repodata/primary.sqlite  21.57s user 0.57s system 95% cpu 23.145 total
% time xz /tmp/kerns-repo/repodata/primary.sqlite
xz /tmp/kerns-repo/repodata/primary.sqlite  20.96s user 0.24s system 96% cpu 21.945 total
% time unxz /tmp/kerns-repo/repodata/primary.sqlite.xz
unxz /tmp/kerns-repo/repodata/primary.sqlite.xz  0.45s user 0.11s system 95% cpu 0.584 total

...note that "xz -9" and "xz" both produced a 2MB primary.sqlite.

Version-Release number of selected component (if applicable):

 However note that this would just be the client side change, we'd also need a createrepo change (and an RHN change) to create the .sqlite.xz files.

Comment 1 James Antill 2010-05-10 15:31:46 UTC
 Also, in case it wasn't obvious, yes if we _have_ to we can probably delay this until 6.1 _however_ that means anyone installing GA for all of RHEL-6 is going to download a bigger .sqlite.bz2 (and/or anyone who refuses to update yum).
 Where if we do the yum change for 6.0, then even if we don't do the server change until later all users will get the benefit.

Comment 2 RHEL Program Management 2010-05-10 15:40:40 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for

Comment 4 RHEL Program Management 2010-05-14 14:32:49 UTC
Quality Engineering Management has reviewed and declined this request.  You may
appeal this decision by reopening this request.

Comment 5 James Antill 2010-06-23 16:50:01 UTC
Moving to 6.1 ... note that we have the bindings now, so we just need the minor createrepo and yum changes to use it.

Comment 6 James Antill 2010-06-23 17:21:56 UTC
Removing the bindings blocker

Comment 8 James Antill 2010-11-22 20:15:46 UTC
moving to 6.2.x ... it's not done upstream, yet. And pyliblzma is only just available in Fed-14.

Comment 10 James Antill 2011-06-09 18:30:54 UTC
Still not done upstream ... need to think about again, WRT repodata.

Comment 12 James Antill 2011-12-20 19:25:26 UTC
 Given we opened it, and it's not on in Fedora, and there have been a couple of weird issues when people have tried it outside Fedora ... and that we'd still need to change the server side. I think we can just drop this for RHEL-6.

Comment 14 Suzanne Yeghiayan 2012-05-18 20:45:40 UTC
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 15 James Antill 2012-06-26 19:33:37 UTC
 Not much has changed since comment #12, closing this out.

Note You need to log in before you can comment on or make changes to this bug.