Bug 1111871

Summary: [rfe] dnf should use sqlite/.xz repodata
Product: [Fedora] Fedora Reporter: Kevin Fenzi <kevin>
Component: dnfAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: akozumpl, packaging-team-maint, pnemade, rholy, samuel-rhbugs, tim.lauridsen
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-06-23 08:43:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kevin Fenzi 2014-06-21 19:54:07 UTC 
Rawhide is running with xz repodata enabled, but dnf is still pulling some of the old .gz repodata. 

In particular dnf uses: 

primary.xml.gz
filelists.xml.gz

while there is primary.sqlite.xz and filelists.sqlite.xz available. 
The primary sqlite is actually larger than the primary xml, but 
the filelists sqlite is a good deal smaller than the xml. 

It might make sense to adjust on the createrepo side to xz compress the xml files, but I don't know what that might break compatibility wise.
Comment 1 Ales Kozumplik 2014-06-23 08:43:25 UTC 
I'm looking at 

https://ftp.fau.de/fedora/linux/releases/20/Everything/x86_64/os/repodata/

and 

.filelist.xml.gz is 26M while filelist.sqlite.bz2 is is 27MB.

In rawhide where .xz is used there is some saved bandwidth but it is only 12%:

https://ftp.fau.de/fedora/linux/development/rawhide/x86_64/os/repodata/

That doesn't buy us enough to justify more complex handling and additional processing time from .sqlite to the libsolv format. We're moving away from sqlite, perhaps towards distributing the libsolv MD files directly.
Comment 2 Kevin Fenzi 2014-06-23 14:28:23 UTC 
ok. 

Perhaps however it's worth getting createrepo to make .xz copies of the xml files?