Bug 1683769 - dnf missing zchunk support
Summary: dnf missing zchunk support
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: All
OS: Linux
unspecified
urgent
Target Milestone: ---
Assignee: Jaroslav Rohel
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-02-27 18:37 UTC by Adam Williamson
Modified: 2019-03-21 14:40 UTC (History)
11 users (show)

Fixed In Version: dnf-4.2.1-1.fc30 dnf-4.2.1-1.fc29
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-03-19 19:38:43 UTC
Type: Bug


Attachments (Terms of Use)

Description Adam Williamson 2019-02-27 18:37:39 UTC
As discovered and discussed in investigation of the failed Fedora-Rawhide-20190227.n.0 compose:

https://pagure.io/dusty/failed-composes/issue/1529

with current versions of dnf, libdnf, createrepo_c and librepo, we wind up generating repositories containing a zchunk-compressed comps file with repomd type 'group_gz_zck', which libdnf `Repo.getCompsFn()` returns when asked. This is a problem because dnf `Base.read_comps()` calls `getCompsFn()` then tries to decompress and read whatever file is returned itself, but it does not have the ability to recognize or decompress zchunk files, so it winds up trying to read the compressed file as plain XML, and blowing up. This breaks composes.

There are various ways we can work around this in the short term, but making dnf actually capable of recognizing and decompressing zchunk files (or moving this functionality into libdnf, I guess) seems like it should be done.

Comment 1 Jonathan Dieter 2019-02-28 22:19:08 UTC
While this is definitely should be done, I believe the correct short-term fix is to have librepo only automatically return zchunked files for primary, filelists and other, since they are decompressed by libsolv.  All other file types should be returned in the format requested, with the zchunk versions only being returned if specifically requested.  There's a PR at https://github.com/rpm-software-management/librepo/pull/141 that implements this behavior.

Comment 3 Fedora Update System 2019-03-13 13:41:51 UTC
dnf-plugins-core-4.0.6-1.fc29 libdnf-0.28.0-1.fc29 libcomps-0.1.11-1.fc29 createrepo_c-0.12.2-1.fc29 librepo-1.9.5-1.fc29 dnf-4.2.1-1.fc29 dnf-plugins-extras-4.0.4-1.fc29 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-a03be2cebe

Comment 4 Fedora Update System 2019-03-13 13:42:45 UTC
dnf-plugins-core-4.0.6-1.fc30 libdnf-0.28.0-1.fc30 libcomps-0.1.11-1.fc30 createrepo_c-0.12.2-1.fc30 librepo-1.9.5-1.fc30 dnf-4.2.1-1.fc30 dnf-plugins-extras-4.0.4-1.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-bc9607a8dc

Comment 5 Fedora Update System 2019-03-13 15:37:04 UTC
createrepo_c-0.12.2-1.fc30, dnf-4.2.1-1.fc30, dnf-plugins-core-4.0.6-1.fc30, dnf-plugins-extras-4.0.4-1.fc30, libcomps-0.1.11-1.fc30, libdnf-0.28.0-1.fc30, librepo-1.9.5-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-bc9607a8dc

Comment 6 Jonathan Dieter 2019-03-13 23:06:40 UTC
I've tested libdnf-0.28.0-1.fc30 with a test repository with zchunked updateinfo.xml.zck, and dnf group list worked perfectly.  I also tested GNOME software, and it showed the update descriptions from the zchunked updatedinfo.xml.zck.  It looks like this is fixed now.

Comment 7 Fedora Update System 2019-03-14 18:51:34 UTC
createrepo_c-0.12.2-1.fc29, dnf-4.2.1-1.fc29, dnf-plugins-core-4.0.6-1.fc29, dnf-plugins-extras-4.0.4-1.fc29, libcomps-0.1.11-1.fc29, libdnf-0.28.0-1.fc29, librepo-1.9.5-1.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-a03be2cebe

Comment 8 Fedora Update System 2019-03-15 13:09:37 UTC
createrepo_c-0.12.2-1.fc30 dnf-4.2.1-1.fc30 dnf-plugins-core-4.0.6-1.fc30 dnf-plugins-extras-4.0.4-1.fc30 libcomps-0.1.11-1.fc30 libdnf-0.28.0-1.fc30 librepo-1.9.5-1.fc30 swid-tools-0.7.3-1.fc30 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-bc9607a8dc

Comment 9 Fedora Update System 2019-03-16 20:17:33 UTC
createrepo_c-0.12.2-1.fc30, dnf-4.2.1-1.fc30, dnf-plugins-core-4.0.6-1.fc30, dnf-plugins-extras-4.0.4-1.fc30, libcomps-0.1.11-1.fc30, libdnf-0.28.0-1.fc30, librepo-1.9.5-1.fc30, swid-tools-0.7.3-1.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-bc9607a8dc

Comment 10 Fedora Update System 2019-03-19 19:38:43 UTC
createrepo_c-0.12.2-1.fc30, dnf-4.2.1-1.fc30, dnf-plugins-core-4.0.6-1.fc30, dnf-plugins-extras-4.0.4-1.fc30, libcomps-0.1.11-1.fc30, libdnf-0.28.0-1.fc30, librepo-1.9.5-1.fc30, swid-tools-0.7.3-1.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 11 Fedora Update System 2019-03-20 22:13:04 UTC
createrepo_c-0.12.2-1.fc29, dnf-4.2.1-1.fc29, dnf-plugins-core-4.0.6-1.fc29, dnf-plugins-extras-4.0.4-1.fc29, libcomps-0.1.11-1.fc29, libdnf-0.28.0-1.fc29, librepo-1.9.5-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.

Comment 12 Fedora Update System 2019-03-21 14:40:41 UTC
createrepo_c-0.12.2-1.fc29, dnf-4.2.1-1.fc29, dnf-plugins-core-4.0.6-1.fc29, dnf-plugins-extras-4.0.4-1.fc29, libcomps-0.1.11-1.fc29, libdnf-0.28.0-1.fc29, librepo-1.9.5-1.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.