Bug 1715799 - Re-enable zstd compression support in RHEL-8 rpm
Summary: Re-enable zstd compression support in RHEL-8 rpm
Keywords:
Status: VERIFIED
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: rpm
Version: 8.0
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: rc
: 8.2
Assignee: Packaging Maintenance Team
QA Contact: Luca Berton
URL:
Whiteboard:
: 1768147 (view as bug list)
Depends On: 1750648
Blocks: 1755139
TreeView+ depends on / blocked
 
Reported: 2019-05-31 10:08 UTC by Tomas Mraz
Modified: 2020-02-17 13:02 UTC (History)
37 users (show)

Fixed In Version: rpm-4.14.2-26.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Feature Request
Target Upstream Version:


Attachments (Terms of Use)

Description Tomas Mraz 2019-05-31 10:08:20 UTC
As Fedora plans to eventually switch to zstd compression it would be unfortunate if RHEL-8 would not be able to unpack the rpms produced with the default settings on Fedora.

Please consider re-enabling of zstd compression support. I understand that it means adding zstd compressor to RHEL-8, but it might be useful anyway.

Comment 1 Panu Matilainen 2019-06-04 10:32:14 UTC
Re-enabling is a non-issue from rpm point of view. It's the inclusion of zstd that needs to be accounted for.

Comment 2 Dennis Gilmore 2019-07-05 20:03:57 UTC
I will note that you can not build fedora rawhide rpms on RHEL 8 due to the change https://fedoraproject.org/wiki/Changes/Switch_RPMs_to_zstd_compression

Running transaction check
Error: transaction check vs depsolve:
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by binutils-2.32-17.fc31.s390x
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by binutils-gold-2.32-17.fc31.s390x
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by gdb-minimal-8.3.50.20190702-19.fc31.s390x
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by guile22-2.2.6-1.fc31.s390x
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by krb5-libs-1.17-34.fc31.s390x
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by libssh-0.9.0-2.fc31.s390x
rpmlib(PayloadIsZstd) <= 5.4.18-1 is needed by pcre2-10.33-6.fc31.s390x
To diagnose the problem, try running: 'rpm -Va --nofiles --nodigest'.
You probably have corrupted RPMDB, running 'rpm --rebuilddb' might fix the issue.

Comment 3 Jaroslav Mracek 2019-08-27 07:25:41 UTC
From comment 1 it is not clear how to resolve the issue.

Please could you provide additional information how to build RPMs on RHEL-8 for Fedora 31+ (resolve the problem from Comment 2)?

I would like to ask you for a procedure how to build RPM package with support of rpmlib(PayloadIsZstd) for RHEL-8? 

Could you also provide a build of RPM package with rpmlib(PayloadIsZstd) for RHEL-8?

Comment 10 Neal Gompa 2019-08-28 04:38:37 UTC
(In reply to Jaroslav Mracek from comment #3)
> From comment 1 it is not clear how to resolve the issue.
> 
> Please could you provide additional information how to build RPMs on RHEL-8
> for Fedora 31+ (resolve the problem from Comment 2)?
> 
> I would like to ask you for a procedure how to build RPM package with
> support of rpmlib(PayloadIsZstd) for RHEL-8? 
> 
> Could you also provide a build of RPM package with rpmlib(PayloadIsZstd) for
> RHEL-8?

I would have figured it's obvious? The zstd package needs to be imported into RHEL, rpm's build condition switch for zstd support needs to be flipped on, then rpm needs to be rebuilt. That would make it so rpm in RHEL 8 would support zstd payloads in RPMs, and restore compatibility.

Comment 20 Troy Dawson 2019-08-29 13:41:16 UTC
I want to point out that before switching, Fedora did a study on the installation time of gzip and zstd rpm's.
https://fedoraproject.org/wiki/Changes/Switch_RPMs_to_zstd_compression

The speed increase is not trivial.

- 4x faster on tmpfs (8 seconds vs 2 seconds)
- 2.5x faster on ext4 (11 seconds vs 4 seconds)

Three of their four benefits to Fedora, would also be benefitial to RHEL.
- Faster installations/upgrades of user systems
- Faster koji builds (installations in build roots)
- Faster container builds

Have you every tested out Ubuntu or Debian and wondered why their package installs seem so snappy and fast.  This is why.

Faster package installs make the user feel like their system is faster.*

* - This is an opinion, but I have seen it before, so I believe it.

Comment 22 Neal Gompa 2019-08-29 18:53:49 UTC
> Have you every tested out Ubuntu or Debian and wondered why their package installs seem so snappy and fast.  This is why.

Debian/Ubuntu systems are not using zstd compression. They started using xz compression with Debian 9 / Ubuntu 18.04, and prior to that, they used gzip compression.

They are exploring the usage of zstd compression for Debian 11 / Ubuntu 20.04 for similar reasons, though.

Comment 23 Neal Gompa 2019-08-29 18:57:06 UTC
(In reply to Neal Gompa from comment #22)
> 
> Debian/Ubuntu systems are not using zstd compression. They started using xz
> compression with Debian 9 / Ubuntu 18.04, and prior to that, they used gzip
> compression.
> 

To clarify, they used gzip compression for the longest time *because* it's faster than xz and cheaper memory-wise.

Comment 24 Phil Wyett 2019-08-29 19:58:24 UTC
We seem to be drifting off topic here.

What is I believe is desired is the adding of zstd package(s) to RHEL 8 and enabling of zstd in rpm of the distro. This will allow others and myself the ability to build/unpack etc. fedora packages that use zstd package compression on RHEL 8.

I do not believe anyone is pushing for use of zstd package compression, well not for RHEL 8 anyway, but future possibly. ;-)

Comment 25 Pádraig Brady 2019-09-03 09:58:50 UTC
I think it would make a lot of sense to add zstd to RHEL8 to increase compat with Fedora and due to its general usefulness.

For reference bug 1743983 is tracking addition to EPEL, but that would be much less desirable.

Comment 38 Pádraig Brady 2019-10-18 13:56:54 UTC
I don't have access to bugs 1750648 and 1755139.
Am I correct in assuming zstd is not included in the main RHEL repos?
I guess I should deprecate the EPEL one so?

Comment 39 Nico Kadel-Garcia 2019-11-03 18:37:53 UTC
zstd is an added compilation time capability for RPM itself. *Exaxtly* the same problem happened about 10 years ago, when RPM switched from using gzip to using xz.

There is some discussion of updating RPM on RHEL 8.2 and thus on CentOS 8.2 to include zstd. Until then, there is a workaround for "mock", at least.

* yum install podman
* Add these lines to /etc/mock/templates/fedoa-31.tpl
* * config_opts['use_bootstrap_container'] = True
* * config_opts['use_bootstrap_images'] = True

The bug over in the "mock" issues is discussed at: https://github.com/rpm-software-management/mock/issues/390 .

This works for mock, but does not solve the issue for "rpm2cpio" on RHEL 7, RHEL 8, or the CentOS equivalents.

Comment 40 Neal Gompa 2019-11-03 18:48:49 UTC
(In reply to Nico Kadel-Garcia from comment #39)
> zstd is an added compilation time capability for RPM itself. *Exaxtly* the
> same problem happened about 10 years ago, when RPM switched from using gzip
> to using xz.

Well, it wasn't exactly the same. Back then, RPM in RHEL 5 didn't support xz at all. There was the experimental lzma thing, but when Fedora switched, we had the finalized xz payload. That had to be backported to RHEL so that our Koji builders (which ran on RHEL back then) would function properly.

Nowadays, the Fedora Koji almost entirely runs on Fedora now, so most of those problems don't exist anymore. It was forced to switch because of the introduction of mandatory rich dependencies in the core package set.

I guess we could technically go back to using RHEL for builders with RHEL 8, but I hope we don't. As a project, it makes life a lot better when we dogfood our own distribution, especially since the community can control and improve things on demand, unlike with RHEL (as evidenced by this ticket...).

Comment 41 Nico Kadel-Garcia 2019-11-03 20:38:22 UTC
"Older versions of RPM cannot handle the new compression algorithm used in new RPM releases and in new operating systems" sounds like exactly the same issue to me.

If the builders in koji are Fedora based, I'd see no reason to roll them back to RHEL 8. That would trigger precisely this problem where it's not currently an issue.

Comment 42 Panu Matilainen 2019-11-04 09:38:09 UTC
*** Bug 1768147 has been marked as a duplicate of this bug. ***

Comment 46 Robert Hancock 2020-01-09 01:32:56 UTC
Another reason this should be a priority is that the retrace server for Fedora bug reporting apparently runs on CentOS 8, and is unable to unpack Fedora 31 RPMs, which means it is entirely broken for Fedora 31: https://github.com/abrt/retrace-server/issues/258

Comment 47 Miroslav Suchý 2020-01-09 12:44:42 UTC
@hancockrwd retrace server runs on RHEL 7. We plan to upgrade to RHEL 8. But yeah, the issue on retrace server is there. We are aware of that. Retrace server use mock for retracing and there was one issue in Mock which did not allow retrace to use Bootstrap Image feature. In upcoming release retrace will use podman directly.

To sum it up, yes, it does not work now, but will be fixed soon and zstd presence in RHEL8 will have no impact on this.

Comment 48 Troy Dawson 2020-01-22 22:13:15 UTC
CentOS Stream was just updated with rpm-4.14.2-36.el8.x86_64 and libzstd-1.4.2-2.el8.x86_64.

I have verified that a Fedora 32 rpm, that I was not able to open before, I can now open.

# rpm -q rpm
rpm-4.14.2-25.el8.x86_64
# rpm2cpio marble-19.08.3-1.fc32.x86_64.rpm | cpio -idmv
<garbage on the screen>
cpio: premature end of file

# dnf update rpm
<Updated rpm as well as installed libzstd>
# rpm -q rpm libzstd
rpm-4.14.2-36.el8.x86_64
libzstd-1.4.2-2.el8.x86_64
# rpm2cpio marble-19.08.3-1.fc32.x86_64.rpm | cpio -idmv
<files in marble are listed and written>
1190 blocks
#

I think that is why these are in CentOS Stream, so they can be tested before this hits the next RHEL release (hoping 8.2).
Anyway, just want to say, success.  And thank you for getting this done.
I know it will be a few months before this get's released, if it's going out with 8.2, but I'm glad I can test it now.


Note You need to log in before you can comment on or make changes to this bug.