Bug 2043609 - pulpcore-workers grow very large when repositories have many changelog entries
Summary: pulpcore-workers grow very large when repositories have many changelog entries
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.10.2
Hardware: Unspecified
OS: Unspecified
Target Milestone: 6.11.0
Assignee: satellite6-bugs
QA Contact: Lai
Depends On:
TreeView+ depends on / blocked
Reported: 2022-01-21 15:52 UTC by wclark
Modified: 2022-09-28 17:15 UTC (History)
5 users (show)

Fixed In Version: tfm-pulpcore-python-pulp-rpm-3.17.3
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2043712 (view as bug list)
Last Closed: 2022-07-05 14:32:15 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Private Priority Status Summary Last Updated
Github pulp pulp_rpm pull 2333 0 None Merged Implement a changelog limit during parsing of metadata and upload of RPMs 2022-01-21 15:52:58 UTC
Red Hat Product Errata RHSA-2022:5498 0 None None None 2022-07-05 14:32:32 UTC

Description wclark 2022-01-21 15:52:58 UTC
Split off from https://bugzilla.redhat.com/show_bug.cgi?id=1994397 which had grown to include several other distinct issues

Some repositories, such as the RHEL 7 repositories, contain a large and growing # of changelogs, which is causing pulpcore workers to consume several GB of additional memory

The patch https://github.com/pulp/pulp_rpm/pull/2333 limits the # of changelog entries kept by Pulp to 10 by default. this achieves a lower memory footprint for the pulpcore workers when using such repositories, by limiting the # of changelog entries seen by clients using the `dnf changelog` command

Comment 2 Daniel Alley 2022-01-21 16:26:09 UTC
Additional context in terms of communicating about this change with customers:

The full list of raw changelogs are always available if you have the RPM on hand. If it's installed on the system you can view them with "rpm -qa --changelogs $package".

All of this only applies to the separate copy of the changelogs kept repo-wide, which is queried when you use "dnf changelog $package" or "dnf repoquery --changelogs $package", and available for packages that aren't installed on your system.

It's rare to need more than a couple of changelogs for a package you don't have installed, which is why distros consolidated on keeping only 10 in the repo metadata.

Comment 3 Daniel Alley 2022-01-24 22:48:41 UTC
I'm not sure what annotations to add, but the pulp_rpm releases which have this patch are 3.14.11 or 3.17.2, for satellite 6.10 and 7.0 respectively.

Comment 8 Griffin Sullivan 2022-05-03 15:36:31 UTC
See verification in https://bugzilla.redhat.com/show_bug.cgi?id=1994397.

Comment 12 errata-xmlrpc 2022-07-05 14:32:15 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.