Bug 1889950

Summary: [RFE] Foreman rake to remove old rpms from Satellite.
Product: Red Hat Satellite Reporter: Gary Scarborough <gscarbor>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED WONTFIX QA Contact: Vladimír Sedmík <vsedmik>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.7.0CC: ahumbe, dalley, dkliban, faguiard, ggainey, rchan, ttereshc, wpinheir
Target Milestone: UnspecifiedKeywords: FutureFeature, Reopened, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-10-28 18:01:59 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Gary Scarborough 2020-10-21 04:29:10 UTC
Description of problem:

We should have a foreman rake or other mechanism to remove old and un-needed packages from Satellite.  While this can be accomplished fairly easily on Capsules, there is no easy way to do this on Satellite.

Even with on-demand downloading, a Satellite that has been upgraded several times will have an enormous amount of space used by packages that are no longer used.  Removing a repo and re-adding it to Satellite is virtually impossible.  There are tons of packages that are not being deployed anymore that could easily be removed to free up space.

Having a mechanism to remove packages that haven't been used in 30, 60, or 90 days would greatly reduce space needed.  Then, if for some reason that package is needed again, the on-demand nature of the Satellite should allow the package to be re-downloaded from the CDN.

Comment 1 pulp-infra@redhat.com 2021-04-19 14:11:38 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 2 pulp-infra@redhat.com 2021-04-19 14:11:39 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 3 Gary Scarborough 2021-04-19 14:33:16 UTC
The following KCS could be used to solve this problem:

https://access.redhat.com/solutions/2785021

It is an all or nothing solution, so a rake with some options would be better, but this will technically work.

Comment 4 Tanya Tereshchenko 2021-06-04 20:15:45 UTC
This RFE is not on the roadmap in the short term, we will re-evaluate it in a few months.

Comment 5 Tanya Tereshchenko 2021-10-30 15:20:55 UTC
This RFE is not on the roadmap in the short term, we will re-evaluate it in a few months.

Comment 6 Mike McCune 2022-01-28 22:33:49 UTC
Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team. Thank you.

Comment 7 Mike McCune 2022-03-18 19:44:22 UTC
Thank you for your interest in Red Hat Satellite. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this feel free to contact your Red Hat Account Team. Thank you.

Comment 10 Daniel Alley 2022-08-29 22:35:31 UTC
While we don't have this exact feature we do already have a number of features that enable a similar workflow.

Reclaiming disk space for repositories - convert artifacts for a repository to "on_demand" unless they are part of repository versions present in a provided list to keep
https://pulp.plan.io/issues/8459 

Retain package versions (specific to RPM plugin) - keep the latest N versions of each package and remove too-old packages from the repository
https://pulp.plan.io/issues/5367

There are some limitations with these, the first one doesn't work within a repository version and doesn't discriminate between recently or not-recently downloaded packages, and with the latter you still need to clean up old repository versions to orphan the content entirely (unless combined with the first).  I'm not sure if that is enough to satisfy the RFE?

Comment 11 Ashish Humbe 2022-08-30 14:03:32 UTC
Thank you Daniel for the update. 

I was feeling that this RFE had the main intent to reduce the amount of space required on the satellite server for storing synced packages. With Sat6.11 we introduced "retain_package_versions=X" setting which is not yet very popular but that should help customers in reducing the space consumed on the Satellite server for Pulp storage. 

One thing to confirm with you, if we set retain_package_versions=3 and the repository has an On_Demand sync policy, when any client tries to install an old version ( older than 3 versions kept on satellite) of the kernel or any other package, will it be downloaded on clients request or the client will face 404 error?

Comment 12 Daniel Alley 2022-08-30 14:16:29 UTC
> when any client tries to install an old version ( older than 3 versions kept on satellite) of the kernel or any other package, will it be downloaded on clients request or the client will face 404 error?

It won't be present in the metadata, so clients won't even know about it.  It also wouldn't appear if you browsed the repo directories.

Comment 13 Ashish Humbe 2022-08-30 14:47:32 UTC
Thank you Daniel for the clarity about this feature!

Comment 14 Brad Buckingham 2022-09-02 20:25:18 UTC
Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team.  Thank you.

Comment 15 Brad Buckingham 2022-09-05 22:56:27 UTC
Upon review of our valid but aging backlog the Satellite Team has concluded that this Bugzilla does not meet the criteria for a resolution in the near term, and are planning to close in a month. This message may be a repeat of a previous update and the bug is again being considered to be closed. If you have any concerns about this, please contact your Red Hat Account team.  Thank you.

Comment 16 Gary Scarborough 2022-09-05 23:35:41 UTC
In my opinion, retain_package_versions=3 is not an acceptable replacement for having the ability to clear out old packages.  Putting a limit on packages prevents older packages from every being accessed.  What if I need an older version of the kernel?  I shouldn't be prevented from downloading it.

But at the same time I should be able to clear old packages which have not been access in some amount of time and set those packages back to a status of "hasn't been downloaded".  That frees up space without causing downstream issues when someone needs an old package for testing.

Comment 18 Brad Buckingham 2022-10-28 18:01:59 UTC
Thank you for your interest in Red Hat Satellite. We have evaluated this request, and while we recognize that it is a valid request, we do not expect this to be implemented in the product in the foreseeable future. This is due to other priorities for the product, and not a reflection on the request itself. We are therefore closing this out as WONTFIX. If you have any concerns about this feel free to contact your Red Hat Account Team. Thank you.

Comment 19 Red Hat Bugzilla 2023-09-18 00:23:05 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 120 days