Description of problem: New feature request. Possibility to clean up repositories on capsules to clean up old packages that it is not need it. 'Is there an easy way to cleanup repositories? I mean without delete them and rebuild content views, etc. It should be great if I could remove all packages and let satellite download them again only "on demand".' Unfortunately, this is not possible and I'm not aware of any RFE requesting this functionality. Version-Release number of selected component (if applicable): Satellite 6.2.4 How reproducible: Steps to Reproduce: 1. 2. 3. Actual results: Expected results: Additional info:
When you mention deleting old packages that are not needed, can you provide an example? Is this a scenario, where an existing repository is synced, it brings in a new version of a package and the old one remains? If so, does the user have 'Mirror on Sync' set on the repositories? If so, that should remove the old versions.
it seems that we found Satellite: server1 "du -sh /var/lib/pulp/content" 129G /var/lib/pulp/content Capsule: capsule1 "du -sh /var/lib/pulp/content" 118G /var/lib/pulp/content just provisioned a new Satellite Capsule capsule1 and our default_download_policy is on_demand. Disk usage is 130G in /var/lib, after first synchronization with no hosts associated to this capsule. After that we have self associated this capsule (no package update has been done). Is that the expected behavior of "lazy sync"?
Since there is a lot of other data stored in /var/lib/ (such as databases), it's hard to compare the new capsule's disk usage at /var/lib/ to the old capsule's usage at /var/lib/pulp/content/ I'm moving this to the "Capsule Content" component for now, since I think the request is for new or different behavior with regards to managing content in Pulp. If there's a new Pulp feature to implement, I'm happy to talk about that, but this looks more like Katello's domain for now.
Hi, this affects on Satellite server too
> When you mention deleting old packages that are not needed, can you provide an example? This applies to rolling releases 6Server, 7Server: sssd-1.11.2-65.el7.x86_64 sssd-1.11.2-68.el7_0.5.x86_64 sssd-1.11.2-68.el7_0.6.x86_64 sssd-1.12.2-58.el7.x86_64 sssd-1.12.2-58.el7_1.6.x86_64 sssd-1.12.2-58.el7_1.14.x86_64 sssd-1.12.2-58.el7_1.17.x86_64 sssd-1.12.2-58.el7_1.18.x86_64 sssd-1.13.0-40.el7.x86_64 sssd-1.13.0-40.el7_2.1.x86_64 sssd-1.13.0-40.el7_2.2.x86_64 sssd-1.13.0-40.el7_2.4.x86_64 sssd-1.13.0-40.el7_2.9.x86_64 sssd-1.13.0-40.el7_2.12.x86_64 sssd-1.14.0-43.el7.x86_64 sssd-1.14.0-43.el7_3.4.x86_64 Usually you only need the latest revision of packages, and over time it tends to store obsolete packages.
Hi Edu and carlospec, Regarding comment 7, do the repositories containing sssd have the 'mirror_on_sync' setting enabled? The goal of that feature is to have the Satellite repository mirror exactly the content of the source repository. So, if the source repository removes the older versions and the repository is re-synced, they will no longer exist on the Satellite either.
Justin, would the solution for bug 1380799 address the observation provided in comment 4?
Hello Brad, Regarding comment #8, mirror_on_sync is enabled. Note that all versions should be available (you could need to reinstall or downgrade to an old version) but they needn't be downloaded if you use "on demand" policy. Imo the key is that, once a package is downloaded, it's stored "forever". But in a few weeks the probability that you need it again will be low because you should use newer versions.
Brad, I dont' think it would. The only thing i can of would be to have some sort of auto-expire mode that removes packages after they haven't been used for some amount of time. This would require some changes in pulp and katello. Its sounding more and more like a squid proxy :)
Thanks for the feedback all. The bugzilla that I referenced in comment 9 (bug 1380799) should help reduce some of the space/resource utilization on the capsule. That feature essentially allows for the capsule to be configured for 'on_demand', similar to the behavior on Satellite. That feature is not currently available in 6.2.z; however, it looks like it should be coming in 6.3. That said, bug 1380799 does not address a scenario where content has been synced, but is no longer used and possibly obsolete. Is it correct to assume that that behavior is is requested? (e.g. A means to define a policy for content expiration or obsoleting, which will result in that content being deleted from the filesystem when met).
> A means to define a policy for content expiration or obsoleting, which will result in that content being deleted from the filesystem when met Right, that would be great.
(In reply to Justin Sherrill from comment #11) > Brad, I dont' think it would. The only thing i can of would be to have > some sort of auto-expire mode that removes packages after they haven't been > used for some amount of time. This would require some changes in pulp and > katello. > > Its sounding more and more like a squid proxy :) Pulp's yum importer does have a setting for how many versions to keep of any particular package. That addresses this use case of not wanting to keep older versions. But it does not make older versions available on-demand, which is an interesting idea. It removes older versions from the repo.
Thats a good point Michael and that might be good enough for this use case. carlospec if we exposed that option would that help? It would give you the ability to say only keep X number of versions of each package. So you could only keep the latest 3 version for example. We could make that configurable globally for a capsule or possibly at the individual repo level on the main Satellite
I think it would be preferable to make older versions available (downgrades, missed dependencies ¿?). Anyway this parameter could open the door for a repository clean up (sync policy "on demand"): "retain_old_count": "1" -> sync -> remove "retain_old_count" -> sync I've tried it without success: # hammer repository info --id 12 | grep Packages Packages: 18428 # grep retain /etc/pulp/server/plugins.conf.d/yum_importer.json "retain_old_count": "1" # hammer repository synchronize --id 12 [...................................................................................................................................................] [100%] No new packages. # hammer repository info --id 12 | grep Packages Packages: 18428 # hammer package list --repository-id 12 --search sssd-client | wc -l 58 Maybe I'm missing something (I've restarted satellite services).
Carlos, That feature isn't there, but it is present in a subsystem of Satellite (pulp), so it would be trivial to expose and more likely to come sooner than a true 'expire old packages' feature. I'll go ahead and close this BZ to an RFE to implement true "old package expiring" and use this one to implement 'retain_old_Count' -Justin
(In reply to Justin Sherrill from comment #17) > Carlos, > > That feature isn't there, but it is present in a subsystem of Satellite > (pulp), so it would be trivial to expose and more likely to come sooner than > a true 'expire old packages' feature. > > I'll go ahead and close this BZ to an RFE to implement true "old package > expiring" and use this one to implement 'retain_old_Count' > > -Justin Thanks Justin, please notify to us the new RFE to attach into the support case.
Created redmine issue http://projects.theforeman.org/issues/21395 from this bug
Thank you for your interest in Satellite 6. We have evaluated this request, and we do not expect this to be implemented in product in the forseeable future. We are therefore closing this out as WONTFIX. If you have any concerns about this, please feel free to contact Rich Jerrido or Bryan Kearney. Thank you.