Bug 1773202

Summary: PackageKit accumulates useless RPM packages in /var/cache/PackageKit/metadata and may fill root file system
Product: [Fedora] Fedora Reporter: mlaverdiere <marco.laverdiere>
Component: PackageKitAssignee: Richard Hughes <rhughes>
Status: CLOSED NOTABUG QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: dcb314, fweimer, hugh, jan.skowron, jonathan, klember, patdung100+redhat, rdieter, rhughes, samuel-rhbugs, smakarov, smparrish, tomek, yann
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-11-21 07:18:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description mlaverdiere 2019-11-16 13:23:10 UTC
This is the following up of this bug report: https://bugzilla.redhat.com/show_bug.cgi?id=1306992

The problem is that PackageKit (with gnome-software) automatically download rpm files to be updated, but if the user updates its system with dnf, these rpm files will be kept in /var/cache/PackageKit/metadata and may accumulate to such amount that it can saturate the disk space (problem experienced and reported by some users in the previous bug report).

A partial fix has been found, as now, the PackageKit cache cleanup would occur, but only when a new RPM for the same package is available, which is a progress, but maybe not enough to prevent disk space saturation with useless rpm files. See this: https://github.com/hughsie/PackageKit/commit/66f2966fb6484a86324f98302659c5f27a02b1b5

The ideal solution would be that PackageKit deletes rpm files whenever they are already installed or outdated. Otherwise, maybe another solution could be to not set gnome-software by default to download rpm files for updates.

The workaround for a user who wants to avoid the risk of disk space saturation on its root partition with useless rpm files, involves the following:

1. Disabling PackageKit (gnome-software) automatic download of updates, by issuing this command in a terminal:

gsettings set org.gnome.software download-updates false

2. Setting up PackageKit to not cache packages at all (not sure if it is sill effective, according to this: https://unix.stackexchange.com/questions/265755/fedora-23-can-i-safely-delete-files-in-var-cache-packagekit-metadata-updates/283556).

Comment 1 Kalev Lember 2019-11-16 13:52:33 UTC
(In reply to mlaverdiere from comment #0)
> This is the following up of this bug report:
> https://bugzilla.redhat.com/show_bug.cgi?id=1306992
> 
> The problem is that PackageKit (with gnome-software) automatically download
> rpm files to be updated, but if the user updates its system with dnf, these
> rpm files will be kept in /var/cache/PackageKit/metadata and may accumulate
> to such amount that it can saturate the disk space (problem experienced and
> reported by some users in the previous bug report).
> 
> A partial fix has been found, as now, the PackageKit cache cleanup would
> occur, but only when a new RPM for the same package is available, which is a
> progress, but maybe not enough to prevent disk space saturation with useless
> rpm files. See this:
> https://github.com/hughsie/PackageKit/commit/
> 66f2966fb6484a86324f98302659c5f27a02b1b5
> 
> The ideal solution would be that PackageKit deletes rpm files whenever they
> are already installed or outdated. Otherwise, maybe another solution could
> be to not set gnome-software by default to download rpm files for updates.

What you describe here is exactly what the commit above does. Is it not working for you?

Comment 2 mlaverdiere 2019-11-16 14:59:00 UTC
No it does not seem to work, on a freshly installed Fedora 31 Workstation

Here is how I propose to test it:

1. On a Fedora workstation, wait for gnome-software notification on system/package updates availability

2. Proceed with dnf to update the system/packages.

3. Reboot the system

4. Open and refresh gnome-software

5. Look in /var/cache/PackageKit/(version)/metadata...: there should be rpm packages that have already been installed with dnf on the system; these package are useless and when they accumulate, they may saturate disk space

To my understanding (correct me if I'm wrong), the commit makes it so that an rpm package in /var/cache/PackageKit... will only be replaced if a newer package becomes available; it wont clean up (delete/erase) already installed packages

Comment 3 David Binderman 2019-11-16 15:17:20 UTC
Maybe another way to reduce the problem is to have
PackageKit look at how much free disk space is left and
use no more than a user defined percentage of the disk.

10% might be a sensible default. It is only a cache after all.

Comment 4 Kalev Lember 2019-11-16 15:44:34 UTC
(In reply to mlaverdiere from comment #2)
> No it does not seem to work, on a freshly installed Fedora 31 Workstation
> 
> Here is how I propose to test it:
> 
> 1. On a Fedora workstation, wait for gnome-software notification on
> system/package updates availability
> 
> 2. Proceed with dnf to update the system/packages.
> 
> 3. Reboot the system
> 
> 4. Open and refresh gnome-software
> 
> 5. Look in /var/cache/PackageKit/(version)/metadata...: there should be rpm
> packages that have already been installed with dnf on the system; these
> package are useless and when they accumulate, they may saturate disk space

Can you be more specific please? What packages are you seeing there? What versions of these do you have installed? This all works correctly in my testing.

Also, step 5 is not enough to trigger gnome-software to delete the old files. They are deleted once a day when the background downloads are done (this is controlled by org.gnome.software download-updates gsetting, so make sure you have this enabled when testing this). The same operation that downloads the rpms also cleans up the old rpms from the cache. You can manually trigger it from the UI by clicking "Download" in the "Requires Restart" section, or from the CLI with 'pkcon update --only-download'.


> To my understanding (correct me if I'm wrong), the commit makes it so that
> an rpm package in /var/cache/PackageKit... will only be replaced if a newer
> package becomes available; it wont clean up (delete/erase) already installed
> packages

No, that's incorrect. It cleans up already installed packages and only keeps those that are needed for updating the system.

Comment 5 mlaverdiere 2019-11-16 17:12:42 UTC
OK, I'll do some more testing and report back.

Comment 6 mlaverdiere 2019-11-21 00:19:04 UTC
According to my testing in the last days, it appears true that there's a clean up of the rpm packages downloaded in /var/cache/PackageKit... that seems to happen at some point, so already installed rpm packages (with dnf or otherwise) are deleted. This clean-up seems to happen when gnome-software checks/download a new set of packages to install. In other words, the clean-up won't happen immediately after a reboot or after a refresh of gnome-software. 

So basically, Kalev (comment 4) seems to be right (except that I didn't manage to trigger the clean-up with the command "pkcon update --only-download". Issuing this command in the terminal (as root) did nothing...).

Maybe time to close this report, but to be sure that we're done with this question/issue once and for all, maybe we could wait for some other input.

Comment 7 Kalev Lember 2019-11-21 07:18:54 UTC
Thanks for verifying. Let's close it then.