Summary: Would it be possible to create the 'gen' subdirectory in '/var/cache/dnf/updates-<hex-string>/repodata' up front as soon as that directory is created, and to avoid removing it while that cache is still in use? Details: The Mozilla Firefox build scripts have some code to "bootstrap" your local machine to install build prerequisites for devs. This code should be run regularly to ensure dependencies are up to date or else weird build failures can occur. To avoid being annoying, the script tries to avoid prompting the user for their password to `sudo dnf install` where possible by checking whether the prerequisites are all present: https://searchfox.org/mozilla-central/rev/eea0fac2cb8cae1e5a2e38959eeaa0b7f6d23456/python/mozboot/mozboot/base.py#466,473,485-486 It does that using `dnf --cacheonly group list --installed' (without using sudo, obviously, or that would defeat the purpose of the check!) Unfortunately, this can sometimes fail: ``` $ dnf --cacheonly group list --installed Last metadata expiration check: 0:42:44 ago on Wed 08 Dec 2021 09:00:58 GMT. [Errno 13] Permission denied: '/var/cache/dnf/updates-7eea87b22825bc0d/repodata/gen' $ sudo ls -al /var/cache/dnf/updates-7eea87b22825bc0d/repodata/ total 15200 drwxr-xr-x. 1 root root 868 Dec 8 09:00 . drwxr-xr-x. 1 root root 56 Dec 8 09:00 .. -rw-r--r--. 1 root root 16908 Dec 8 09:00 <a-bunch-of-these>.xml.zck -rw-r--r--. 1 root root 8306 Dec 8 09:00 repomd.xml ``` Retrying with sudo, it succeeds and the 'gen' subdirectory is created: ``` $ sudo dnf --cacheonly group list --installed Last metadata expiration check: 0:43:02 ago on Wed 08 Dec 2021 09:00:58 GMT. Installed Groups: <a-bunch-of-groups-listed> $ sudo ls -al /var/cache/dnf/updates-7eea87b22825bc0d/repodata/ total 15200 drwxr-xr-x. 1 root root 874 Dec 8 09:44 . drwxr-xr-x. 1 root root 56 Dec 8 09:00 .. -rw-r--r--. 1 root root 16908 Dec 8 09:00 <a-bunch-of-these>.xml.zck drwxr-xr-x. 1 root root 0 Dec 8 09:44 gen -rw-r--r--. 1 root root 8306 Dec 8 09:00 repomd.xml ``` Now if I rerun `dnf --cacheonly group list --installed` (without sudo) it too succeeds. So it looks like the 'gen' subdirectory being missing breaks `dnf --cacheonly` (when run without sudo) because the permissions on the parent directory don't allow regular users to create it. If it already exists, everything works.
Looking at the output from `sudo ls /var/cache/dnf/updates-*/repodata/` it seems that only one of the caches contains a 'gen' subdirectory. If that indicates that the 'gen' subdirectory is sometimes removed, that would also be an issue.
Hi, the `gen` subdirectory is used by dnf to store decompressed versions of group metadata. E.g. repodata/e9328e1b5448d9e9f9b4908632a146aa768fd07ef2bc52c176ad85af06233170-comps-Everything.x86_64.xml.gz decompress into repodata/gen/groups.xml. The `gen` subdirectory does not get removed, but it is only part of repositories with comps groups metadata. Unfortunately the groups metadata are not decompressed when they are downloaded (thus as a root) but only once they are needed (e.g. by `dnf group` command which might by executed as regular user). Until this is fixed, can you as a workaround omit the `--cacheonly` parameter?
Thanks for your reply and for the explanation, Marek. For now we have indeed dropped the `--cacheonly` parameter. It would be nice if we could keep using it but it's a relatively small inconvenience.
PR which should fix the issue: https://github.com/rpm-software-management/dnf/pull/1803
FEDORA-2022-e119fcc7d5 has been submitted as an update to Fedora 37. https://bodhi.fedoraproject.org/updates/FEDORA-2022-e119fcc7d5
FEDORA-2022-e119fcc7d5 has been pushed to the Fedora 37 stable repository. If problem still persists, please make note of it in this bug report.
PR with test: https://github.com/rpm-software-management/ci-dnf-stack/pull/1087