Bug 1253806 - Can't clean metadata of a repo spcified via CLI
Can't clean metadata of a repo spcified via CLI
Status: CLOSED DUPLICATE of bug 1278225
Product: Fedora
Classification: Fedora
Component: dnf (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: packaging-team-maint
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-14 14:29 EDT by Tomas Tomecek
Modified: 2017-02-17 07:48 EST (History)
10 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-01-07 12:41:19 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Tomas Tomecek 2015-08-14 14:29:43 EDT
Description of problem:

When I specify a repo via CLI using `--repofrompath`, there is no way to `clean` that repo except manually remove stuff in `/var/dnf/cache`. I even tried `metadata_expire=0` and that didn't help.


Version-Release number of selected component (if applicable):
dnf-1.0.2-3.fc24.noarch


How reproducible:
$ dnf --repofrompath f22,http://download.fedoraproject.org/pub/fedora/linux/releases/22/Everything/x86_64/os/ --enablerepo f22 update
Added f22 repo from http://download.fedoraproject.org/pub/fedora/linux/releases/22/Everything/x86_64/os/
Last metadata expiration check performed 1 day, 22:50:31 ago on Wed Aug 12 15:22:00 2015.
Dependencies resolved.
Nothing to do.
Complete!

$ dnf --repofrompath f22,http://download.fedoraproject.org/pub/fedora/linux/releases/22/Everything/x86_64/os/ --enablerepo f22 clean all
Cleaning repos: f22
Cleaning up Everything

$ dnf --repofrompath f22,http://download.fedoraproject.org/pub/fedora/linux/releases/22/Everything/x86_64/os/ --enablerepo f22 update
Added f22 repo from http://download.fedoraproject.org/pub/fedora/linux/releases/22/Everything/x86_64/os/
Last metadata expiration check performed 1 day, 22:52:01 ago on Wed Aug 12 15:22:00 2015.
Dependencies resolved.
Nothing to do.
Complete!

And when I delete `/var/cache/dnf/f22*` I'll get my update.
Comment 1 Michal Domonkos 2015-12-10 13:10:11 EST
Tomas,

Are you still seeing this?  If yes, could you please attach
1) the output of "tree /var/cache/dnf" and
2) the /var/log/dnf.log file after you reproduce the issue?

For me it does clean the metadata for a repo specified with --repofrompath when I run "dnf clean all" (tried the exact steps as you).

Thanks!
Comment 2 Michal Domonkos 2015-12-11 12:18:01 EST
While your original report doesn't imply any of that, 2 cases come to mind which would cause this behavior:

1) Running the "clean all" command as a regular user, which would only clean the user cache instead of the system one

2) Leaving out --repofrompath when running the "clean all" command, which would not clean up any pre-existing cache for that particular repo
Comment 3 Tomas Tomecek 2016-01-04 05:35:06 EST
Still happening with dnf-1.1.5-1.fc24.noarch

Here's what I'm doing:

1. Trying to get kernel update from different Fedora release by adding repo paths via command line:

$ sudo dnf --disablerepo=* --repofrompath f23,http://download.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/ --enablerepo f23 --repofrompath f23-u,http://ftp.linux.cz/pub/linux/fedora/linux/updates/23/x86_64/ --enablerepo f23-u --repofrompath f23-ut,http://ftp.linux.cz/pub/linux/fedora/linux/updates/testing/23/x86_64/ --enablerepo f23-ut --disableexcludes all update kernel
Added f23-u repo from http://ftp.linux.cz/pub/linux/fedora/linux/updates/23/x86_64/
Added f23-ut repo from http://ftp.linux.cz/pub/linux/fedora/linux/updates/testing/23/x86_64/
Added f23 repo from http://download.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/
f23-u     4.3 MB/s |  16 MB     00:03
f23       8.8 MB/s |  43 MB     00:04
f23-ut    5.7 MB/s | 5.4 MB     00:00
Last metadata expiration check performed 0:00:01 ago on Mon Jan  4 11:10:47 2016.
Dependencies resolved.
Nothing to do.
Complete!

2. in root console I clean everything

# dnf clean all
Cleaning repos: rpmfusion-free-updates rawhide rpmfusion-free-updates-testing rpmfusion-free
Cleaning up Everything


There is no f23* in the list.

3. And now when I rerun the first command, it won't even try to fetch metadata b/c they were not cleaned in second step:

$ sudo dnf --disablerepo=* --repofrompath f23,http://download.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/ --enablerepo f23 --repofrompath f23-u,http://ftp.linux.cz/pub/linux/fedora/linux/updates/23/x86_64/ --enablerepo f23-u --repofrompath f23-ut,http://ftp.linux.cz/pub/linux/fedora/linux/updates/testing/23/x86_64/ --enablerepo f23-ut --disableexcludes all update kernel
Added f23-u repo from http://ftp.linux.cz/pub/linux/fedora/linux/updates/23/x86_64/
Added f23-ut repo from http://ftp.linux.cz/pub/linux/fedora/linux/updates/testing/23/x86_64/
Added f23 repo from http://download.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/
Last metadata expiration check performed 0:11:57 ago on Mon Jan  4 11:10:47 2016.
Dependencies resolved.
Nothing to do.
Complete!
Comment 4 Michal Domonkos 2016-01-04 11:05:55 EST
OK, so the reason dnf doesn't fetch anything in step 3 is that the previous "clean all" command didn't clean the metadata originally fetched with the --repofrompath in step 2.  That's because the clean command operates on the enabled repos only (see the man page).

The correct usage therefore is to define the repos also when issuing the clean command:

# dnf --disablerepo=* --repofrompath f23,http://download.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/ --enablerepo f23 --repofrompath f23-u,http://ftp.linux.cz/pub/linux/fedora/linux/updates/23/x86_64/ --enablerepo f23-u --repofrompath f23-ut,http://ftp.linux.cz/pub/linux/fedora/linux/updates/testing/23/x86_64/ --enablerepo f23-ut clean all

That will clean the f23* metadata for you.  (To make it less verbose, you can always set up an alias (e.g. dnf23) for all f23 operations, or just create a .repo file.)

Now, the question is whether this is really what we want the clean command to be like.  Maybe we should just include any repos that have some metadata in the cache (and not just the enabled ones) when cleaning up.  On the other hand, that would cost the users some flexibility.  I'm going to include this BZ in a related topic [1] and leave it open for the time being.

Thanks!
Comment 5 Michal Domonkos 2016-01-04 11:08:00 EST
[1] https://github.com/rpm-software-management/dnf/pull/411
Comment 6 Michal Domonkos 2016-01-04 11:20:21 EST
Also, please note that using --refresh (instead of a running the clean command separately) is the recommended method in your use case:

# dnf --disablerepo=* --repofrompath f23,http://download.fedoraproject.org/pub/fedora/linux/releases/23/Everything/x86_64/os/ --enablerepo f23 --repofrompath f23-u,http://ftp.linux.cz/pub/linux/fedora/linux/updates/23/x86_64/ --enablerepo f23-u --repofrompath f23-ut,http://ftp.linux.cz/pub/linux/fedora/linux/updates/testing/23/x86_64/ --enablerepo f23-ut --disableexcludes all --refresh update kernel
Comment 7 Tomas Tomecek 2016-01-05 03:11:10 EST
Michal, thanks for the explanation.

It's up to you if you want to change the behavior, I'm glad for the answer.

(Maybe an option to `clean` could change the behavior, e.g. `--enabled`, `--all`, `--disabled`)
Comment 8 Michal Domonkos 2016-01-07 12:41:19 EST

*** This bug has been marked as a duplicate of bug 1278225 ***
Comment 9 Michal Domonkos 2016-01-11 13:29:06 EST
We decided to make the clean command purge everything there is, regardless of the enabled repos.

https://github.com/rpm-software-management/dnf/pull/411
Comment 10 Pavel Raiskup 2017-02-17 07:48:07 EST
Being late to the party:
> We decided to make the clean command purge everything there is, regardless
> of the enabled repos.

This was the easiest, but IMO not good decision. Working with dnf repos is
terribly slow job ... and with Copr services, I now have several tens of
repos.  It will be much worse with modules, right?

Not being able to clean up *only one* repository makes my Fedora
maintainer's life really painful now (reported as bug 1423399).

FWIW, when I try to iterate some rpm repository with testing by --repofrompath,
I usually work-around the issue by changing repo url (by adding redundant
slashes like 'https://domain/os///').

Note You need to log in before you can comment on or make changes to this bug.