Bug 1397174 - Create internal repo from packages in cache
Summary: Create internal repo from packages in cache
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 24
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: ---
Assignee: rpm-software-management
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-21 19:15 UTC by Robin A. Meade
Modified: 2017-05-19 19:45 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-19 08:47:52 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)

Description Robin A. Meade 2016-11-21 19:15:02 UTC
A while back I enbabled keepcache=1 so that I'd be able to undo transactions.

Today I tried and it didn't work. I wanted to undo transaction 151, the last transaction. This is what I did:

$ sudo dnf history list | tac | tail
   144 | --refresh update -y      | 2016-11-10 08:00 | I, U           |   24   
   145 | --refresh update -y      | 2016-11-12 12:41 | I, U           |   52 EE
   146 | --refresh update -y      | 2016-11-14 08:03 | Update         |   14 EE
   147 | --refresh update -y      | 2016-11-15 10:47 | Update         |    1   
   148 | --refresh update -y      | 2016-11-17 08:40 | E, I, U        |   18   
   149 | install key-mon          | 2016-11-17 09:17 | Install        |    2   
   150 | --refresh update -y      | 2016-11-19 18:21 | E, I, U        |   34   
   151 | --refresh update -y      | 2016-11-21 08:03 | Update         |    3   

$ sudo dnf history undo 151
Last metadata expiration check: 1:03:05 ago on Mon Nov 21 08:02:31 2016.
Undoing transaction 151, from Mon Nov 21 08:03:03 2016
    Upgraded libsndfile-1.0.25-20.fc24.x86_64 @koji-override-0
    Upgrade             1.0.27-1.fc24.x86_64  @updates
    Upgraded wxBase3-3.0.2-26.fc24.x86_64     @updates
    Upgrade          3.0.2-29.fc24.x86_64     @updates
    Upgraded wxGTK3-3.0.2-26.fc24.x86_64      @updates
    Upgrade         3.0.2-29.fc24.x86_64      @updates
No package wxGTK3-0:3.0.2-26.fc24.x86_64 available.
Error: An operation cannot be undone

It says the package is not available, but it is present in the cache:

$ ls -l /var/cache/dnf/updates-c4f1c95f64c2b794/packages/wxGTK3*
-rw-r--r--. 1 root root 5566614 Oct 14 08:28 /var/cache/dnf/updates-c4f1c95f64c2b794/packages/wxGTK3-3.0.2-26.fc24.x86_64.rpm
-rw-r--r--. 1 root root 5566486 Nov 21 08:03 /var/cache/dnf/updates-c4f1c95f64c2b794/packages/wxGTK3-3.0.2-29.fc24.x86_64.rpm

Comment 1 Vít Ondruch 2016-11-23 10:56:59 UTC
I second that ...

Comment 2 Michal Luscon 2016-11-28 13:25:13 UTC
Hi,

DNF currently doesn't create local repos from downloaded packages in cache. You will have to specify those repos manually until we implement this RFE.

Comment 3 Vít Ondruch 2016-11-28 14:18:52 UTC
This makes the reverts and downgrades very hard and I'd say this is regression in comparison to YUM. In YUM, I could always use the "history undo" ...

Comment 4 Michal Luscon 2016-11-28 14:24:10 UTC
Ok, fair point -> raising prio

Comment 5 Jaroslav Mracek 2017-03-02 08:13:08 UTC
According to my knowledge there is local plugin that support requested functionality. Well I am not a user of that plugin and probably there is a behavior that is different from request presented here. Please can you provide a feedback?

Comment 6 Vít Ondruch 2017-03-02 11:02:52 UTC
"Automatically copy all downloaded packages to a repository on the local filesystem"

I don't want to copy the packages anywhere, they are stored in DNF cache directory, why would I want to copy them somewhere? I just want to be able to use them for "downgrade" it that case happens. Using Rawhide, this makes big difference.

Comment 7 Robin A. Meade 2017-03-03 21:13:33 UTC
I installed the 'local' plugin to give it a try.

Ran into bug 1209862. Workaround:
sudo mkdir -p /var/lib/dnf/plugins/local
sudo /usr/bin/createrepo_c /var/lib/dnf/plugins/local

I disabled keepcache in dnf.conf because otherwise the disk usage devoted to cached packages on my machine would double from approximately 6 GB to 12 GB over a 6 month period.

It will take some time to accumulate packages in my 'local' repo and be able to test whether it makes dnf's undo functionality work in the situation of old package missing from remote repo but present in 'local' repo. I'll report back when I've tested this.

Comment 8 Jaroslav Mracek 2017-03-06 12:58:12 UTC
The problem with missing directory in LOCAL plugin will be fixed with pull-request https://github.com/rpm-software-management/dnf-plugins-extras/pull/88 . 
Thanks all for participation and testing.

Comment 9 Jaroslav Mracek 2017-03-07 18:32:19 UTC
Hi everyone, I have create pull-request (https://github.com/rpm-software-management/dnf/pull/759) that introduce requested feature. Hope that it will work like you expect. Please if you will try it, I will be very happy. On request I can provide packages for your Fedora version.

Comment 10 Robin A. Meade 2017-03-08 04:14:34 UTC
I'd like to try it. May I have packages for Fedora 25?

Comment 11 Jaroslav Mracek 2017-03-08 07:52:58 UTC
You will need to enable two copr repositories, then install dnf with patch
How to install:
1. sudo dnf copr enable rpmsoftwaremanagement/dnf-nightly
2. sudo dnf copr enable jmracek/dnf-use-cached-rpm 
3. sudo dnf install dnf-2.1.0-1.git.26.e9603f8.fc25 --exclude dnf-2.1.0-1.git.26.e9603f8.fc25.src 

Hope that it will work

Comment 12 Vít Ondruch 2017-03-08 09:08:46 UTC
(In reply to Jaroslav Mracek from comment #11)
> 1. sudo dnf copr enable rpmsoftwaremanagement/dnf-nightly

Could you enable builds for Rawhide/F27 plzz

Comment 13 Jaroslav Mracek 2017-03-08 10:26:03 UTC
For rawhide:

1. sudo dnf copr enable jmracek/dnf-use-cached-rpm 
2. sudo dnf install dnf-2.1.0-1.git.26.e9603f8*

Hope that it will work

PS: The enabling of rawhide builds for dnf-nightly repo is on the way :-)

Comment 14 Jaroslav Mracek 2017-03-09 18:00:41 UTC
The proposed patch has downside therefore I made an investigation how yum handle it. I have found no support for requested feature in yum. I asked for help Valentina Mukhamedzhanova as a YUM expert, but she also didn't find any support for loading of cached rpms that are not anymore in repos. I would like to ask you if the information in Comment 3 is based on rawhide experience or RHEL?

Comment 15 Vít Ondruch 2017-03-10 10:56:16 UTC
(In reply to Jaroslav Mracek from comment #14)
> The proposed patch has downside therefore I made an investigation how yum
> handle it. I have found no support for requested feature in yum. I asked for
> help Valentina Mukhamedzhanova as a YUM expert, but she also didn't find any
> support for loading of cached rpms that are not anymore in repos. I would
> like to ask you if the information in Comment 3 is based on rawhide
> experience or RHEL?

Strange ... I'd swear that I used either "history undo" or "downgrade" on Fedora and is stopped to work with DNF. Unfortunately, I cannot reproduce the behavior now, since I don't use YUM and I don't have the YUM cache populated :/

Comment 16 Jaroslav Mracek 2017-03-13 08:14:59 UTC
I am sorry but I have to reconsider my decision how to provide requested functionality. From my point of view the best solution would be to use local plugin, because presented solution in pullreguest from Comment 9 is not a secure approach (The packages added into repo don't have checksum from repodata therefore we cannot verify them, but exclude them from verification is even worse solution). I know that local plugin do not do that either (it copies files and on top of them it creates new metadata), but this is plugin, and not default dnf behavior. If there will be a problem with local plugin I can make a promise that I will try to solve it. Just report it in this bugzilla.

Comment 17 Jaroslav Mracek 2017-05-19 08:47:52 UTC
It looks like that work around with local plugin works, therefore we can close this request.

Comment 18 Robin A. Meade 2017-05-19 19:45:25 UTC
Thank you. I confirm that the 'local' plugin is a solution to the problem of dnf's undo functionality not working when old package is missing from remote repo.

A preferable solution, IMO, is for the remote repo to retain old packages (at least 1 previous). I'll make a separate request for that.


Note You need to log in before you can comment on or make changes to this bug.