Bug 1256313
| Summary: | DNF should download packages specified by URL into the cache directory | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [Fedora] Fedora | Reporter: | Ali Akcaagac <aliakc> | ||||
| Component: | dnf | Assignee: | rpm-software-management | ||||
| Status: | CLOSED ERRATA | QA Contact: | Fedora Extras Quality Assurance <extras-qa> | ||||
| Severity: | unspecified | Docs Contact: | |||||
| Priority: | low | ||||||
| Version: | 25 | CC: | jmracek, jzeleny, mluscon, mmraka, packaging-team-maint, tim.lauridsen, vmukhame | ||||
| Target Milestone: | --- | Keywords: | Reopened, Triaged | ||||
| Target Release: | --- | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | dnf-2.4.0-1.fc26 | Doc Type: | Bug Fix | ||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2017-05-05 13:34:35 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
use `dnf install ... --setopt=keepcache=1` and locate pkgs in "/var/cache/dnf/<repoid>-<hash>/packages/" dir. If you didn't want to install packages -> use download plugin. Do not use stuff from /tmp. I think you are ignoring or missing the point here. Have you looked at the attachment script ? dnf install http://koji.someurl.org/somepath/wine-version.rpm There is no such thing as repoid.hash and so on, because these packages are being downloaded directly into /tmp directory with these strange names and directly processed inside there. The script run's dnf install <url>. It downloads the file and installs it - as intended. No repository being used. Yum did the same but kept the correct names. ... and ... if you took the time reading my initial bugreport carefully (which you obviously haven), then you understand that I need to use this approach to deal with packages that haven't been pushed to any repository yet. e.g. build but not pushed... this is a different use case. It makes things hard for us, reporting bugs and issues and then see them being closed just before we can react with a suitable response - or - if the bugreport hasn't been read (or understood) with the necessary care. The answer is out of scope with the report. We can use wget to grab the packages and use dnf install afterwards but this IS not what I have reported here. We know howto do things differently. But this is not about doing things differently, its about an issue we discover by comparing the behavior of yum vs. dnf. Thus we react and this we report. If no further bugs, feedback or other things is wanted then please let me know. We are going to spread the word whover is interested. Hi Ali, Jan probably overlooked your script. Could you please explain to us why you need those tmp files to be properly named? And regarding the location, I would doubt that we will be able to come up with a better place for storing temporary objects than tmp itself.
> And regarding the location, I would doubt that we will be able to come up
> with a better place for storing temporary objects than tmp itself.
Well, /var/cache/dnf seems to be a perfect place for such packages ;).
(In reply to Michal Luscon from comment #4) > Could you please explain to us why you need those tmp files to be properly > named? This is easy to answer. We provide packages for offline installation and need them to be named correctly. The use case is as follows: - The Wine packages usually show up with a 1-2 week of delay in Fedora 22 updates-(testing) (compared to Fedora 23 or Rawhide as for this example). - We depend on Wine because we need to run some proprietary stuff on Fedora and thus need to test Wine before forwarding it to other offline installations. - When we download the Wine packages (around 100-120mb) then we usually know that these koji packages are the ones that get pushed to updates-(testing). So the packages are usually the same as the ones that appear later in the official repositories. - To avoid downloading these packages again, we usually move the files into --downloaddir=<somepath>. Once we run dnf update --downloaddir=<somepath>. - We have "keep_cache=1" already enabled by default in /etc/dnf/dnf.conf. - Therefore naming these files correctly would be a benefit. The changes inside the code should be quite trivial I assume. Rather than naming the files as tmp[0-9|a-z|A-Z]*.rpm, the file names could be retrived from command line (they are retrived anyways, otherwise a download process couldn't be initiated). (In reply to Michael Mráka from comment #5) > Well, /var/cache/dnf seems to be a perfect place for such packages ;). I agree. So "dnf clean all" could be run to remove these temporarily directories as well (as example). Even if you like to keep /tmp, then please put the files into a subdirectory inside /tmp and have the files named correctly. e.g. /tmp/dnf-<date><time>-tmp/wine.*.rpm or simply /tmp/dnf-tmp/... Wine was just one example. There are other packages as well, which might be affected similarily. We might end up in 50-100 tmp*.rpm files in the root of tmp in just a couple of minutest. Not knowing which file belongs to what and how. This also makes readability (in case others store other information in /tmp) quite impossible because they have to dive through all the tmp*.rpm files first. Right, I believe that the problem here is not where does DNF store its temporary files but that keepcache does not work with packages specified as URLs/paths... We should keep them in the cache as well... At least that's my opinion. Oh, it works with keepcache but weirdly. It keeps the files in /tmp. Now, it makes sense to me. Right, I don't think that this is intentional. OK, so we should create a directory for these packages in the cache directory and place the files there. We can name the directory "@commandline". This is the name which appears in the transaction summary if user installs packages specified by URL. The directory should probably has the same structure as the other directories. Sounds ok to me. But please save the files in real-name rather than tmp[0-9]*.rpm This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component. Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed. Reopened because missing information about actual state. I try to implement it in pull request: https://github.com/rpm-software-management/dnf/pull/778 . Hope that it helps dnf-plugins-core-2.0.0-1.fc26 libdnf-0.8.2-1.fc26 dnf-2.4.0-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-4e95959f0d dnf-2.4.0-1.fc26 dnf-plugins-core-2.0.0-1.fc26 dnf-plugins-extras-2.0.0-1.fc26 libdnf-0.8.2-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-4e95959f0d dnf-2.4.0-1.fc26, dnf-plugins-core-2.0.0-1.fc26, dnf-plugins-extras-2.0.0-1.fc26, libdnf-0.8.2-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-4e95959f0d dnf-2.4.0-1.fc26, dnf-plugins-core-2.0.0-1.fc26, dnf-plugins-extras-2.0.0-1.fc26, libdnf-0.8.2-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report. |
Created attachment 1066304 [details] downloader script I have a script, that downloads newest (but not yet released) wine from Fedora koji. I depend on this because wine is usually being pushed to testing with a delay or so. Anyways this is not just wine related. Assume running the attached script. This will download wine from the koji locations and install it. This simple script was written a few years back and used to use yum. So basicly this is a s/yum/dnf/g nobrainer. What I end is getting this: -bash-4.3$ ls -l /tmp/*.rpm -rw------- 1 root root 78796 Aug 24 12:44 /tmp/tmp0zWXUX.rpm -rw------- 1 root root 1737796 Aug 24 12:44 /tmp/tmp1mzm1T.rpm -rw------- 1 root root 142504 Aug 24 12:44 /tmp/tmp2eD9rK.rpm -rw------- 1 root root 114440 Aug 24 12:44 /tmp/tmp2VWI1f.rpm -rw------- 1 root root 52464 Aug 24 12:44 /tmp/tmp3UU378.rpm -rw------- 1 root root 67896 Aug 24 12:44 /tmp/tmp43s0B1.rpm -rw------- 1 root root 83180 Aug 24 12:44 /tmp/tmp6RJuLY.rpm -rw------- 1 root root 73340 Aug 24 12:43 /tmp/tmp9EBbaY.rpm -rw------- 1 root root 65412 Aug 24 12:44 /tmp/tmp9LKbJA.rpm -rw------- 1 root root 65388 Aug 24 12:44 /tmp/tmp9OZKlJ.rpm -rw------- 1 root root 100632 Aug 24 12:44 /tmp/tmpAYzh1k.rpm -rw------- 1 root root 64772 Aug 24 12:44 /tmp/tmpB2Lg4l.rpm -rw------- 1 root root 65628 Aug 24 12:44 /tmp/tmp_B6bo1.rpm -rw------- 1 root root 104860 Aug 24 12:43 /tmp/tmpb8b0OY.rpm -rw------- 1 root root 158272 Aug 24 12:44 /tmp/tmpdI5qYY.rpm -rw------- 1 root root 107660 Aug 24 12:44 /tmp/tmpHpNsTG.rpm -rw------- 1 root root 22168476 Aug 24 12:44 /tmp/tmpioO89J.rpm -rw------- 1 root root 71844 Aug 24 12:44 /tmp/tmpLeJiD8.rpm -rw------- 1 root root 84696 Aug 24 12:44 /tmp/tmplPHyVU.rpm -rw------- 1 root root 63968 Aug 24 12:44 /tmp/tmpM0rI_l.rpm -rw------- 1 root root 94412 Aug 24 12:44 /tmp/tmpn4w5bd.rpm -rw------- 1 root root 52932 Aug 24 12:43 /tmp/tmpQjJZNK.rpm -rw------- 1 root root 58504 Aug 24 12:43 /tmp/tmpqZJL9n.rpm -rw------- 1 root root 159396 Aug 24 12:44 /tmp/tmpsD8ygd.rpm -rw------- 1 root root 53128 Aug 24 12:44 /tmp/tmpSIwyom.rpm -rw------- 1 root root 53136060 Aug 24 12:45 /tmp/tmpvedFOi.rpm -rw------- 1 root root 31613004 Aug 24 12:45 /tmp/tmpxiISCM.rpm -bash-4.3$ Now running the same script with yum instead of dnf: -bash-4.3$ su -c "ls -l /var/tmp/yum-root-3eoD0D" Password: total 108200 -rw-r--r-- 1 root root 31613004 Aug 16 23:09 mingw32-wine-gecko-2.40-1.fc22.noarch.rpm -rw-r--r-- 1 root root 52932 Aug 15 01:27 wine-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 104860 Aug 15 01:27 wine-alsa-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 142504 Aug 15 02:00 wine-arial-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 58504 Aug 15 01:27 wine-capi-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 73340 Aug 15 01:28 wine-cms-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 107660 Aug 15 02:00 wine-common-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 22168476 Aug 15 01:28 wine-core-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 100632 Aug 15 02:00 wine-courier-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 158272 Aug 15 02:00 wine-desktop-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 83180 Aug 15 02:00 wine-filesystem-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 65628 Aug 15 02:00 wine-fixedsys-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 52464 Aug 15 02:00 wine-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 114440 Aug 15 01:27 wine-ldap-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 64772 Aug 15 02:00 wine-marlett-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 53136060 Jun 10 16:10 wine-mono-4.5.6-4.fc22.noarch.rpm -rw-r--r-- 1 root root 1737796 Aug 15 02:00 wine-ms-sans-serif-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 65412 Aug 15 01:27 wine-openal-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 63968 Aug 15 01:27 wine-opencl-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 94412 Aug 15 01:27 wine-pulseaudio-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 67896 Aug 15 02:00 wine-small-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 78796 Aug 15 02:00 wine-symbol-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 53128 Aug 15 02:00 wine-systemd-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 71844 Aug 15 02:00 wine-system-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 159396 Aug 15 02:00 wine-tahoma-fonts-1.7.49-2.fc22.noarch.rpm -rw-r--r-- 1 root root 84696 Aug 15 01:27 wine-twain-1.7.49-2.fc22.i686.rpm -rw-r--r-- 1 root root 65388 Aug 15 02:00 wine-wingdings-fonts-1.7.49-2.fc22.noarch.rpm -bash-4.3$ Now: 1) Dumping the downloaded files in /tmp (root of tmp) is not the right path because /tmp could be kept in ram (some ram file system). 2) I believe it should be no problem to name the files correctly even with DNF rather than naming them tmp<somestuff>.rpm in case one like to store the downloaded files to some other place for later usage. 3) Using a different path (and subdir) for storing such downloads would be much nicer. I like the way yum is doing it.