Bug 1539803 - dnf repoquery --requires --resolve is very slow
Summary: dnf repoquery --requires --resolve is very slow
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf   
(Show other bugs)
Version: 27
Hardware: x86_64
OS: Linux
Target Milestone: ---
Assignee: Marek Blaha
QA Contact: Fedora Extras Quality Assurance
Keywords: Triaged
Depends On:
TreeView+ depends on / blocked
Reported: 2018-01-29 16:08 UTC by abenkovskii
Modified: 2018-06-28 08:22 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2018-06-28 08:22:09 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
partial edited log of my bash session (1.50 KB, text/plain)
2018-01-29 16:08 UTC, abenkovskii
no flags Details

Description abenkovskii 2018-01-29 16:08:03 UTC
Created attachment 1387880 [details]
partial edited log of my bash session

Description of problem:
dnf repoquery --requires --resolve takes an unexpectedly long time to execute for some packages.

Version-Release number of selected component (if applicable):
$ dnf --version
  Installed: dnf-0:2.7.5-2.fc27.noarch at Mon 29 Jan 2018 08:13:24 AM GMT
  Built    : Fedora Project at Wed 29 Nov 2017 09:48:50 AM GMT

  Installed: rpm-0:4.14.0-2.fc27.x86_64 at Mon 29 Jan 2018 08:11:23 AM GMT
  Built    : Fedora Project at Thu 12 Oct 2017 02:18:15 PM GMT

How reproducible:
Depending on the package you want to resolve the requirements for the execution time varies: it takes more than 25 minutes for libreoffice-writer, about 15 minutes for firefox but for bash it only takes 15 seconds.

Steps to Reproduce:
1. run: time dnf repoqery libreoffice-writer --requires --resolve
2. compare to: time dnf repoqery libreoffice-writer --requires --tree

Actual results:
The --resolve variation took more than 27 minutes while the --tree variation took about 15 seconds.

$ time dnf repoquery libreoffice-writer --requires --tree > ~/tree.txt
Last metadata expiration check: 0:05:53 ago on Mon 29 Jan 2018 05:35:01 PM MSK.

real	0m14.956s
user	0m14.357s
sys	0m0.435s
$ time dnf repoquery libreoffice-writer --requires --resolve > ~/pkg.txt
Last metadata expiration check: 0:07:02 ago on Mon 29 Jan 2018 05:35:01 PM MSK.

real	27m2.014s
user	26m44.140s
sys	0m10.115s

Expected results:
I would expect --resolve to take at most as much as --tree because --tree reports all the information --resolve does and then some.

Additional info:
It looks like this issue was fixed in 2016 https://bugzilla.redhat.com/show_bug.cgi?id=1279538 but now it seams to have returned.

Comment 1 Santiago León 2018-03-27 01:46:53 UTC
This happened to me too with the same dnf version. Also, using the --installed flag seems to solve it.

Inside the RepoQueryCommand class, the problem seems to be here:

if self.opts.list == "installed":
    query = self.filter_repo_arch(self.opts, self.base.sack.query())
    query = self.filter_repo_arch(self.opts, self.base.sack.query().available())

I don't know why using available() makes things so slow.

Comment 3 XU Guang-zhao 2018-04-05 08:44:04 UTC
Possible duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1279538

BTW, the issue is marked as solved but not, might be a regression

Comment 4 Marek Blaha 2018-04-05 10:55:05 UTC
PR mentioned in comment 2 is not merged into upstream yet. If you want to test it, you can update dnf from my testing copr repository https://copr.fedorainfracloud.org/coprs/mblaha/dnf/

# dnf copr enable mblaha/dnf
# dnf update dnf
# time dnf repoquery -Cq --requires --resolve wine-core-0:3.4-1.fc27.x86_64
real    0m0.670s
user    0m0.598s
sys     0m0.057s

Regarding the solved bug 1279538, this bug was filed against dnf version 1. But solution is very similar - not using glob when not necessary.

Comment 5 Santiago León 2018-04-05 14:51:38 UTC
I can confirm that your PR solves the issue.

Comment 6 Jaroslav Mracek 2018-06-28 08:22:09 UTC
The issue is solved by dnf-3.0.1-1 that was released into rawhide.

Note You need to log in before you can comment on or make changes to this bug.