Bug 1539803

Summary: dnf repoquery --requires --resolve is very slow
Product: [Fedora] Fedora Reporter: abenkovskii
Component: dnfAssignee: Marek Blaha <mblaha>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 27CC: abenkovskii, dmach, jmracek, mblaha, mhatina, packaging-team-maint, rpm-software-management, santileortiz, taisfmq, vmukhame
Target Milestone: ---Keywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-06-28 08:22:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
partial edited log of my bash session none

Description abenkovskii 2018-01-29 16:08:03 UTC 
Created attachment 1387880 [details]
partial edited log of my bash session

Description of problem:
dnf repoquery --requires --resolve takes an unexpectedly long time to execute for some packages.


Version-Release number of selected component (if applicable):
$ dnf --version
2.7.5
  Installed: dnf-0:2.7.5-2.fc27.noarch at Mon 29 Jan 2018 08:13:24 AM GMT
  Built    : Fedora Project at Wed 29 Nov 2017 09:48:50 AM GMT

  Installed: rpm-0:4.14.0-2.fc27.x86_64 at Mon 29 Jan 2018 08:11:23 AM GMT
  Built    : Fedora Project at Thu 12 Oct 2017 02:18:15 PM GMT


How reproducible:
Depending on the package you want to resolve the requirements for the execution time varies: it takes more than 25 minutes for libreoffice-writer, about 15 minutes for firefox but for bash it only takes 15 seconds.


Steps to Reproduce:
1. run: time dnf repoqery libreoffice-writer --requires --resolve
2. compare to: time dnf repoqery libreoffice-writer --requires --tree


Actual results:
The --resolve variation took more than 27 minutes while the --tree variation took about 15 seconds.

$ time dnf repoquery libreoffice-writer --requires --tree > ~/tree.txt
Last metadata expiration check: 0:05:53 ago on Mon 29 Jan 2018 05:35:01 PM MSK.

real	0m14.956s
user	0m14.357s
sys	0m0.435s
$ time dnf repoquery libreoffice-writer --requires --resolve > ~/pkg.txt
Last metadata expiration check: 0:07:02 ago on Mon 29 Jan 2018 05:35:01 PM MSK.

real	27m2.014s
user	26m44.140s
sys	0m10.115s


Expected results:
I would expect --resolve to take at most as much as --tree because --tree reports all the information --resolve does and then some.


Additional info:
It looks like this issue was fixed in 2016 https://bugzilla.redhat.com/show_bug.cgi?id=1279538 but now it seams to have returned.
Comment 1 Santiago León 2018-03-27 01:46:53 UTC 
This happened to me too with the same dnf version. Also, using the --installed flag seems to solve it.

Inside the RepoQueryCommand class, the problem seems to be here:

if self.opts.list == "installed":
    query = self.filter_repo_arch(self.opts, self.base.sack.query())
else:
    query = self.filter_repo_arch(self.opts, self.base.sack.query().available())

I don't know why using available() makes things so slow.
Comment 2 Marek Blaha 2018-03-27 08:08:22 UTC 
PR https://github.com/rpm-software-management/dnf/pull/1045
Comment 3 XU Guang-zhao 2018-04-05 08:44:04 UTC 
Possible duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1279538

BTW, the issue is marked as solved but not, might be a regression
Comment 4 Marek Blaha 2018-04-05 10:55:05 UTC 
PR mentioned in comment 2 is not merged into upstream yet. If you want to test it, you can update dnf from my testing copr repository https://copr.fedorainfracloud.org/coprs/mblaha/dnf/

# dnf copr enable mblaha/dnf
# dnf update dnf
# time dnf repoquery -Cq --requires --resolve wine-core-0:3.4-1.fc27.x86_64
.
.
real    0m0.670s
user    0m0.598s
sys     0m0.057s

Regarding the solved bug 1279538, this bug was filed against dnf version 1. But solution is very similar - not using glob when not necessary.
Comment 5 Santiago León 2018-04-05 14:51:38 UTC 
I can confirm that your PR solves the issue.
Comment 6 Jaroslav Mracek 2018-06-28 08:22:09 UTC 
The issue is solved by dnf-3.0.1-1 that was released into rawhide.