Bug 1322365

Summary: fedora-review takes ages to complete
Product: [Fedora] Fedora Reporter: Jaroslav Škarvada <jskarvad>
Component: fedora-reviewAssignee: Stanislav Ochotnicky <sochotni>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 23CC: leamas.alec, pingou, sochotni
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-01 20:37:36 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jaroslav Škarvada 2016-03-30 12:03:23 UTC
Description of problem:
It seems it repeatedly resolves some deps, why it is doing this remains mystery for me:
03-30 13:43 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve ghc-mtl
03-30 13:44 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve ghc-pipes
03-30 13:45 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve ghc-transformers
03-30 13:46 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 13:50 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 13:55 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve gmp
03-30 13:57 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc

Version-Release number of selected component (if applicable):
fedora-review-0.6.0-2.fc23.noarch

How reproducible:
Always

Steps to Reproduce:
1. fedora-review -b 1321963

Actual results:
It takes pretty long to complete, consuming most of the time in repeatedly calling:
dnf repoquery -q -C --requires --resolve glibc

Expected results:
Completion in few minutes in the worst case

Additional info:

Comment 1 Jaroslav Škarvada 2016-03-30 14:27:26 UTC
And kisslicer:
03-30 15:32 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve libX11
03-30 15:34 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 15:39 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 15:43 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 15:48 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 15:53 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 15:57 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 16:02 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 16:06 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 16:11 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve libgcc
03-30 16:11 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 16:16 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 16:20 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc
03-30 16:25 root         DEBUG    Running: dnf repoquery -q -C --requires --resolve glibc

It runs for hours without completion, this is really insane.

Comment 2 Jaroslav Škarvada 2016-03-30 14:30:44 UTC
It's not only glibc, it seems to re-run other queries as well, but the glibc is probably the most time expensive.

Comment 3 Alec Leamas 2016-04-01 20:37:36 UTC
Basically, this is a DNF issue, see bug  bug #1279538.

That said, thanks for reporting!

*** This bug has been marked as a duplicate of bug 1275275 ***

Comment 4 Jaroslav Škarvada 2016-04-03 23:05:25 UTC
(In reply to Alec Leamas from comment #3)
> Basically, this is a DNF issue, see bug  bug #1279538.
> 
> That said, thanks for reporting!
> 
> *** This bug has been marked as a duplicate of bug 1275275 ***

OK, bug 1279538 is about why --resolve is so slow, but why is there the same query in the debug log listed multiple times? E.g.:

Running: dnf repoquery -q -C --requires --resolve glibc
Running: dnf repoquery -q -C --requires --resolve gmp
Running: dnf repoquery -q -C --requires --resolve glibc
Running: dnf repoquery -q -C --requires --resolve glibc

Cannot it cache the results?

Comment 5 Alec Leamas 2016-04-04 15:17:10 UTC
There are patches in the upstream tree addressing this. It helps somewhat, but IMHO not enough to be meaningful.

You might want to try the upstream version [1] and/or use the '-x CheckOwnDirs' to work around the issue.

[1] https://fedorahosted.org/FedoraReview/wiki/UseDevelopmentVersion

Comment 6 Jaroslav Škarvada 2016-04-04 16:16:30 UTC
In this case, until dnf issue will be resolved, it will be huge speedup not to call it repeatedly, i.e. 10x glibc query, 5 minutes each, it is:

- 5 minutes total if cached
- 50 minutes total if not cached

Comment 7 Jaroslav Škarvada 2016-04-04 16:20:21 UTC
The "factor" is there always, so it should be always faster and it is not dependant on the bug 1279538. The bug only amplifies this bad practice.