Bug 1500361 - DNF in Fedora 26 is terrible slow
Summary: DNF in Fedora 26 is terrible slow
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 26
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Jaroslav Mracek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-10-10 12:57 UTC by Harald Reindl
Modified: 2017-12-22 10:35 UTC (History)
9 users (show)

Fixed In Version: dnf-2.7.5-1.fc26 dnf-2.7.5-1.fc27
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-10-21 22:21:51 UTC


Attachments (Terms of Use)

Description Harald Reindl 2017-10-10 12:57:33 UTC
fine, now at least it seems to solve deps properly and give out something useful in case of broken deps but the performacne is unacceptable while one of the arguments replacing yum was performance

real    1m49,496s
user    1m47,129s
sys     0m1,211s

for "dnf -y install git" while a) caches are frseh and b) even the packages are already downloaded - are you kidding me? yum needed a fractal of second for the same depsolving

do you realize that this is another running in circles:

* dnf replaced yum (not only but also) because of yum was slow
  whoever pretened it was slow....
* dnf as it was was fast did not work relieable and was completly
  unuseable for many tasks
* now as it beomces feature complete und more bugfree it becomes slower
* but it becomes not only "slower" - it becomes laughable slow
* it becomes slower than yum was all the years

[root@testserver:~]$ rpm -qa | grep dnf
dnf-conf-2.7.3-1.fc26.noarch
python3-dnf-plugin-show-leaves-2.1.5-1.fc26.noarch
python3-dnf-plugins-core-2.1.5-1.fc26.noarch
python3-dnf-plugin-leaves-2.1.5-1.fc26.noarch
dnf-yum-2.7.3-1.fc26.noarch
libdnf-0.10.1-1.fc26.x86_64
dnf-2.7.3-1.fc26.noarch
python3-dnf-2.7.3-1.fc26.noarch
python2-dnf-2.7.3-1.fc26.noarch

[root@testserver:~]$ time dnf -y install git
Letzte Prüfung auf abgelaufene Metadaten: vor 0:05:01 am Di 10 Okt 2017 14:45:51 CEST.
Abhängigkeiten sind aufgelöst.
===================================================================================================================================================================================================================
 Paket                                                Arch                                           Version                                                 Paketquelle                                     Größe
===================================================================================================================================================================================================================
Installieren:
 git                                                  x86_64                                         2.13.6-1.fc26                                           updates                                         1.1 M
Installiere Abhängigkeiten:
 git-core                                             x86_64                                         2.13.6-1.fc26                                           updates                                         4.0 M
 git-core-doc                                         x86_64                                         2.13.6-1.fc26                                           updates                                         2.2 M
 libsecret                                            x86_64                                         0.18.5-3.fc26                                           fedora                                          155 k
 perl-Git                                             noarch                                         2.13.6-1.fc26                                           updates                                          68 k

Transaktionsübersicht
===================================================================================================================================================================================================================
Installieren  5 Pakete

Gesamtgröße: 7.6 M
Installationsgröße: 36 M
Pakete werden heruntergeladen:
[SKIPPED] git-2.13.6-1.fc26.x86_64.rpm: Already downloaded
[SKIPPED] git-core-2.13.6-1.fc26.x86_64.rpm: Already downloaded
[SKIPPED] git-core-doc-2.13.6-1.fc26.x86_64.rpm: Already downloaded
[SKIPPED] perl-Git-2.13.6-1.fc26.noarch.rpm: Already downloaded
[SKIPPED] libsecret-0.18.5-3.fc26.x86_64.rpm: Already downloaded
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Gesamt                                                                                                                                                                             758 MB/s | 7.6 MB     00:00
Transaktionsüberprüfung wird ausgeführt
Transaktionsprüfung war erfolgreich.
Transaktion wird getestet
Transaktionstest war erfolgreich.
Transaktion wird ausgeführt
  Vorbereitung läuft    :                                                                                                                                                                                      1/1
  Installieren          : git-core-2.13.6-1.fc26.x86_64                                                                                                                                                        1/5
  Installieren          : git-core-doc-2.13.6-1.fc26.x86_64                                                                                                                                                    2/5
  Installieren          : libsecret-0.18.5-3.fc26.x86_64                                                                                                                                                       3/5
  Ausgeführtes Scriptlet: libsecret-0.18.5-3.fc26.x86_64                                                                                                                                                       3/5
  Installieren          : perl-Git-2.13.6-1.fc26.noarch                                                                                                                                                        4/5
  Installieren          : git-2.13.6-1.fc26.x86_64                                                                                                                                                             5/5
  Überprüfung läuft     : git-2.13.6-1.fc26.x86_64                                                                                                                                                             1/5
  Überprüfung läuft     : git-core-2.13.6-1.fc26.x86_64                                                                                                                                                        2/5
  Überprüfung läuft     : git-core-doc-2.13.6-1.fc26.x86_64                                                                                                                                                    3/5
  Überprüfung läuft     : perl-Git-2.13.6-1.fc26.noarch                                                                                                                                                        4/5
  Überprüfung läuft     : libsecret-0.18.5-3.fc26.x86_64                                                                                                                                                       5/5

Installiert:
  git.x86_64 2.13.6-1.fc26             git-core.x86_64 2.13.6-1.fc26             git-core-doc.x86_64 2.13.6-1.fc26             libsecret.x86_64 0.18.5-3.fc26             perl-Git.noarch 2.13.6-1.fc26

Fertig.

real    1m49,496s
user    1m47,129s
sys     0m1,211s

Comment 1 Jaroslav Mracek 2017-10-10 16:30:03 UTC
Please can you test it with command like:
Of course please remove git before test.

``time dnf -y install git --assumeno --disableplugin=* --disableexcludes``

Comment 2 Harald Reindl 2017-10-10 16:34:41 UTC
[root@testserver:~]$ time dnf -y install git --assumeno --disableplugin=* --disableexcludes=*
rhsoft-repo for fedora                                                                                                                                                             7.2 MB/s | 150 kB     00:00
rhsoft-repo with generic rpms                                                                                                                                                      302 kB/s | 2.5 kB     00:00
Letzte Prüfung auf abgelaufene Metadaten: vor 0:00:00 am Di 10 Okt 2017 18:32:19 CEST.
Abhängigkeiten sind aufgelöst.
===================================================================================================================================================================================================================
 Paket                                                Arch                                           Version                                                 Paketquelle                                     Größe
===================================================================================================================================================================================================================
Installieren:
 git                                                  x86_64                                         2.13.6-1.fc26                                           updates                                         1.1 M
Installiere Abhängigkeiten:
 git-core                                             x86_64                                         2.13.6-1.fc26                                           updates                                         4.0 M
 git-core-doc                                         x86_64                                         2.13.6-1.fc26                                           updates                                         2.2 M
 libsecret                                            x86_64                                         0.18.5-3.fc26                                           fedora                                          155 k
 perl-Git                                             noarch                                         2.13.6-1.fc26                                           updates                                          68 k

Transaktionsübersicht
===================================================================================================================================================================================================================
Installieren  5 Pakete

Gesamtgröße: 7.6 M
Installationsgröße: 36 M
Vorgang abgebrochen.

real    1m52,235s
user    1m49,122s
sys     0m1,061s

Comment 3 Scott Dowdle 2017-10-10 20:38:48 UTC
9 out of 10 times, dnf seems very slow for me too.  For some reason it seems to be rebuilding the rpm database frequently that seems to be a root cause of the slowness.

Comment 4 Harald Reindl 2017-10-10 20:45:34 UTC
rebuilding the rpm database? how has dnf any business to ever do this? anyways, a rpm --rebuilddb don't take that long and in my case it sits around like a dead dog after the message of last metadata check

Comment 5 Jaroslav Mracek 2017-10-11 10:36:42 UTC
Sorry, the command to test should be:
``time dnf -y install git --assumeno --disableplugin=* --disableexcludes=all``

Comment 6 Jaroslav Mracek 2017-10-11 10:38:32 UTC
If this is a problem with excludes, then problem was already fixed by https://github.com/rpm-software-management/dnf/pull/948.

Comment 7 Harald Reindl 2017-10-11 10:46:34 UTC
it's excludes that it hangs after "Last metadata expiration check: 0:01:02 ago on Wed Oct 11 12:41:56 2017" for no good reason

packages where i maintain override and i686 in genereal are excluded since yum/dnf tend to trying install i686 packages when deps can not be satisfied and on my machines there is no single non-x86_64 piece installed for years

that excludes below are not new, they exist for at least 8 years only switching "excludepkgs" and "exclude" in the hope that you developers now are done with rename forwards and backwards where you never should have renamed anything to start with

excludepkgs=*.i686,aespipe,apr,apr-util,apr-util-devel,apr-util-nss,apr-util-openssl,cantata,dbmail*,dovecot*,ffmpeg*,fsync-tester,GeoIP*,gmime,gmime-devel,hostapd,httpd,httpd-devel,httpd-tools,hylafax,iat,iaxmodem,lame*,libevent*,libmpdclient*,libnss-mysql,libzdb,mariadb*,mod_cgi,mod_dav,mod_info,mod_proxy,mod_proxy_http,mod_security,mod_ssl,mod_status,mpc,mpd,mpdscribble,mysql-client,netatalk*,open-vm-tools,openvpn,perl-File-Scan-ClamAV,perl-Net-DRI,php,php-bcmath,php-bz2,php-calendar,php-cli,php-common,php-ctype,php-curl,php-dba,php-devel,php-dom,php-embedded,php-enchant,php-exif,php-feedcreator,php-fileinfo,php-gd,php-gettext,php-hash,php-iconv,php-imap,php-interbase,php-intl,php-json,php-ldap,php-mbstring,php-mcrypt,php-mssql,phpMyAdmin,php-mysql,php-mysqli,php-mysqlnd,php-odbc,php-opcache,php-openssl,php-pcntl,php-pdo,php-pear,php-pecl-apc,php-pecl-apcu,php-pecl-bbcode,php-pecl-crack,php-pecl-geoip,php-pecl-id3,php-pecl-imagick,php-pecl-mailparse,php-pecl-mcrypt,php-pecl-mysqlnd_qc,php-pecl-ssh2,php-pecl-uploadprogress,php-pecl-vld,php-pecl-xdebug,php-pgsql,php-phar,php-posix,php-process,php-pspell,php-readline,php-recode,php-session,php-simplexml,php-snmp,php-soap,php-sockets,php-suhosin,php-tidy,php-tokenizer,php-xml,php-xmlreader,php-xmlrpc,php-xmlwriter,php-zip,php-zlib,postfix*,pure-ftpd,roundcubemail,spamassassin-bogofilter,spamassassin-clamav,spamassassin-mxpf,trafficserver*,up-imapproxy,vnstat,webalizer-xtended,woff-code

Comment 8 Jaroslav Mracek 2017-10-11 10:52:12 UTC

*** This bug has been marked as a duplicate of bug 1499534 ***

Comment 9 Harald Reindl 2017-10-11 17:32:51 UTC
that it seems to operate on every available package in every available version instead only the highest ones until downgrade/dtstro-sync comes in play also don't make it faster

i have never seen such outputs like below in yum and they are still *not* helpful as yum dependecy problem display was - yum was really good software, dnf currently is not at all

(fügen Sie »--best --allowerasing« zur Befehlszeile hinzu, um die Aktualisierung zu erzwingen):
 poppler                                          x86_64                                    0.52.0-2.fc26                                                 fedora                                             829 k
 poppler                                          x86_64                                    0.52.0-5.fc26                                                 updates                                            829 k
 poppler                                          x86_64                                    0.52.0-7.fc26                                                 updates-testing                                    830 k
Skipping packages with broken dependencies:
 poppler-glib                                     x86_64                                    0.52.0-7.fc26                                                 updates-testing                                    145 k
 poppler-qt5                                      x86_64                                    0.52.0-2.fc26                                                 fedora                                             175 k
 poppler-qt5                                      x86_64                                    0.52.0-5.fc26                                                 updates                                            175 k
 poppler-qt5                                      x86_64                                    0.52.0-7.fc26                                                 updates-testing                                    174 k
 poppler-utils                                    x86_64                                    0.52.0-7.fc26                                                 updates-testing                                    190 k

Comment 10 Harald Reindl 2017-10-12 17:21:39 UTC
and why does that take ages and hangs around for at least 30 seconds here when with --cache-only the xecludes in the repos must not matter at all?

[root@srv-rhsoft:/downloads]$ dnf --cacheonly upgrade *.rpm
Letzte Prüfung auf abgelaufene Metadaten: vor 7:39:25 am Do 12 Okt 2017 11:40:05 CEST.

Comment 11 Jaroslav Mracek 2017-10-12 21:41:44 UTC
Don't worry, patches will be released very soon.

Comment 12 Harald Reindl 2017-10-16 09:23:53 UTC
it's still a bad joke

"Last metadata expiration check: 0:13:15 ago on Mon Oct 16 11:09:05 2017" and then waiting 

[root@rh:/downloads]$ time dnf --cacheonly upgrade *.rpm
Last metadata expiration check: 0:13:37 ago on Mon Oct 16 11:09:05 2017.
Package dnf of higher version already installed, cannot update it.
Package dnf-conf of higher version already installed, cannot update it.
Package dnf-yum not installed, cannot update it.
No match for argument: dnf-yum-2.7.4-1.fc26.noarch.rpm
Package python2-dnf of higher version already installed, cannot update it.
Package python3-dnf of higher version already installed, cannot update it.
Dependencies resolved.
Nothing to do.
Complete!

real    0m39.843s
user    0m39.388s
sys     0m0.163s

[root@rh:/downloads]$ rpm -qa | grep dnf | sort
dnf-2.7.4-1.fc26.noarch
dnf-conf-2.7.4-1.fc26.noarch
dnf-plugins-core-2.1.5-1.fc26.noarch
etckeeper-dnf-1.18.5-3.fc26.noarch
libdnf-0.11.1-1.fc26.x86_64
python2-dnf-2.7.4-1.fc26.noarch
python3-dnf-2.7.4-1.fc26.noarch
python3-dnf-plugin-leaves-2.1.5-1.fc26.noarch
python3-dnf-plugin-show-leaves-2.1.5-1.fc26.noarch
python3-dnf-plugins-core-2.1.5-1.fc26.noarch
python3-dnf-plugins-extras-common-2.0.3-1.fc26.noarch

Comment 13 Jaroslav Mracek 2017-10-16 16:05:04 UTC
I created two patches that should solve the issue completely:

https://github.com/rpm-software-management/libdnf/pull/346

https://github.com/rpm-software-management/dnf/pull/955

Comment 14 Fedora Update System 2017-10-18 19:07:27 UTC
dnf-2.7.5-1.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-d2e8fdf5d8

Comment 15 Fedora Update System 2017-10-18 19:07:45 UTC
dnf-2.7.5-1.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-ee5c678e8b

Comment 16 Fedora Update System 2017-10-19 15:23:17 UTC
dnf-2.7.5-1.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-ee5c678e8b

Comment 17 Harald Reindl 2017-10-19 15:25:32 UTC
dnf-2.7.5-1.fc26.noarch is way better, no karma because i refuse to use the bodhi-website just because Fedora is not capable to keep it's house clean (https://bugzilla.redhat.com/show_bug.cgi?id=1504089)

Comment 18 Fedora Update System 2017-10-19 19:52:40 UTC
dnf-2.7.5-1.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-d2e8fdf5d8

Comment 19 Fedora Update System 2017-10-21 22:21:51 UTC
dnf-2.7.5-1.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

Comment 20 Fedora Update System 2017-11-11 02:57:00 UTC
dnf-2.7.5-1.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.