Hide Forgot
I have noticed that during `dnf update` the cleanup operation writes tens of megabytes of data (sometimes up to a hundred and more) while `rpm -U` doesn't do that and uses the disk a lot less. I wonder if it's possible to optimize this process so that cleanup worked more efficiently and maybe processed multiple packages in one step (i.e. a single writeout to /var/lib/rpm).
We are using rpm transaction API and just receive callbacks. DNF only deletes packages in the package cache after transaction. No additional cache updates happens. RPM devs, is there any difference WRT IO operations when using RPM API and RPM from CLI for upgrade?
Rpm cli uses the same exact API to do transactions. Only it doesn't do anything at all in the callbacks, whereas dnf does ... I don't know what, but yum used to update the history db and whatnot in there. And then there are plugins and the like to consider. But guessing is waste of time, you need to investigate. I'd suggest you start with strace to see what is being written and where.
There are people who know DNF source inside our and you tell me to strace? WTF? Seriously? I update a 30KB package using DNF and my disk writes are over 5MB(!). That's just insane and I'm not gonna dig into megabytes of strace data when someone perfectly knows what DNF does during update. Let's cehck today's updates: # dnf update --downloadonly Last metadata expiration check: 1:13:12 ago on Tue Feb 7 16:08:35 2017. Dependencies resolved. ============================================================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================================================== Upgrading: bash-completion noarch 1:2.5-1.fc25 updates 268 k gnutls i686 3.5.8-2.fc25 updates 712 k gnutls x86_64 3.5.8-2.fc25 updates 737 k gnutls-c++ i686 3.5.8-2.fc25 updates 37 k gnutls-c++ x86_64 3.5.8-2.fc25 updates 36 k gnutls-dane i686 3.5.8-2.fc25 updates 38 k gnutls-dane x86_64 3.5.8-2.fc25 updates 37 k gnutls-devel i686 3.5.8-2.fc25 updates 1.6 M gnutls-devel x86_64 3.5.8-2.fc25 updates 1.6 M Transaction Summary ============================================================================================================================================================================================== Upgrade 9 Packages Total download size: 5.0 M Is this ok [y/N]: y Downloading Packages: (1/9): bash-completion-2.5-1.fc25.noarch.rpm 431 kB/s | 268 kB 00:00 (2/9): gnutls-dane-3.5.8-2.fc25.x86_64.rpm 1.0 MB/s | 37 kB 00:00 (3/9): gnutls-c++-3.5.8-2.fc25.x86_64.rpm 750 kB/s | 36 kB 00:00 (4/9): gnutls-3.5.8-2.fc25.i686.rpm 3.6 MB/s | 712 kB 00:00 (5/9): gnutls-3.5.8-2.fc25.x86_64.rpm 725 kB/s | 737 kB 00:01 (6/9): gnutls-dane-3.5.8-2.fc25.i686.rpm 692 kB/s | 38 kB 00:00 (7/9): gnutls-devel-3.5.8-2.fc25.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 (8/9): gnutls-c++-3.5.8-2.fc25.i686.rpm 411 kB/s | 37 kB 00:00 (9/9): gnutls-devel-3.5.8-2.fc25.i686.rpm 3.1 MB/s | 1.6 MB 00:00 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 1.4 MB/s | 5.0 MB 00:03 Complete! The downloaded packages were saved in cache until the next successful transaction. You can remove cached packages by executing 'dnf clean packages'. Let's note the disk writes at this point after running `sync`: 689,627,136 bytes (I'm writing this in a text editor, so at the moment there are no applications which are writing anything to the disk). Now let's run dnf for real: # dnf update and Ctrl+Z it right before the cleanup stage begins Disk writes: 712,798,208 bytes After cleanup finishes and dnf exists, disk writes: 736,256,000 bytes IOW DNF cleanup has written over 24 megabytes of data for 9 petty packages.
Artem, I was merely pointing out that *somebody* needs to actually investigate what is being written and where. Typically that'd be the developers, assuming they can reproduce.
DNF team will investigate.
This message is a reminder that Fedora 25 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 25. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '25'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 25 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
DNF install/update/delete all write too much data to the HDD. It would be great if DNF operations could be optimized.
We will definitely improve a performance of dnf. But the reason why RPM transaction requires less resources than DNF is quite obvious. We use RPM to perform a transaction therefore this part is same. Additionally we: 1. Read metadata and creates *.solv files (Required by solver) even for installed packages 2. We have our own database about transactions (history commands) 3. We store information about installed groups, expired repository, downloaded rpms or so on. We provide more functionality than RPM, therefore we need more resources. I think that it is not necessary to keep the bug open, because this is anyway long term plan of DNF team. Thanks for your report.
This message is a reminder that Fedora 27 is nearing its end of life. On 2018-Nov-30 Fedora will stop maintaining and issuing updates for Fedora 27. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '27'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 27 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
(In reply to Ben Cotton from comment #9) Having plans and having an open bug report are close things but this bug report could still serve as a friendly reminder. If you insist on this bug report being closed I won't object.