Bug 1644241 - dnf enforces reinstalling custom changed packages
Summary: dnf enforces reinstalling custom changed packages
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: 29
Hardware: Unspecified
OS: Unspecified
medium
unspecified
Target Milestone: ---
Assignee: Jaroslav Mracek
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1667358 (view as bug list)
Depends On:
Blocks: 1779584
TreeView+ depends on / blocked
 
Reported: 2018-10-30 09:57 UTC by Ali Akcaagac
Modified: 2020-03-13 09:46 UTC (History)
13 users (show)

Fixed In Version: dnf-4.2.8-2.fc30 dnf-4.2.9-3.fc31 dnf-4.2.5-5.fc29
Clone Of:
Environment:
Last Closed: 2019-09-14 01:12:56 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
Tetcase: recompiled garcon from Xfce Git by using Fedora *.spec from Git (135.37 KB, application/x-rpm)
2018-11-02 10:19 UTC, Ali Akcaagac
no flags Details
dnf 3.5.1 seems to be working (72.61 KB, text/plain)
2018-11-02 18:42 UTC, Ali Akcaagac
no flags Details
dnf 4.0.4 doesn't (30.07 KB, text/plain)
2018-11-02 18:43 UTC, Ali Akcaagac
no flags Details

Description Ali Akcaagac 2018-10-30 09:57:06 UTC
I just learned about a new situation here.

In our workflow we have to adjust some Fedora packages for special demands and re-pack them accordingly using the same versioning as the packages found within Fedora.

We usually issue the following commands to update or re-install the packages:

dnf update *.rpm
and
dnf reinstall *.rpm

to simulate as if everything is how it should be. The packages will then be installed from filesystem correctly.

After issuing an

dnf update

afterwards DNF now wants to re-install all previously customized packages with the packages from repository. This wasn't the case before and we now learned about the changes.

We strongly don't want this feature and it interferes with out workflow and automation of backgroundtasks that we run here for years.

I also can't find a command line argument to prevent enforced re-installation with repository packages.

Please provide an option to stop enforced re-installations.

Comment 1 Ali Akcaagac 2018-10-30 09:59:05 UTC
Please no replies about "change your versions" or "change packaging names or something similar". We can't simply change the workflow here because the backend is much bigger than a simple "change few lines" in an already set environment.

Comment 2 Jaroslav Mracek 2018-11-02 08:41:33 UTC
I am really sorry, but can you explain your problem with examples (former, present, and expected behavior)? I am not sure what is exactly issue here. Thanks a lot.

Comment 3 Ali Akcaagac 2018-11-02 10:18:14 UTC
Ok let me try an explaination:

As I report a couple of bugs for DNF in the past, it might have been known that we have some sort of automate process for package management and package delivery, that has been written around yum (way back when we first migrated to Fedora with Fedora 16).

Over the years the system has matured, documented and does some other tasks as well. Like building packages or altering packages - just how the customer may expect to have it.

One of the parts of this "system" is building packages and applying our patches to it *or* (optional) building packages from upstream, that hasn't made it yet into Fedora. For some packages we can provide version numbering changes, for others we can't.

One of these packages we can't change (or don't want to change) the versioning is XFCE4

Some of the XFCE4 packages we build are simple XFCE Git checkouts that gets build using the Fedora *.spec files as found on Fedora Git.

e.g. Thunar:
https://src.fedoraproject.org/rpms/Thunar/blob/master/f/Thunar.spec

We do this because we *use*, *test*, and contribute to XFCE.

We use this way of compiling XFCE for years now - regardless whether there may be optional or different approaches in doing so. This way has proven to work for us - for years.

The new packages that get build and packaged then end up having the same name, version, release, arch, .rpm as the ones found within Fedora repositories.

In the past we were able to simply issue:

dnf update *.rpm

from within the path that the new generated subset of XFCE rpm packages were and dnf updated them correctly. It still does this as of today with the latest DNF version available.

After wards we can issue a normal

dnf update

to accordingly update the rest of other packages and updates that got pushed by Fedora (the normal package management process).

In the past this only updated what came from Fedora, RPMFusion, Copr, and so on.

Since shortly - and we just discovered this - dnf also tries to reinstall all our own changed XFCE packages.

That means: All the packages that we updated from command line and from files stored within a directory are now subject to enforced reinstallation.

This wasn't the case before. Neither by using yum, neither by using dnf.

Since this situation hasn't been protocolled by dnf's changelog and documentation - as can be found here:

https://dnf.readthedocs.io/en/latest/release_notes.html

We therefore believe, that this may be an regression that slipped in silently due to some modular changes (or other changes) that got addressed by the developers.

Right now there is no way to tell dnf to not reinstall our customized and altered packages from the packages that are offered by Fedoras own repositories.

I commented about this on another bug here:

https://bugzilla.redhat.com/show_bug.cgi?id=1616167#c21

And I was told that if I change a package and name it the same, that the content of course is differently - even if the package remains with the same naming. Yes technically this is correct.

But the question is:
Who decides which package is the right one ? You can't simply assume that only the package provided by Fedora is the right one. The other package with the same naming can be right as well.

As said: This wasn't the case before and showed up until recently. So something must have changed.

Please also note, that we can not simply go and change a whole infrastructure that we build up around the needs of third (customer) by doing things differently or simply adding different version tags or a different naming scheme. This would cause a lot of change requests, change management, documentation of the changes and so on. We already went through this once and won't want to do this once again.

Please find attached a RPM file to illustrate this:

garcon-<someversion>.rpm

This is just a compile of garcon Git found on Xfce Git using Fedoras *.spec files as illustrated above with the Thunar example.

You may not be able to update this but reinstall it to your system (in case you don't use Xfce). After another normal dnf update, dnf tries reinstalling same package from Fedora repositories - thus overwriting the compiled and customized Git version. - note, this is just an example.

If questions remain, then feel free replying. I try to cover my answers as good I can.

Comment 4 Ali Akcaagac 2018-11-02 10:19:04 UTC
Created attachment 1500470 [details]
Tetcase: recompiled garcon from Xfce Git by using Fedora *.spec from Git

Comment 5 Ali Akcaagac 2018-11-02 10:26:44 UTC
> dnf update *.rpm
> 
> from within the path that the new generated subset of XFCE rpm packages were
> and dnf updated them correctly. It still does this as of today with the latest
> DNF version available.

In case of same versions we of course also need to do:

dnf reinstall *.rpm

from within the path, so we can replace already installed packages with newer versions :)

This also used to work and still works as of today. No complaints.

Only the regular "dnf update" afterwards want's to replace all customized packages with those found within Fedora... And this is the complaint. It wasn't the case before. So something that remains undocumented within DNF - has changed.

Enforcing reinstallation of packages and replace installed packages with packages found within Fedora repos (because of e.g. changed checksums) is a *no go*. You can't assume that this is the right approach of dealing with installed software and thus enforce overriding what an admin may have thought was right.

Packages with same name and altered content can be intended and wanted... and not a mistake that needs to be addressed by dnf by enforcing reinstallation because of some checksum mismatch (or whatever causes this).

Comment 6 Jaroslav Mracek 2018-11-02 12:52:47 UTC
Thanks. The problem is reproducible with dnf-4.0.4-2 and libdnf-0.22.0-6. It is probably result of changed transaction type from un-targeted to targeted for "dnf upgrade" command without argument.

Comment 7 Jaroslav Mracek 2018-11-02 12:54:29 UTC
Please can you check if the problem is reproducible with dnf-2.7.5 or lower with ``dnf upgrade "*"`` command.

Comment 8 Ali Akcaagac 2018-11-02 18:41:45 UTC
(In reply to Jaroslav Mracek from comment #7)
> Please can you check if the problem is reproducible with dnf-2.7.5 or lower
> with ``dnf upgrade "*"`` command.

No I don't think this is going back that far. iirc 2.7.x worked.

I just installed an early beta of Fedora 29 on VirtualBox and reproduced everything.

Fedora-Xfce-Live-x86_64-29_Beta-1.5.iso

This image contained DNF 3.5.1 and this seems to be working correctly. So the regression must have slipped in somwehre between there.

Please find attached two log files that shows the processes step by step.

Comment 9 Ali Akcaagac 2018-11-02 18:42:18 UTC
Created attachment 1500665 [details]
dnf 3.5.1 seems to be working

Comment 10 Ali Akcaagac 2018-11-02 18:43:12 UTC
Created attachment 1500667 [details]
dnf 4.0.4 doesn't

404 is a strange version number :)

Comment 12 Marek Blaha 2018-11-06 12:45:23 UTC
I've filed an issue on libsolv: https://github.com/openSUSE/libsolv/issues/287

Comment 13 Marek Blaha 2018-11-06 16:43:43 UTC
Meanwhile, I can offer you some workarounds for the issue:

1. you can use repository excludes in fedora/updates repositories for packages you are altering (see excludepkgs in https://dnf.readthedocs.io/en/latest/conf_ref.html#options-for-both-main-and-repo )

2. or you can move all your altered packages into your own repository and give that repository higher priority than fedora repo has (see priority in https://dnf.readthedocs.io/en/latest/conf_ref.html#repo-options )

Is any of these acceptable for you?

Comment 14 Ali Akcaagac 2018-11-06 17:13:53 UTC
(In reply to Marek Blaha from comment #13)
> Meanwhile, I can offer you some workarounds for the issue:

[...]

> 2. or you can move all your altered packages into your own repository [...]

Thanks for the feedback. We temporarely switched to an own repository. Still we hope, the issue will be addressed and fixed.

Comment 15 Ali Akcaagac 2018-11-18 10:09:04 UTC
Any progress with this ?

Regarding this Mail on the Fedora Devel List

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/OYAFP5MOLLRXPZUKE4EGIHA72LJFW3Q7/#2MSUYZ454T7QPZZC2E2NTQGV6T5KWIRW

If this is showing up in Fedora 28, we can throw our entire Infrastructure to /dev/null. Because the same situation will show up on Fedora 28 as well and cause a different behaviour in program functionality.

It would therefore be urgently required to solve this regression before Fedora 28 EOL and before Fedora 29 EOL.

Comment 16 Ali Akcaagac 2018-11-18 10:11:52 UTC
Also the comment made by igorgnatenko, that this rebase causes absolutely no risk, is not true. It entirely changes the behaviour of how DNF operates on Fedora 28.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/5ZA6HR57JDPZVOIB4OUOEO6ANNHZSYYO/

This should be worth yet another Fesco ticket.

Comment 17 Michael Schröder 2019-01-09 11:02:05 UTC
Note that this is *not* a regression in libsolv, libsolv always behaved the same way. This bug is caused by a change in dnf, as somewhat hinted at in comment #6. Don't blame libsolv for this.

Comment 18 Marek Blaha 2019-01-21 13:50:49 UTC
*** Bug 1667358 has been marked as a duplicate of this bug. ***

Comment 19 Vasiliy Glazov 2019-01-21 13:55:38 UTC
I am also need to solve this bug.
Can you add dnf command line parameter or dnf.conf entry to disable this behavior?

Comment 20 Sami Farin 2019-03-15 10:53:18 UTC
Also dnf 4.2.1 does this.
Is someone working on this?

Comment 21 Sami Farin 2019-03-15 11:01:40 UTC
as an example, I downloaded libjpeg-turbo*.src.rpm and then rpmbuild --rebuild, and installed it.
then after "dnf upgrade --debugsolver":
# head -n 1 ./debugdata/rpms/solver.result
change libjpeg-turbo-2.0.2-1.fc29.x86_64@@System libjpeg-turbo-2.0.2-1.fc29.x86_64@updates

Comment 22 Daniel 2019-05-27 06:32:17 UTC
I upgraded to Fedora 30 yesterday and confirmed that this bug still exists on Fedora 30.

Comment 23 Jaroslav Mracek 2019-07-30 16:44:31 UTC
I opened a new issue on libsolv - https://github.com/openSUSE/libsolv/issues/342.

Comment 24 Michael Schröder 2019-07-31 10:03:44 UTC
Opening new issues doesn't help, everything was already said in https://github.com/openSUSE/libsolv/issues/287. Somebody needs to fix the dnf code.

Comment 25 Jaroslav Mracek 2019-09-03 12:21:40 UTC
Thanks to Michael Schroeder and discussion in issues we have a solution: https://github.com/rpm-software-management/dnf/pull/1474.

Comment 26 Jaroslav Mracek 2019-09-10 11:46:03 UTC
The problem will be fixed in Fedora 30.

Comment 27 Fedora Update System 2019-09-10 12:19:45 UTC
FEDORA-2019-d79d0c470c has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-d79d0c470c

Comment 28 Fedora Update System 2019-09-10 12:19:49 UTC
FEDORA-2019-431726b949 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-431726b949

Comment 29 Fedora Update System 2019-09-10 14:32:29 UTC
dnf-4.2.9-3.fc31, libdnf-0.35.3-3.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-d79d0c470c

Comment 30 Fedora Update System 2019-09-11 02:58:55 UTC
dnf-4.2.8-2.fc30, libdnf-0.35.2-2.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-431726b949

Comment 31 Fedora Update System 2019-09-11 18:24:53 UTC
FEDORA-2019-d79d0c470c has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2019-d79d0c470c

Comment 32 Fedora Update System 2019-09-11 18:26:27 UTC
FEDORA-2019-431726b949 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2019-431726b949

Comment 33 Fedora Update System 2019-09-11 18:28:58 UTC
FEDORA-2019-d4dd43f749 has been submitted as an update to Fedora 29. https://bodhi.fedoraproject.org/updates/FEDORA-2019-d4dd43f749

Comment 34 Fedora Update System 2019-09-12 14:44:34 UTC
dnf-4.2.9-3.fc31, libdnf-0.35.3-4.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-d79d0c470c

Comment 35 Fedora Update System 2019-09-12 20:59:40 UTC
dnf-4.2.8-2.fc30, libdnf-0.35.2-3.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-431726b949

Comment 36 Fedora Update System 2019-09-12 22:39:21 UTC
dnf-4.2.5-5.fc29, libdnf-0.31.0-7.fc29 has been pushed to the Fedora 29 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2019-d4dd43f749

Comment 37 Fedora Update System 2019-09-14 01:12:56 UTC
dnf-4.2.8-2.fc30, libdnf-0.35.2-3.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 38 Fedora Update System 2019-09-14 16:34:29 UTC
dnf-4.2.9-3.fc31, libdnf-0.35.3-4.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 39 Fedora Update System 2019-09-27 01:44:59 UTC
dnf-4.2.5-5.fc29, libdnf-0.31.0-7.fc29 has been pushed to the Fedora 29 stable repository. If problems still persist, please make note of it in this bug report.


Note You need to log in before you can comment on or make changes to this bug.