Bug 1261034 - dnf doesn't handle Obsoletes (for split packages)
dnf doesn't handle Obsoletes (for split packages)
Status: ASSIGNED
Product: Fedora
Classification: Fedora
Component: libsolv (Show other bugs)
rawhide
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Igor Gnatenko
Fedora Extras Quality Assurance
: Documentation, FutureFeature, Reopened, Triaged
: 1264937 (view as bug list)
Depends On:
Blocks: 1222097 1250939 1260394
  Show dependency treegraph
 
Reported: 2015-09-08 09:04 EDT by Rex Dieter
Modified: 2017-12-07 19:05 EST (History)
25 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-07-19 13:49:23 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rex Dieter 2015-09-08 09:04:35 EDT
Following:

https://fedoraproject.org/wiki/Upgrade_paths_%E2%80%94_renaming_or_splitting_packages#.28n:m.29_Many_to_many_replacement

I implemented in plasma-workspace.spec:

%package -n sddm-breeze
Summary:    SDDM breeze theme
# upgrade path, when sddm-breeze was split out
Obsoletes: plasma-workspace < 5.3.2-8
Requires:  plasma-workspace = %{version}-%{release}
...

But it would appear dnf is not processing the Obsoletes and pulling sddm-breeze into the update transaction.

$ rpm -q dnf plasma-workspace sddm-breeze
dnf-1.1.1-1.fc23.noarch
plasma-workspace-5.3.2-2.fc23.x86_64
package sddm-breeze is not installed

$ sudo dnf update 
...

$ rpm -q plasma-workspace sddm-breeze
plasma-workspace-5.4.0-4.fc23.x86_64
package sddm-breeze is not installed

We've reports of f22 dnf behaving the same way, see bug #1260394 (the badness caused by this bug)
Comment 1 Kevin Kofler 2015-09-08 10:16:25 EDT
Looks like what's missing in particular is the hack yum had that would prefer an Obsoletes to a normal update, which is what allows splitting with versioned Obsoletes. DNF apparently figures that the updated package is no longer Obsoleted and so thinks it can just use that and ignore the Obsoletes.
Comment 2 Rex Dieter 2015-09-08 10:23:10 EDT
I wouldn't call yum's behavior a hack, rather, documented behavior for how Obsoletes should be handled.  :)
Comment 3 Fedora Blocker Bugs Application 2015-09-08 11:02:18 EDT
Proposed as a Blocker for 23-final by Fedora user rdieter using the blocker tracking app because:

 dnf not handling Obsoletes properly breaks any package that expects upgrade paths to work as documented in:
https://fedoraproject.org/wiki/Upgrade_paths_%E2%80%94_renaming_or_splitting_packages#.28n:m.29_Many_to_many_replacement

kde-sig already hit one such case in f22 -testing due to this dnf behavior, bug #1260394 , f22->f23 upgraders could experience the same (unless this is fixed)
Comment 4 Adam Williamson 2015-09-10 15:28:37 EDT
Discussed at 2015-09-10 blocker review meeting: https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2015-09-10/f23-blocker-review.2015-09-10-16.00.log.txt . We decided that it was better to take #1260394 (the specific known practical *consequence* of this problem) as a blocker, rather than the underlying DNF issue. However, we acknowledge there may be other problems caused by the same DNF behaviour, and we are monitoring this bug closely for the future.
Comment 5 Honza Silhan 2015-09-16 05:24:57 EDT
We will take a look whether the packages is wrongly splitted or dnf also selects Obsoletes badly in general.

Shouldn't be in the splitted package `Requires:  <new-package> = %{version}-%{release}`?
Comment 6 Rex Dieter 2015-09-16 09:19:03 EDT
Sometimes yes, sometimes not.  You cannot assume that.
Comment 7 Rex Dieter 2015-09-16 09:41:41 EDT
To be clear, I'm guessing you mispoke saying

"the splitted package `Requires:  <new-package> = %{version}-%{release}`?

to me, you're asking if the new (split) package should Requires: itself.  I assumed you meant, 

should the original (non-split) package Requires: new-package
to which I'm answering "no".  
(if the old package is just going to have a hard dependency on the new one, what's the point of splitting at all?)
Comment 8 Kevin Kofler 2015-09-16 20:03:30 EDT
The idea is to reliably upgrade one package to multiple ones without requiring them to depend on each other. How this worked in yum is that:
* if one or more Obsoletes matched, they would always be preferred to the normal upgrade,
* if multiple Obsoletes matches, yum would always pick ALL Obsoleting packages,
* so if you want the normal update to also remain installed, you'd use a versioned self-Obsoletes in addition to the Obsoletes in the other packages. (An alternative solution when splitting a foo-split out of foo is to have foo-split Obsolete the old foo EVR and Require the new foo EVR. Of course, that works if and only if foo-split is supposed to Require foo. Otherwise, the self-Obsoletes is the way.)

That complex behavior was specifically tuned to allow reliable and deterministic package splits, and as Rex Dieter pointed out, their use is also codified by the packaging guidelines.

The hard question is of course how that behavior can be emulated with a SAT solver such as libsolv.
Comment 9 Kevin Kofler 2015-09-16 20:11:09 EDT
So, to give some examples (replacing the EVR with a single integer for simplicity):

Original:
foo-1

Scenario A (foo Requires foo-split):
foo-2 ← Requires: foo-split
foo-split-2 ← may or may not Require foo, does not matter

Scenario B (foo-split Requires foo):
foo-2
foo-split-2 ← Obsoletes: foo < 2, Requires: foo >= 2

Scenario C (neither Requires the other):
foo-2 ← Obsoletes: foo < 2
foo-split-2 ← Obsoletes: foo < 2

All 3 scenarios should really work (i.e., users who had foo-1 installed should get both foo-2 and foo-split-2 when upgrading). It looks like DNF only correctly supports scenario A. Scenario B works in yum because it always prefers Obsoletes to a normal update (higher EVR of the same package). Scenario C works in yum because it always uses all Obsoletes.
Comment 10 Honza Silhan 2015-09-30 04:47:15 EDT
*** Bug 1264937 has been marked as a duplicate of this bug. ***
Comment 11 Michal Luscon 2015-10-02 12:18:13 EDT
Hi Kevin,

did you test all of scenarios, since they seems to be working for me. The only trick was to add Obsoletes into both foo-2 and foo-split-2.
Comment 12 Kevin Kofler 2015-10-02 18:01:46 EDT
I haven't tested anything myself, but bug #1260394 is clear evidence that scenario B is not working in practice. It shouldn't be necessary to add Obsoletes to both packages in that scenario, because Obsoletes should be preferred to a normal upgrade. If adding Obsoletes to both (as in scenario C) works reliably, that'd be a workaround we can use.
Comment 13 Michal Luscon 2015-11-24 09:33:55 EST
Concerning to what dnf should or should not do, is there any official guideline for package splitting process? Wiki page linked above seems to be focused on font packages and I haven't been able to find any better source of information.
Comment 14 Rex Dieter 2015-11-24 09:57:00 EST
While the referenced wiki specifically mentions fonts a lot, that's just one particular class of problems solved by the guidelines there.

If you want a short version, see comment #8 , and specific examples in comment #9

Doing anything else means you're knowingly changing from historical (yum-based) Obsoletes behavior, and we have this bug is a testament to that.

Now, if you think changing behavior is an superior option, I'm open to that idea, provided the new behavior can handle all the required cases outlined in the wiki:
* many to one replacement
* many to many replacement
Comment 15 Fedora Admin XMLRPC Client 2016-07-08 05:34:12 EDT
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.
Comment 16 Fedora End Of Life 2016-07-19 13:49:23 EDT
Fedora 22 changed to end-of-life (EOL) status on 2016-07-19. Fedora 22 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.
Comment 17 Kamil Páral 2016-07-20 04:46:09 EDT
It seems this is still relevant.
Comment 18 Igor Gnatenko 2016-07-20 05:06:03 EDT
(In reply to Kevin Kofler from comment #9)
> So, to give some examples (replacing the EVR with a single integer for
> simplicity):
> 
> Original:
> foo-1
> 
> Scenario A (foo Requires foo-split):
> foo-2 ← Requires: foo-split
> foo-split-2 ← may or may not Require foo, does not matter
This is just works:
repo system 0 testtags <inline>
#>=Pkg: foo 1 1 noarch
repo available 0 testtags <inline>
#>=Pkg: foo 2 1 noarch
#>=Req: foo-split
#>=Pkg: foo-split 2 1 noarch

system x86_64 rpm system

poolflags implicitobsoleteusescolors
solverflags allowvendorchange keepexplicitobsoletes bestobeypolicy keeporphans yumobsoletes
job update all packages
result transaction,problems <inline>
#>upgrade foo-1-1.noarch@system foo-2-1.noarch@available
#>install foo-split-2-1.noarch@available
Transaction summary:

1 upgraded packages:
  - foo-1-1.noarch -> foo-2-1.noarch

1 installed packages:
  - foo-split-2-1.noarch


> 
> Scenario B (foo-split Requires foo):
> foo-2
> foo-split-2 ← Obsoletes: foo < 2, Requires: foo >= 2
Filed bug at: https://github.com/openSUSE/libsolv/issues/146
> 
> Scenario C (neither Requires the other):
> foo-2 ← Obsoletes: foo < 2
> foo-split-2 ← Obsoletes: foo < 2
This is just works:
repo system 0 testtags <inline>
#>=Pkg: foo 1 1 noarch
repo available 0 testtags <inline>
#>=Pkg: foo 2 1 noarch
#>=Obs: foo < 2
#>=Pkg: foo-split 2 1 noarch
#>=Obs: foo < 2

system x86_64 rpm system

poolflags implicitobsoleteusescolors
solverflags allowvendorchange keepexplicitobsoletes bestobeypolicy keeporphans yumobsoletes
job update all packages
result transaction,problems <inline>
#>upgrade foo-1-1.noarch@system foo-2-1.noarch@available
#>install foo-split-2-1.noarch@available
Transaction summary:

1 upgraded packages:
  - foo-1-1.noarch -> foo-2-1.noarch

1 installed packages:
  - foo-split-2-1.noarch
> 
> All 3 scenarios should really work (i.e., users who had foo-1 installed
> should get both foo-2 and foo-split-2 when upgrading). It looks like DNF
> only correctly supports scenario A. Scenario B works in yum because it
> always prefers Obsoletes to a normal update (higher EVR of the same
> package). Scenario C works in yum because it always uses all Obsoletes.
Comment 19 Jan Kurik 2016-07-26 00:20:08 EDT
This bug appears to have been reported against 'rawhide' during the Fedora 25 development cycle.
Changing version to '25'.
Comment 20 Fedora End Of Life 2017-02-28 04:48:54 EST
This bug appears to have been reported against 'rawhide' during the Fedora 26 development cycle.
Changing version to '26'.
Comment 21 Rex Dieter 2017-11-09 15:52:41 EST
marking FutureFeature to avoid autoclose and more EOL messages)
Comment 22 Tom "spot" Callaway 2017-11-09 16:10:00 EST
Practical example:

Right now, I have texlive-luatex, texlive-luatex-bin, and texlive-luatex-doc. There is no good reason for this split, and I want texlive-luatex to eat texlive-luatex-bin and texlive-luatex-doc. This is what the packaging looks like with that change:

%package -n texlive-luatex
Provides: texlive-luatex-bin = %{epoch}:%{source_date}-%{release}
Provides: tex-luatex-bin = %{epoch}:%{source_date}-%{release}
Obsoletes: texlive-luatex-bin < 20170520
Provides: texlive-luatex-doc = %{epoch}:%{source_date}-%{release}
Obsoletes: texlive-luatex-doc < 20170520

dnf tries to install this new texlive-luatex, but it doesn't obsolete the -bin or -doc that are present, I get a screen full of errors like this:

file /usr/bin/luajittex from install of texlive-luatex-7:20170520-6.fc27.x86_64 conflicts with file from package texlive-luatex-bin-6:svn41091-36.20160520.fc27.5.x86_64

Removing the versioning from the Obsoletes does not change this at all.
Comment 23 Igor Gnatenko 2017-11-09 18:10:38 EST
Tom, can you run dnf with --debugsolver and attach debugdata directory?
Comment 24 Tom "spot" Callaway 2017-11-10 10:45:41 EST
https://spot.fedorapeople.org/texlive-error-debugdata.tar.xz
Comment 25 Igor Gnatenko 2017-11-10 10:59:52 EST
(In reply to Tom "spot" Callaway from comment #24)
> https://spot.fedorapeople.org/texlive-error-debugdata.tar.xz

Thanks, it seems that you have bug there. Once I did
=Pkg: texlive-luatex 7:20170520 6.fc27 x86_64
...
+Obs:
-texlive-luatex-bin < 20170520
-texlive-luatex-doc < 20170520
+texlive-luatex-bin
+texlive-luatex-doc
-Obs:

I've got:
+erase texlive-luatex-bin-6:svn41091-36.20160520.fc27.5.x86_64@@System texlive-luatex-7:20170520-6.fc27.x86_64@spot-texlive

which means libsolv replaced texlive-luatex-bin with texlive-luatex..

Your versioned obsoletes are wrong because you miss Epoch. they should be Obsoletes: texlive-luatex-bin < 7:20170520 or so.

Also once I added [forcebest], I've got:
+erase texlive-kpathsea-bin-6:svn40473-36.20160520.fc27.5.x86_64@@System texlive-kpathsea-7:20170520-6.fc27.x86_64@spot-texlive
+erase texlive-kpathsea-doc-6:svn41139-36.fc27.5.noarch@@System texlive-kpathsea-7:20170520-6.fc27.x86_64@spot-texlive
+erase texlive-pdftex-doc-6:svn41149-36.fc27.5.noarch@@System

You most likely miss some more Obsoletes.

Note You need to log in before you can comment on or make changes to this bug.