Created attachment 907506 [details] debugdata from debugsolver 1. bump version 2. add new subpackage 3. for this new subpkg add Obsoletes: %{name} <= %{version}-%{release} 4. build rpms 5. create repo 6. try to update Yum: ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: shiny-test x86_64 0.3-8.git411ac43.fc20 local 4.1 k replacing shiny.x86_64 0.3-1.gitdc53364.fc20 Transaction Summary ================================================================================ Install 1 Package Total download size: 4.1 k Is this ok [y/d/N]: Dnf: ================================================================================ Package Arch Version Repository Size ================================================================================ Upgrading: shiny x86_64 0.3-8.git411ac43.fc20 local 288 k Transaction Summary ================================================================================ Upgrade 1 Package Total size: 288 k Is this ok [y/N]:
also we can add obsoletes for main package. yum will update base and install subpkg dnf will update base
Thanks Igor, we'll take a look.
Hello, I debugged this in detail today. The behavior seen with DNF (and libsolv) is the correct one, shiny-test obsoletes "shiny <= 0.3-8", i.e. all such "shiny" packages with release-version less than or equal "0.3-8". But in this case, shiny-0.3-8.git411ac43.fc20.x86_64 is present in the 'local' repository. In the EVR algebra, it is technically correct to consider 0.3-8.git411ac43.fc20.x86_64 greater than 0.3-8 and so the solver decided shiny-test does not in fact obsolete the latest available 'shiny' package and correctly proceeds with an upgrade instead of obsoleting.
FWIW, the latest release of NetworkManager does the same packaging mistake: [akozumpl@localhost ~/Downloads]$ rpm -qp --obsoletes NetworkManager-wifi-0.9.9.95-1.git20140609.fc21.i686.rpm NetworkManager < 1:0.9.9.95-1
(In reply to Ales Kozumplik from comment #3) > Hello, I debugged this in detail today. The behavior seen with DNF (and > libsolv) is the correct one, shiny-test obsoletes "shiny <= 0.3-8", i.e. all > such "shiny" packages with release-version less than or equal "0.3-8". But > in this case, shiny-0.3-8.git411ac43.fc20.x86_64 is present in the 'local' > repository. In the EVR algebra, it is technically correct to consider > 0.3-8.git411ac43.fc20.x86_64 greater than 0.3-8 and so the solver decided > shiny-test does not in fact obsolete the latest available 'shiny' package > and correctly proceeds with an upgrade instead of obsoleting. I don't quite understand this explanation... What should the NetworkManager package do to ensure that the new sub-package gets correctly installed on upgrade?
(just re-opening to make sure that you see my question, I'd like to figure out how to make this work so I can rebuild NM if necessary to fix the issue) But a larger question too... if the obsoletes behavior of dnf is different than yum, should that be noted somewhere, and should the packaging guidelines be updated to account for it? If so, then it's going to be messy to make packages that work for both yum and dnf (eg, same specfile for F20 as F21).
Yum's behavior for Obsoletes was intentionally special-cased to support package splits, this was explicitly requested by packagers. The rules in yum were: 1. Obsoletes *takes precedence* over a regular new version! I.e., if you have foo-1, and if the repository contains: * foo-2 (no Obsoletes) * bar-0 which Obsoletes: foo < 2 then bar will be preferred over foo. This is a very important feature for package renames (like qt→qt3 and qt4→qt). If you do not want foo to get replaced by bar, you have to use self-Obsoletes and rely on rule 2 below, i.e.: * foo-2 "redundantly" Obsoletes: foo < 2 * bar-0 still Obsoletes: foo < 2 2. If multiple packages Obsolete the same existing package, ALL possible replacements will get dragged in (e.g., in the second example above, BOTH foo-2 and bar-0 will be picked to replace foo-1). This is the rule for package splits. Splitting packages is a real PITA without those rules/hacks. Almost all the possibly surprising behavior in yum was there for a reason!
(In reply to Ales Kozumplik from comment #4) > FWIW, the latest release of NetworkManager does the same packaging mistake: > > [akozumpl@localhost ~/Downloads]$ rpm -qp --obsoletes > NetworkManager-wifi-0.9.9.95-1.git20140609.fc21.i686.rpm > NetworkManager < 1:0.9.9.95-1 I believe NetworkManager's obsoletes are correct here and there is no packaging mistake. Previous NM build had EVR "1:0.9.9.1-6.git20140319.fc21" which is covered by the obsoletes above.
Yep - I've had a real big fun trying to discover what went wrong when I've lost Wifi connection. Learning few 'nmcli' command - and looking how to make 'Unmange' connection to be managed.... Such changes should not be scheduled around 'mass-rebuilds'....
My understanding here is the same as KK's: yum's behaviour in this case was explicitly designed to allow the introduction of subpackages in such a way that they will be transparently installed on update for existing users. I *believe* the relevant guideline in this situation is https://fedoraproject.org/wiki/Packaging:Conflicts#Splitting_Packages . To clarify for the dnf folks: what happened here is that part of NetworkManager was split into a new subpackage called NetworkManager-wifi . The intent is that systems which currently have a pre-split version of NetworkManager installed should, after an update to a post-split version, have both NetworkManager and NetworkManager-wifi installed. This is what the "Obsoletes" is intended to achieve. What we're considering is a situation like this: foo-1.0.rpm is installed foo-2.0.rpm and foo-bar-2.0.rpm are both available foo-bar-2.0.rpm and foo-2.0.rpm both have "Obsoletes: foo < 1.1" yum's behaviour in this case is to process the obsoletion ahead of the upgrade availability, and so install both packages that obsolete foo. dnf's behaviour is to process the update availability ahead of the obsoletion: it notes that foo-2.0 is an update to foo-1.0, adds it to the package set, *then* processes the obsoletion. Now the obsoletion doesn't take effect, because so far as dnf is concerned, we're onto foo-2.0 already. Neither behaviour seems absolutely and objectively incorrect; both are plausible interpretations of an ambiguous situation. *However*, yum's behaviour provides a mechanism for causing a new subpackage to be transparently installed on update (without a hard dependency on the main package, which would make the subpackage split pointless). dnf's does not. Fedora has a clear need for such a mechanism. If dnf is not going to switch to yum's behavior in this case, the dnf devs need to provide (or explain, if there is already one of which we're not aware) a mechanism for doing this kind of subpackage split, and we need to make sure that all F20+ cases use it, otherwise we may cause problems in supported cases when we switch to dnf with F22.
(In reply to Dan Williams from comment #5) > (In reply to Ales Kozumplik from comment #3) > > Hello, I debugged this in detail today. The behavior seen with DNF (and > > libsolv) is the correct one, shiny-test obsoletes "shiny <= 0.3-8", i.e. all > > such "shiny" packages with release-version less than or equal "0.3-8". But > > in this case, shiny-0.3-8.git411ac43.fc20.x86_64 is present in the 'local' > > repository. In the EVR algebra, it is technically correct to consider > > 0.3-8.git411ac43.fc20.x86_64 greater than 0.3-8 and so the solver decided > > shiny-test does not in fact obsolete the latest available 'shiny' package > > and correctly proceeds with an upgrade instead of obsoleting. > > I don't quite understand this explanation... > > What should the NetworkManager package do to ensure that the new sub-package > gets correctly installed on upgrade? Sorry, it looks like what I described above is relevant for the comment 0 only. NM might be a different situation after all. Looking into how this should work.
Michael, are there some capabilities in libsolv to facilitate package splitting, something similar to what Adam described in comment 10 perhaps?
We talked about package splitting at 2013's DevConf, AFAIK yum looks at the obsoletes (and not conflicts) to detect splits. It's not hard to teach libsolv about splits, we just need to define the exact semantics. For example I don't think libsolv should try to install *all* packages obsoleting an installed package, there needs to be a way to determine which packages belong to a split (e.g. they need to come from the same repository).
misc: the page title is somewhat misleading, the guideline in question does talk about Obsoletes. It's not laser-focused on this particular issue, but it does sort of cover it. I believe KK's correct to suggest that yum's behaviour in this case was actually explicitly designed to be used for this situation of subpackage splits, but I couldn't find a really great reference for that in our guidelines / docs anywhere - it was probably one of those things which only ever got written down on a mailing list.
(In reply to Michael Schröder from comment #13) > It's not hard to teach libsolv about splits, we just need to define the > exact semantics. For example I don't think libsolv should try to install > *all* packages obsoleting an installed package, there needs to be a way to > determine which packages belong to a split (e.g. they need to come from the > same repository). I saw SUSE using some special provides for that. It is mentioned as a legacy way in libsolv somewhere. Is that really wrong? What are the SUSE packagers typically doing now?
SUSE uses a very old feature called "split-provides": the split-off package provides a special "package-name:file-name" string, this gets auto-translated into a supplements by libsolv. It has some drawbacks, so please do not consider to use the same mechanism for Fedora. If you really want to switch to a new way package splits work, please switch to something easy and sane ;)
After much open discussion about supporting package splitting the team has decided to sustain the Yum semantics and Michael has implemented the libsolv support: https://github.com/openSUSE/libsolv/commit/99e2d6786a2f18126932170eccd5da139eb5f438
hawkey commit 8ceb623 enables the splitting semantics.
what's the approximate time frame for that getting downstream into f21 and rawhide?
It happened yesterday: http://koji.fedoraproject.org/koji/buildinfo?buildID=548316 http://koji.fedoraproject.org/koji/buildinfo?buildID=548349 According to [1] I am supposed to make f21 updates but bodhi won't let me ("libsolv-0.6.4-1.fc21 not tagged as an update candidate"). [1] https://fedoraproject.org/wiki/Updates_Policy
Bodhi hasn't kicked in for F21 yet, it doesn't come in right after branching. we should probably mention that. For now it works like rawhide, your build will go right into f21 in the next compose.
We definitely should mention that, otherwise people could question the relevance of the documentation and the processes. Should I open a ticket somewhere?
ales: I updated the wiki page last night. thanks for the catch. dgilmore tells me we used to branch and freeze simultaneously, hence there was no gap, but we changed that a while ago and didn't update that wiki page.
Thanks Adam.