Hi Adam, thanks for further investigation of dependency problem. DNF doesn't compute all possible solutions and find the smallest set, neither does yum. Yum has hack of rules [1]. We want it to solve it in more sophisticated way by weak dependencies [2] on package level i.e. add `Recommends: hawtjni-runtime` to jansi-native package.

[1] http://yum.baseurl.org/wiki/CompareProviders
[2] http://www.rpm.org/wiki/DynamicDependencies

yum has some heuristics, yeah, which you can argue is hacky, but it *works*. Weak dependencies are all very well, but they're not a part of Fedora 22, and even when they become a part of Fedora, not all packages are magically going to start using them overnight; if you're relying on every individual case of this biting someone who then files a bug report which then causes the packager to deal with it via weak dependencies, that's a ton of pain and inconvenience for users that will drag on for years. I don't think that's a very nice way to go about it.

I understand it's nice to try and keep the design 'clean', but I'm afraid so long as you want to push for dnf to be made the system package manager on an aggressive schedule, we might need to compromise about issues like this.

compare_providers is a thing that people worked hard on in response to a clear need, that has proven itself to provide a clear benefit; it's not a good idea to just throw it away because it's not clean enough for Perfect Spherical Package Manager Land.

This is not a bug in hatwjni or jansi-native.

hawtjni-runtime has ben split as subpackage of hawtjni specifically to optimize dependency chain (see bug #1166607).  Currently only hawtjni-runtime provides mvn(org.fusesource.hawtjni:hawtjni-runtime), so DNF should pick it as provider.

Weak dependencies are currently not allowed in Fedora, but they shouldn't be needed either. Packagers can't be expected to add Recommends each time provide is moved between packages, DNF should handle this case out of the box.

yep, I should've pointed that out more strongly too. Thanks for the addition.

I would like to disprove one unfortunate misconception. Weak dependencies actually are in Fedora as of F21. The fact they are not explicitly approved doesn't mean they are forbidden or not available. The only part that might be missing (I'd have to check but I think it's working as well) is support on the Fedora infrastructure (particularly in createrepo that is used to create official Fedora repos).

Now to the point of this bug. What is strange to me is that dnf uses hawtjni-1.10-2.fc21 even though its newer version (hawtjni-1.10-3.fc21) should be available. Considering the package was split in the -3 release, the bug might have something to do with this specific scenarion, as there were some problems with it in the past.

"The only part that might be missing (I'd have to check but I think it's working as well) is support on the Fedora infrastructure (particularly in createrepo that is used to create official Fedora repos)."

According to Miroslav, "I acknowledge that Recommends and Suggests can be problematic now because DNF and PackageKit is not ready yet." - https://lists.fedoraproject.org/pipermail/packaging/2015-January/010417.html .

Their use in F22 is under discussion by FPC (I thought it was already '23 at the earliest stuff' but apparently not), but has not yet been agreed upon. https://fedorahosted.org/fesco/ticket/1353 .

DNF is not ready in terms of repoquery command not supporting weak deps and a few other things like that. The basic functionality to include weak deps in transactions is there. Therefore weak deps are not technically blocked in Fedora.

(In reply to Jan Zeleny from comment #5)
> Now to the point of this bug. What is strange to me is that dnf uses
> hawtjni-1.10-2.fc21 even though its newer version (hawtjni-1.10-3.fc21)
> should be available. Considering the package was split in the -3 release,
> the bug might have something to do with this specific scenarion, as there
> were some problems with it in the past.

This is because dnf compares possible provides only based on package name (actually NVR), which is nonsense. It should look at epoch, version, release and only if those are the same, then look at name. Or it should use any other (ideally simple) but documented and usable algorithm. The current one is not usable at all unless we start prefixing packages names with numbers.

Similar discussion is at bz#1149160.

doesn't comparing E, V, R only make much sense for binary packages from the same source? Comparing E, V, R first would work for *this* case, but what about this one?

cat-1.0 from srpm 'cat' provides animal
dog-3.7 from srpm 'dog' provides animal

cat requires mouse
dog requires leash, dogpound, citypark, dogsitter, ball, poopbags...

why prefer dog to cat in that case, when their versions really aren't comparable at all?

(In reply to Adam Williamson (Red Hat) from comment #9)
> doesn't comparing E, V, R only make much sense for binary packages from the
> same source? Comparing E, V, R first would work for *this* case, but what
> about this one?
> 
> cat-1.0 from srpm 'cat' provides animal
> dog-3.7 from srpm 'dog' provides animal
> 
> cat requires mouse
> dog requires leash, dogpound, citypark, dogsitter, ball, poopbags...
> 
> why prefer dog to cat in that case, when their versions really aren't
> comparable at all?

Provides themselves can be versioned and provided version can be different from package version (it's actually quite common).

So adjusting your example accordingly:

cat-1.0 from srpm 'cat' provides 'animal=1.5'
dog-3.7 from srpm 'dog' provides 'animal=2.0-0.1.alpha1'

dog should be preferred because it is "better animal" than cat - provided version is greater.

They *can* be, but there is no grounds for assuming that they *are*. We don't have a guideline telling people to do things that way. In practice, most virtual provides I have seen are unversioned.

At the moment in rawhide we have 279618 provides, 209834 are versioned. That's over 75 %.  In the exact case of hawtjni provides are versioned too.

I don't think many are versioned in the way you suggest, though. They just use the version of the source package.

Some more notes:

This precise chain is hit when installed 'icedtea-web', which is a pretty common thing to install.

We hit another interesting case of bad ambiguous resolution during Anaconda DNF Test Day. There is a package xl2tpd which requires "kmod(l2tp_ppp.ko)". That is provided by all kernel -modules-extra packages. When you run 'yum install xl2tpd' without a kernel installed (e.g. from a mock chroot), yum selects kernel-modules-extra to provide it. When you run 'dnf install xl2tpd', DNF selects kernel-debug-modules-extra .

This causes an interesting effect at install time. If your package set happens to have xl2tpd in it, when anaconda first resolves the deps of the packages selected for install, 'kernel-debug-core' is pulled in. Later, anaconda runs a method to ensure the 'preferred' kernel is installed, dnfpayload.py's DNFPayload._select_kernel_package() . The logic of this method is (and has been for years, it's pretty much identical to the yumpayload one):

1. If 'kernel' is explicitly excluded figure the user knows what they're doing and don't install anything
2. If 'kernel-PAE' is available, install it
3. Otherwise if we're on ARM and 'kernel-(platform)' or 'kernel-lpae' is available, install it
4. Otherwise install 'kernel'

So the upshot is, that if your package set has xl2tpd in it, and the kernel currently in the repos is a 'production' build (so kernel-debug exists), you get two kernels installed: kernel-core (as a dep of 'kernel') and kernel-debug-core (as a dep of 'kernel-debug-modules', a dep of 'kernel-debug-modules-extra').

That was fun to work out!

Adam, I can see why you consider this a problem. On the other hand, strictly speaking, this is not a bug. You requested to install rhino. rhino was installed including all its dependencies. There is nothing more that DNF promises to user. Except that the provides are selected deterministically.
So, would it be acceptable for you if we close this bug and you open an RFE that describes the fitness function that you would like to be applied to select packages in case there are multiple choices?

Well, the thing I worry about is the commonly-encountered icedtea-web case. That's one that real people are going to hit quite a lot.

Sorry for my ignorance but I don't think that you've answered my question. Simple yes or not would be enough.
Still I think that you are interested in the general DNF's candidates selection algorithm. Not just this case of icedtea-web. Otherwise you wouldn't make so much effort in reporting the other cases and even the "tracker" bug. I believe that you don't want a rule in DNF that handles specifically the "mvn(org.fusesource.hawtjni:hawtjni-runtime)" provide.
Could you please answer my question whether it would be OK if we close this bug and you open the new general RFE?
I apologize if it's just a language barrier that I didn't understand your answer.

Sorry if it wasn't clear - my answer is usually I wouldn't mind (except I don't think there's a need for a downstream RFE, the upstream one is probably fine?), but this bug actually is quite likely to be commonly encountered - the original report shows the 'rhino' case, but you run into the same chain trying to install 'icedtea-web', which is something people commonly do as it's the standard Firefox Java plugin.

There's a couple of problem with closing bugs that are commonly encountered as something like 'oh, see this RFE':

1) it can look like we don't care
2) closed bugs do not show up in search, so you're likely to just get a duplicate filed soon if you close this one

so, that's why I might say it'd be better to leave it open. but ultimately, it's your call.

oh, and then there's the fact that this specific case might actually be resolvable with a pretty clear-cut rule: for multiple providers that come from the same .src.rpm, prefer the one with a higher EVR.

OK, you are probably right. Currently, this is not a bug but since we care about this issue, it may change in the future. OK, let's track it here.

Let's see how libsolv will be improved...

So this just landed upstream:

https://github.com/openSUSE/libsolv/commit/aa583e801055e3a17bca7f84feeef6898eecbda4

which I think may help. Could we backport it to Rawhide and F22's libsolv package so we can try it out?

As today is Alpha freeze, nominating as a freeze exception, it seems like we keep getting bugs along these lines (see also https://bugzilla.redhat.com/show_bug.cgi?id=1192182 ) and if we're going to try and make it better it would make sense to get it in early; I wouldn't want to land this last-minute before Alpha release, but right now is probably still early enough.

This reminds me
https://bugzilla.redhat.com/show_bug.cgi?id=1070109

Discussed at today's blocker review meeting [1].

This bug was rejected as Freeze Exception - We didn't get to this bug in time for it to be approved for Alpha FE status. Please submit before Beta freeze.

http://meetbot.fedoraproject.org/fedora-blocker-review/2015-03-02/

This bug appears to have been reported against 'rawhide' during the Fedora 22 development cycle.
Changing version to '22'.

More information and reason for this action is here:
https://fedoraproject.org/wiki/Fedora_Program_Management/HouseKeeping/Fedora22

The algorithm of libsolv has been improved in version 0.6.14 and the main use cases should have find smallest set of dependencies. Moreover there is a guideline [1] where you can prefer one package over another. I am closing this, feel free to reopen and report other issues you encounter.

[1] https://fedoraproject.org/w/index.php?title=PackagingDrafts/ProvidesPreferences