Bug 1149160

Summary: DNF does not install the latest version of the requested virtual provide
Product: [Fedora] Fedora Reporter: Honza Horak <hhorak>
Component: hawkeyAssignee: Packaging Maintenance Team <packaging-team-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 21CC: akozumpl, hhorak, jsilhan, jzeleny, lkardos, packaging-team-maint, pnemade, redhat-bugzilla, rholy, vondruch
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: libsolv-0.6.10 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-21 09:02:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 782345, 1148538    
Attachments:
Description Flags
spec file for fakemariadb package
none
spec file for fakecommunitymysql package
none
spec file for fakerandommysql package
none
dnf --debugsolver install mysql none

Description Honza Horak 2014-10-03 11:21:04 UTC
Description of problem:
yum does quite a lot of magic about comparing provides [1], i.e. in cases when more packages satisfy a requirement. That complicated logic, which includes different behaviour if different packages are installed or not, led to many issues in the repository.

As a result, maintainers, or distro as a whole, are not able to deliver more packages that provide the same symbol, while one of the packages would be the main. One example for all -- mariadb is now default MySQL fork, while community-mysql (name chosen to satisfy yum's complicated rules [1]) provides the same versioned symbols (e.g. Provides: mysql = 5.6-1).

An example from real world, how it should work correctly in dnf:
If some package just requires `mysql` or user runs `dnf install mysql` (no package is called that way in repository now), mariadb should be installed.
If somebody requires `community-mysql`, this package should be installed, but all depended packages that require `mysql` are satisfied.

With yum this works.

With dnf this does not work any more.

My proposal is getting rid of complicated logic from yum and define a simple rule -- "if both provides satisfy all dependencies, the provide with greater NEVRA is chosen".

After some testing, dnf does not work this way. I'm not sure where this should be fixed, since hawkey compares versioned provides correctly:

  python> import hawkey
  python> sack = hawkey.Sack()
  python> n1 = hawkey.split_nevra("fakemariadb-1:1.2-1.x86_64")
  python> n2 = hawkey.split_nevra("fakecommunity-1.2-1.x86_64")
  python> n3 = hawkey.split_nevra("fakerandom-1.2-1.x86_64")
  python> n4 = hawkey.split_nevra("fakerandom-2:1.2-1.x86_64")
  python> n1.evr_cmp(n2, sack)
1L
  python> n1.evr_cmp(n3, sack)
1L
  python> n1.evr_cmp(n4, sack)
-1L

But when I create some fake packages, the package whose name (not NEVRA) is greater, always wins, no matter which version nor epoch it has. This is incorrect behaviour, obviously.

[1] http://yum.baseurl.org/wiki/CompareProviders

Version-Release number of selected component (if applicable):
$ rpm -q dnf libsolv hawkey
dnf-0.5.4-2.fc20.noarch
libsolv-0.6.4-0.git2a5c1c4.fc20.x86_64
hawkey-0.4.18-2.fc20.x86_64

How reproducible:
every-time

Steps to Reproduce:
1. build the attached fake RPM packages
2. $> cp ~/rpmbuild/RPMS/x86_64/fake*rpm /tmp/dnftestrepo
3. $> createrepo /tmp/dnftestrepo
4. create repofile for /tmp/dnftestrepo
5. yum install commonsymbol --disablerepo=* --enablerepo=testdnfrepo
6. dnf install commonsymbol --disablerepo=* --enablerepo=testdnfrepo

Actual results:
yum wants to install fakemariadb package
dnf wants to install fakecommunitymysql package

Expected results:
both, yum and dnf, want to install fakemariadb

Additional info:
will include some fake spec files in a minute

Comment 1 Honza Horak 2014-10-03 11:23:27 UTC
Created attachment 943704 [details]
spec file for fakemariadb package

Comment 2 Honza Horak 2014-10-03 11:23:51 UTC
Created attachment 943705 [details]
spec file for fakecommunitymysql package

Comment 3 Honza Horak 2014-10-03 11:24:26 UTC
Created attachment 943706 [details]
spec file for fakerandommysql package

Comment 4 Honza Horak 2014-10-03 11:56:37 UTC
Some older bug reports against yum -- similar unfixed BZ #782345 and fixed in RHEL-7 BZ#998707.

Comment 5 Honza Horak 2014-10-03 12:03:26 UTC
Another thought is related to RPM's new feature -- "soft dependencies" (BZ #1006954, FPC ticket [1]). As soon as we can define "Suggests" and similar in RPMs, dnf/libsolv or whoever could prefer this provide/package, which would makes sense. As an example:

package foo would include:
Suggests: community-mysql
Requires mysql

Then it feels correct to choose community-mysql packge over mariadb, even if mariadb would be preferred otherwise (if not using Suggests). It may add some complexity to the decision formula, but as far as it is documented properly and it supports the most common use cases, I do think we can implement such behaviour.

[1] https://fedorahosted.org/fesco/ticket/1203

Comment 6 Honza Horak 2014-10-03 12:08:37 UTC
hawkey might be better component, but still just guessing, please, re-assign if needed.

Comment 7 Honza Silhan 2014-10-03 16:31:12 UTC
(In reply to Honza Horak from comment #0)
> After some testing, dnf does not work this way. I'm not sure where this
> should be fixed, since hawkey compares versioned provides correctly:
> 
>   python> import hawkey
>   python> sack = hawkey.Sack()
>   python> n1 = hawkey.split_nevra("fakemariadb-1:1.2-1.x86_64")
>   python> n2 = hawkey.split_nevra("fakecommunity-1.2-1.x86_64")
>   python> n3 = hawkey.split_nevra("fakerandom-1.2-1.x86_64")
>   python> n4 = hawkey.split_nevra("fakerandom-2:1.2-1.x86_64")
>   python> n1.evr_cmp(n2, sack)
> 1L
>   python> n1.evr_cmp(n3, sack)
> 1L
>   python> n1.evr_cmp(n4, sack)
> -1L
> 
> But when I create some fake packages, the package whose name (not NEVRA) is
> greater, always wins, no matter which version nor epoch it has. This is
> incorrect behaviour, obviously.

This is correct, you are comparing two different packages - it's actually NEVR comparison. You can compare packages by epoch only.

> [1] http://yum.baseurl.org/wiki/CompareProviders

Huh, we don't want to specify own rules for provides - there will be more and more. We'd rather use rpm solution from comment 5. It is much cleaner than bumping epochs. You can try it, it's already implemented in libsolv and supported in hawkey.

This should be IMO closed. These components doesn't block bug 782345, we are waiting for ticket acceptance so packagers can use it.

Comment 8 Honza Horak 2014-10-06 09:55:41 UTC
(In reply to Jan Silhan from comment #7)
> (In reply to Honza Horak from comment #0)
> > After some testing, dnf does not work this way. I'm not sure where this
> > should be fixed, since hawkey compares versioned provides correctly:
> > 
> >   python> import hawkey
> >   python> sack = hawkey.Sack()
> >   python> n1 = hawkey.split_nevra("fakemariadb-1:1.2-1.x86_64")
> >   python> n2 = hawkey.split_nevra("fakecommunity-1.2-1.x86_64")
> >   python> n3 = hawkey.split_nevra("fakerandom-1.2-1.x86_64")
> >   python> n4 = hawkey.split_nevra("fakerandom-2:1.2-1.x86_64")
> >   python> n1.evr_cmp(n2, sack)
> > 1L
> >   python> n1.evr_cmp(n3, sack)
> > 1L
> >   python> n1.evr_cmp(n4, sack)
> > -1L
> > 
> > But when I create some fake packages, the package whose name (not NEVRA) is
> > greater, always wins, no matter which version nor epoch it has. This is
> > incorrect behaviour, obviously.
> 
> This is correct, you are comparing two different packages - it's actually
> NEVR comparison. You can compare packages by epoch only.

How can I compare packages by epoch only?

> > [1] http://yum.baseurl.org/wiki/CompareProviders
> 
> Huh, we don't want to specify own rules for provides - there will be more
> and more. We'd rather use rpm solution from comment 5. It is much cleaner
> than bumping epochs. You can try it, it's already implemented in libsolv and
> supported in hawkey.

Well, but that can work only for installing dependencies when they are defined in some other package. How can I solve the case when somebody runs:
`dnf install mysql` where the `mysql` is a common virtual provider for community-mysql and mariadb packages and I as a packager want to prioritize mariadb?

Do I have to create a mock package that defines such "Suggests" and which is required by both, community-mysql and mariadb?

> This should be IMO closed.

I'm not sure, having simple rules is nice but if they do not help in real scenarios, I don't think we can be satisfied. Thinking about Fedora.next changes, which aim to include different versions and implementations for similar packages, there might be more cases like mariadb vs. community-mysql.

Comment 9 Radek Holy 2014-10-06 10:48:17 UTC
(In reply to Honza Horak from comment #8)
> Well, but that can work only for installing dependencies when they are
> defined in some other package. How can I solve the case when somebody runs:
> `dnf install mysql` where the `mysql` is a common virtual provider for
> community-mysql and mariadb packages and I as a packager want to prioritize
> mariadb?

If you want to install MariaDB, why would you say "hey, DNF, install any provider of mysql, but woe to you if you choose a wrong one!" instead of "hey, DNF, install MariaDB please"? Why would you complicate DNF's life by forcing it to read your mind and your life by the need to check the DNF's decision?

Comment 10 Honza Horak 2014-10-06 14:04:19 UTC
(In reply to Radek Holy from comment #9)
> (In reply to Honza Horak from comment #8)
> > Well, but that can work only for installing dependencies when they are
> > defined in some other package. How can I solve the case when somebody runs:
> > `dnf install mysql` where the `mysql` is a common virtual provider for
> > community-mysql and mariadb packages and I as a packager want to prioritize
> > mariadb?
> 
> If you want to install MariaDB, why would you say "hey, DNF, install any
> provider of mysql, but woe to you if you choose a wrong one!" instead of
> "hey, DNF, install MariaDB please"? Why would you complicate DNF's life by
> forcing it to read your mind and your life by the need to check the DNF's
> decision?

Well, I see your point, so a bit of clarification... We need it for cases when users do not care how the default package for mysql is called (mysql is usually mentioned in many tutorials for example) and in these cases we want to install the more supported implementation obviously (mariadb). So basically, users who run dnf do not care, but we (package maintainers) do care. Is it now more clear why we care about the default choice?

Anyway, I tried the `Suggests:` and it works if there is a real package that can define "Suggests: mariadb", but when I'm trying to define the same relation using a recursive way (adding "Enhances: mysql" to mariadb) it does not work. Shouldn't Enhances and Suggests be just the opposites?

I'd like to keep this opened until we find some solution, since it feels like an important issue for us.

Comment 11 Radek Holy 2014-10-06 14:38:08 UTC
tl;dr version: I don't think that DNF should prefer a package based on its name.

I simply think that your suggestion is a hack and not a systematic solution. I don't know whather the situation you describe is hypothetical or a real problem. I mean does Fedora really care which implementation should be installed in this case? If the community implementation is not enough supported, we need to solve it there and not in DNF. If the tutorials suggest incorrect "dnf install" command, it should be fixed there. If the community implementation does not provide something that is considered a MySQL API it maybe shouldn't provide "mysql". If none of these is true then I don't understand why should be the MariaDB preferred *when* user requests "install mysql".

And even if it's just me that I don't understand the problem, I don't think your suggestion is the right way. I think that in case it's a real problem, we need either the possibility to specify user preference of packages in DNF (similar to protected packages) or a meta package representing the distro-preferred package or a spec file flag defining the priority on the provides level (ugh, that would be complex).

I'm just surprised that Fedora does not provide any way how to specify a default package. If it's true, that must be an intention.

Comment 12 Honza Silhan 2014-10-08 10:46:53 UTC
(In reply to Honza Horak from comment #10)
> Anyway, I tried the `Suggests:` and it works if there is a real package that
> can define "Suggests: mariadb", but when I'm trying to define the same
> relation using a recursive way (adding "Enhances: mysql" to mariadb) it does
> not work. Shouldn't Enhances and Suggests be just the opposites?

AFAIK this is not documented yet, Florian knows that.

> I'd like to keep this opened until we find some solution, since it feels
> like an important issue for us.

Keep it open, but change component, please, as it has nothing to do with dnf, hawkey or libsolv. Maybe to "fedora-docs"?

Remember that originaly this bug was filed because of bug 1148538, that could be fixed by Suggests.

WRT latter example of "dnf install mysql" I have the same opinion as Radek, DNF does good job by selecting the package that provides it if you are too lazy (or you don't want) to say exact package you want. Thats how are provides designed in RPM management.

> How can I compare packages by epoch only?

hawkey.Query(sack).filter(provides="mysql")[0].epoch...

Comment 13 Honza Horak 2015-01-26 16:23:16 UTC
I think you don't understand how this is important. If it wasn't, why bug#998707 would be fixed in RHEL? And other bugs bug#1183835 bug#1148538 bug#782345 would exist? Because we don't care? I don't think so.

Btw. this is a comment how 'Recommends' should work [1]:
"So installing a package containing Recommends: foo should make the depsolver also select a package either named foo or containing a Provides: foo."

Is this ^^ how dnf should work one day?

(In reply to Radek Holy from comment #11)
> tl;dr version: I don't think that DNF should prefer a package based on its
> name.

I agree, it shouldn't prefer it on it's name. But it does it. DNF always installs a package that is first in the alphabet now. What I say is that DNF should install the package with *the highest epoch-version-release*.

> I simply think that your suggestion is a hack and not a systematic solution.
> I don't know whather the situation you describe is hypothetical or a real
> problem.

It is a real problem with mariadb vs community-mysql package. We chose a name community-mysql to hack yum to work as we wanted and we were told dnf will work nicer. Now, there is the same song again only the opposite way, are we supposed to rename community-mysql to something else again so it works as we expect?

> I mean does Fedora really care which implementation should be
> installed in this case?

Yes, we care.

> If the community implementation is not enough
> supported, we need to solve it there and not in DNF. If the tutorials
> suggest incorrect "dnf install" command, it should be fixed there. If the
> community implementation does not provide something that is considered a
> MySQL API it maybe shouldn't provide "mysql". If none of these is true then
> I don't understand why should be the MariaDB preferred *when* user requests
> "install mysql".

The problem is that "it works with any of the provider" doesn't mean "we don't care which one is chosen". In other words, both packages mariadb and community-mysql satisfy requirements of another package but it doesn't mean both packages are the same.

The problem is that DNF doesn't allow us to distinguish between static dependencies (dependencies that needs to be satisfied in order packages work) and install-time dependencies -- which packages to install..

> And even if it's just me that I don't understand the problem, I don't think
> your suggestion is the right way. I think that in case it's a real problem,
> we need either the possibility to specify user preference of packages in DNF
> (similar to protected packages) or a meta package representing the
> distro-preferred package or a spec file flag defining the priority on the
> provides level (ugh, that would be complex).

No, if dnf and it's dependencies worked as they should (as rpm docs say), we wouldn't need any additional complexity.

> I'm just surprised that Fedora does not provide any way how to specify a
> default package. If it's true, that must be an intention.

I understand that this is the easiest solution for you. But why does yum have so complicated algorithm for choosing one from more providers? Because people care. And the fact that dnf doesn't care is much worse than using any nonperfect (even yum-style) way.

[1] http://www.rpm.org/wiki/PackagerDocs/DependenciesOverview#Weakdependencies

(In reply to Jan Silhan from comment #12)
> Keep it open, but change component, please, as it has nothing to do with
> dnf, hawkey or libsolv. Maybe to "fedora-docs"?

No, until dnf starts picking packages based on something else than nonsense, it does have something to do with those.

> Remember that originaly this bug was filed because of bug 1148538, that
> could be fixed by Suggests.

Sorry, I don't actually think it works. I've tested this few times and dnf always picked the first package in the alphabet, no matter what was stated in Suggests/Recommends.

Comment 14 Radek Holy 2015-01-26 18:45:45 UTC
(In reply to Honza Horak from comment #13)
> (In reply to Radek Holy from comment #11)
> > I mean does Fedora really care which implementation should be
> > installed in this case?
> 
> Yes, we care.

Is this written in an official Fedora statement? Can you provide a link please?

> > And even if it's just me that I don't understand the problem, I don't think
> > your suggestion is the right way. I think that in case it's a real problem,
> > we need either the possibility to specify user preference of packages in DNF
> > (similar to protected packages) or a meta package representing the
> > distro-preferred package or a spec file flag defining the priority on the
> > provides level (ugh, that would be complex).
> 
> No, if dnf and it's dependencies worked as they should (as rpm docs say), we
> wouldn't need any additional complexity.

You mean the weak dependencies? So, if it's broken in DNF and if we fix it, that would be OK and this problem would be solved? Otherwise I'm afraid I don't understand why did you mention it here.

> > I'm just surprised that Fedora does not provide any way how to specify a
> > default package. If it's true, that must be an intention.
> 
> I understand that this is the easiest solution for you. But why does yum
> have so complicated algorithm for choosing one from more providers? Because
> people care. And the fact that dnf doesn't care is much worse than using any
> nonperfect (even yum-style) way.

I think it's the most systematic solution. Otherwise if an alternative manager occurs (or an eventual replacement of DNF in a far future), you would have to repeat this again. I think that it should be at least stated somewhere in an official Fedora document that some packages are preferred and that all Fedora package managers should respect it.

That's my opinion. I'd be happy if anyone can explain me why I'm wrong.

Comment 15 Honza Horak 2015-01-26 21:49:25 UTC
(In reply to Radek Holy from comment #14)
> (In reply to Honza Horak from comment #13)
> > (In reply to Radek Holy from comment #11)
> > > I mean does Fedora really care which implementation should be
> > > installed in this case?
> > 
> > Yes, we care.
> 
> Is this written in an official Fedora statement? Can you provide a link
> please?

No, this is written as Honza's statement, maintainer of packages that we talk about here and who hopefully understands use cases in practice.
 
> > No, if dnf and it's dependencies worked as they should (as rpm docs say), we
> > wouldn't need any additional complexity.
> 
> You mean the weak dependencies? So, if it's broken in DNF and if we fix it,
> that would be OK and this problem would be solved? Otherwise I'm afraid I
> don't understand why did you mention it here.

Yes, if weak dependencies will work as RPM suggests, then we should be able to work-around this issue. Ideal solution with mysql example using weak deps is that mariadb package would include:

 Supplements: mysql #(as opposite to Suggests or Recommends)

and that would mean in case someone (or some package) asks for mysql, mariadb would be preferred before community-mysql.

Is that behavior what you would describe as "fixed dnf"? (I'm trying to find a common ground, since it seems we have not been on the same page)

> > I understand that this is the easiest solution for you. But why does yum
> > have so complicated algorithm for choosing one from more providers? Because
> > people care. And the fact that dnf doesn't care is much worse than using any
> > nonperfect (even yum-style) way.
> 
> I think it's the most systematic solution. Otherwise if an alternative
> manager occurs (or an eventual replacement of DNF in a far future), you
> would have to repeat this again. I think that it should be at least stated
> somewhere in an official Fedora document that some packages are preferred
> and that all Fedora package managers should respect it.
> 
> That's my opinion. I'd be happy if anyone can explain me why I'm wrong.

Other managers will need to handle this use case as well, that's right. So why not to try and find "the correct" way now? Instead of closing eyes and don't see issues it causes.

Comment 16 Honza Silhan 2015-01-27 14:57:50 UTC
(In reply to Honza Horak from comment #15)
> Yes, if weak dependencies will work as RPM suggests, then we should be able
> to work-around this issue. Ideal solution with mysql example using weak deps
> is that mariadb package would include:
> 
>  Supplements: mysql #(as opposite to Suggests or Recommends)
> 
> and that would mean in case someone (or some package) asks for mysql,
> mariadb would be preferred before community-mysql.
> 
> Is that behavior what you would describe as "fixed dnf"? (I'm trying to find
> a common ground, since it seems we have not been on the same page)
> 

Yes, this is how it should work. We'll fix Supplements recognition (bug 1184808).

Comment 17 Ľuboš Kardoš 2015-01-29 09:50:29 UTC
(In reply to Radek Holy from comment #9)
> (In reply to Honza Horak from comment #8)
> > Well, but that can work only for installing dependencies when they are
> > defined in some other package. How can I solve the case when somebody runs:
> > `dnf install mysql` where the `mysql` is a common virtual provider for
> > community-mysql and mariadb packages and I as a packager want to prioritize
> > mariadb?
> 
> If you want to install MariaDB, why would you say "hey, DNF, install any
> provider of mysql, but woe to you if you choose a wrong one!" instead of
> "hey, DNF, install MariaDB please"? Why would you complicate DNF's life by
> forcing it to read your mind and your life by the need to check the DNF's
> decision?

If I have two packages in repo: myqsql-1.1 and mysql-1.2 and I execute command "dnf install mysql" then mysql-1.2 will be installed. Similary if I have package community-mysql that provides mysql-1.1 and package mariadb that provides mysql-1.2 and I execute command "dnf install mysql" then I expect that mariadb will be installed. When I execute command "dnf install mysql" then I expect that I will get the newest version of mysql no matter if mysql is name of package or virtual provide. This is simple rule. There is no need for mind reading.

Comment 18 Radek Holy 2015-01-29 10:04:00 UTC
Did I misunderstand Honza that he wants mariadb to be preferred regardless of the version of the matching packages and their provides?
I guess no. I that case this is irrelevant and you should find a relevant bug report or file a new Bugzilla including the necessary debugging information.
If yes, the problem is much easier than I thought.

Comment 19 Honza Horak 2015-01-29 16:53:44 UTC
(In reply to Radek Holy from comment #18)
> Did I misunderstand Honza that he wants mariadb to be preferred regardless
> of the version of the matching packages and their provides?
> I guess no. I that case this is irrelevant and you should find a relevant
> bug report or file a new Bugzilla including the necessary debugging
> information.
> If yes, the problem is much easier than I thought.

Actually what Lubos describes is the best way to me and it was actually what I had on my mind in the beginning, before I started to think about Suggests, etc.

Btw. mariadb already has Epoch higher than community-mysql, so if higher epoch-version-release (do not forget about the epoch :) ) is chosen, then it's easy to do what I want... It seems clean also.

Thanks, Lubos, for making it more simple and clear.

Comment 20 Radek Holy 2015-01-29 17:32:00 UTC
So the Fedora packagers will have to maintain this version relationship as long as the preference requirement applies. That means to always keep mariadb' epoch greater than community-mysql's or at least always keep the version greater. I don't understand how this can be simpler/cleaner. Weak dependencies are designed to suggest packages while versions are used to communicate the improvements in the package itself (not some relationships)...

Comment 21 Radek Holy 2015-01-30 08:14:11 UTC
(the same applies to versions of provides, not just versions of packages)

OK, I read it all again. It seems that I probably missed that mariadb actually provides a later version of MySQL than community-mysql (not just that there is a hack in the virtual provides to workaround something). That's what was hidden in the general: "the more supported implementation"? So if it happens that community-mysql starts providing a later version of MySQL than mariadb, you would prefer community-mysql in that case, right? In case it's true, the discussion makes much more sense. Sorry for my slowness. I thought that you want to prefer mariadb to community-mysql in any case.

Comment 22 Jan Zeleny 2015-01-30 08:20:01 UTC
Two notes:

First of all I'd like to make sure you guys are on the same page. What Lubos was describing seemed to me like versioned provides. On the other hand, what Honza seems to be talking about is having versioned packages and not provides. I am no expert in this area so it's possible that I'm way off but I want to make sure you are not talking about two different things.

WRT weak deps, I'd like to point out that their primary purpose is not to define preference in packages. Their purpose is to create new relations between packages at places where they did not exist before. If you guys intend to use them to declare preferences, I guess that's not a problem. We should, however, discuss that with the rest of the team and make sure this is well understood, approved and documented/communicated to the users. Otherwise it might cause the same complications as the current practice to utilize specific behavior of yum depsolver does.

Comment 23 Honza Horak 2015-01-31 08:35:28 UTC
(In reply to Radek Holy from comment #21)
> OK, I read it all again. It seems that I probably missed that mariadb
> actually provides a later version of MySQL than community-mysql (not just
> that there is a hack in the virtual provides to workaround something).
> That's what was hidden in the general: "the more supported implementation"?
> So if it happens that community-mysql starts providing a later version of
> MySQL than mariadb, you would prefer community-mysql in that case, right?

Exactly, this is how I'd like to see it works.

(In reply to Jan Zeleny from comment #22)
> Two notes:
> 
> First of all I'd like to make sure you guys are on the same page. What Lubos
> was describing seemed to me like versioned provides. On the other hand, what
> Honza seems to be talking about is having versioned packages and not
> provides. I am no expert in this area so it's possible that I'm way off but
> I want to make sure you are not talking about two different things.

IMHO provides should usually be versioned with the same e-v-r as the package they are used in, so in the end it should be one thing.
 
> WRT weak deps, I'd like to point out that their primary purpose is not to
> define preference in packages. Their purpose is to create new relations
> between packages at places where they did not exist before. If you guys
> intend to use them to declare preferences, I guess that's not a problem. We
> should, however, discuss that with the rest of the team and make sure this
> is well understood, approved and documented/communicated to the users.
> Otherwise it might cause the same complications as the current practice to
> utilize specific behavior of yum depsolver does.

Provided the other solution with preferring higher version (of package or provide) works, I won't push on weak deps to be helpful in this scenario any more. Although, according to my understanding it will be still how RPM documentation describes weak deps functionality in this case.

Comment 24 Radek Holy 2015-02-04 10:22:21 UTC
Well, talking to other colleagues, it seems this is still not clear. Can you please clearly say what is your goal?

In the first comment, you said that "MariaDB is the default". And I understood the last comment that the latest MySQL is supposed to be the default. It sounds like a contradiction. So,

1) you want mariadb to be preferred regardless of the state of community-mysql,
2) or just as long as mariadb contains the later version of MySQL than community-mysql?

(We would like to find the correct solution. Not to force you to abuse provide versions to achieve a package preference. Provides should match the content of the package.)

Comment 25 Honza Horak 2015-02-04 12:13:11 UTC
(In reply to Radek Holy from comment #24)
> Well, talking to other colleagues, it seems this is still not clear. Can you
> please clearly say what is your goal?
>
> In the first comment, you said that "MariaDB is the default". And I
> understood the last comment that the latest MySQL is supposed to be the
> default. It sounds like a contradiction. So,

This seems more like a misunderstanding since now we have packages mariadb and community-mysql that both provide symbol mysql in fedora, but there is no mysql package in its own in fedora any more. It is also not clear every time if MySQL refers to the Oracle's package or just a general term for any MySQL implementation.

So, to make it easier, get rid of mysql/mariadb mess and let's describe it on a more general level:

What is my issue: I have two packages foo and bar that both may be installed in order to satisfy one particular dnf transaction. In that case I need to have an apparatus in dnf to influence the dnf's decision, so e.g. foo will be chosen to ssatisfy the transaction.

> 1) you want mariadb to be preferred regardless of the state of
> community-mysql,
> 2) or just as long as mariadb contains the later version of MySQL than
> community-mysql?

Well, primarily (1) but I'd be fine with both solutions, because I can easily swap versions relation with bumping epoch. And the answer is also related to a specific case, since it may depend on if the provide has a version or not.

So, let's rather see some specific examples..

Example with versioned provides (no soft dependencies are used):
package foo defines Provides: mysym=3.0
package bar defines Provides: mysym=2.0
If some other package has Requires: mysym, I expect foo will be installed because it provides better (higher version) mysym.

Some provides are not versioned or cannot be:
both packages foo and bar provide file /usr/bin/mybin
A third package then defines Requires: /usr/bin/mybin
What should happen in case i want to install the third package? dnf should do a qualified decision based on versions of foo and bar, but also should look at weak dependencies if there are some. Don't ask me what should have a priority :)

Sorry to bring more questions than answers.. and I'm also not sure if I'm doing it clearer or more complicated.

Comment 26 Radek Holy 2015-02-04 12:42:16 UTC
OK, it seems that I have to leave this bug to someone else since it seems I'm not able to understand you...

(In reply to Honza Horak from comment #25)
> Well, primarily (1) but I'd be fine with both solutions, because I can
> easily swap versions relation with bumping epoch. And the answer is also
> related to a specific case, since it may depend on if the provide has a
> version or not.

I understand this that the "mariadb as the default" is what you care. You don't care about the version of the "mysql" symbol that "mariadb" and "community-mysql" packages provide. (But then I don't understand WHY is mariadb preferred. What makes it objectively better than community-mysql. Why it's not just *your* preference.)

> So, let's rather see some specific examples..
> 
> Example with versioned provides (no soft dependencies are used):
> package foo defines Provides: mysym=3.0
> package bar defines Provides: mysym=2.0
> If some other package has Requires: mysym, I expect foo will be installed
> because it provides better (higher version) mysym.

If the above is true, then this example is just misleading because you expect that foo will be installed even if foo provides "mysym=2.0" and bar provides "mysym=3.0". If not, then you goal is *not* "primarily (1)".

> Some provides are not versioned or cannot be:
> both packages foo and bar provide file /usr/bin/mybin
> A third package then defines Requires: /usr/bin/mybin
> What should happen in case i want to install the third package? dnf should
> do a qualified decision based on versions of foo and bar, but also should
> look at weak dependencies if there are some. Don't ask me what should have a
> priority :)

No, I'm strongly against comparing different packages based on their versions. Versions of different packages are not comparable. Every project uses different versioning scheme (look e.g. at Mozilla Firefox).

Comment 27 Radek Holy 2015-02-04 12:44:42 UTC
(In reply to Radek Holy from comment #26)
> (But then I don't understand WHY is
> mariadb preferred. What makes it objectively better than community-mysql.
> Why it's not just *your* preference.)

(And DNF must have access to this objective quality in the *appropriate* package metadata to be able to compare it.)

Comment 28 Radek Holy 2015-02-05 09:37:13 UTC
Created attachment 988462 [details]
dnf --debugsolver install mysql

Comment 29 Radek Holy 2015-02-05 09:37:51 UTC
FTR, we have personally met with Honza. As a result of the meeting we are going to propose means to define distribution-wide preferences.

In the meantime, I can confirm that DNF does not install the package that provides the latest version of the requested virtual provide:

$ sudo dnf install mysql
Dependencies resolved.
================================================================================
 Package                     Arch        Version              Repository   Size
================================================================================
Installing:
 community-mysql             x86_64      5.6.21-5.fc21        fedora      6.4 M

[...]

$ dnf repoquery --provides community-mysql
[...]
mysql = 5.6.21-5.fc21
mysql(x86-64) = 5.6.21-5.fc21
[...]
$ dnf repoquery --provides mariadb
[...]
mysql = 1:10.0.14-7.fc21
mysql(x86-64) = 1:10.0.14-7.fc21
[...]
mysql = 1:10.0.15-4.fc21
mysql(x86-64) = 1:10.0.15-4.fc21
[...]

(I did not investigate yet why "dnf repoquery" lists two different versions of the provide while "repoquery" doesn't and whether that might be the problem.)

although both providers can be installed:

$ sudo dnf install mariadb
Dependencies resolved.
================================================================================
 Package              Arch         Version                  Repository     Size
================================================================================
Installing:
 mariadb              x86_64       1:10.0.15-4.fc21         updates       6.0 M
[...]
$ sudo dnf install community-mysql
Dependencies resolved.
================================================================================
 Package                     Arch        Version              Repository   Size
================================================================================
Installing:
 community-mysql             x86_64      5.6.21-5.fc21        fedora      6.4 M
[...]

So, let me use this bug to track this issue.

Comment 30 Radek Holy 2015-02-05 09:51:17 UTC
Also we are not going to document every detail of the package selection process because we don't want package maintainers to abuse these rules (like they did with YUM). We prefer to find the right solutions for the concrete problems.

And for the further readers: Please note that the upcoming fix fixes this particular requirement (to prefer mariadb to community-mysql), but THIS IS JUST A WORKAROUND since provides are not designed to express preferences (or other relations) between different packages. This also is not a systematic solution of the problem since the maintainers have to maintain the inequality of the virtual provide's version in both packages forever (once EVR of community-mysql's provide bumps the EVR of mariadb's provide might have to be bumped to). And I don't mention the case when there is more than two packages in such a relationship. So, if you need to solve the same problem, use the new DNF's feature that we are going to implement (see above).

Comment 31 Honza Silhan 2015-07-21 09:02:17 UTC
it's fixed in libsolv 0.6.10. The provides with latest version is picked into transaction first.