Bug 1809314 - libdnf doesn't handle filtered packages whose name matches the SRPM properly
Summary: libdnf doesn't handle filtered packages whose name matches the SRPM properly
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: libdnf
Version: 8.2
Hardware: All
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Jaroslav Mracek
QA Contact: Eva Mrakova
URL:
Whiteboard:
Depends On:
Blocks: 1836147 1759510 1895872
TreeView+ depends on / blocked
 
Reported: 2020-03-02 20:21 UTC by Stephen Gallagher
Modified: 2021-05-18 15:01 UTC (History)
16 users (show)

Fixed In Version: libdnf-0.55.0-3.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-05-18 15:00:34 UTC
Type: Bug
Target Upstream Version:


Attachments (Terms of Use)

Description Stephen Gallagher 2020-03-02 20:21:51 UTC
Description of problem:

My best guess is that DNF is taking the name portion of the NEVRA matching the source RPM and using that to exclude the non-modular version of binary RPMs that match. In this case, I created a `libuv` module that builds libuv-devel from the libuv sources that match RHEL and ships them, filtering out `libuv` so it doesn't overwrite the RHEL 8 version. Unfortunately, DNF tries to exclude the non-modular libuv-1:1.23.1-1.el8.x86_64 and the transaction fails (since the -devel package Requires: libuv >= 1.23.1.


Version-Release number of selected component (if applicable):
libdnf-0.22.5-4.el8.x86_64


How reproducible:
Every time

Steps to Reproduce:
1. Install RHEL/CentOS 8 in a container
2. dnf -y install epel-release (needs to be epel-release-8-8.el8.noarch or later)
3. `dnf --enablerepo=epel-testing-modular module install libuv:epel8-buildroot`

Actual results:

Failed to set locale, defaulting to C
Last metadata expiration check: 0:11:02 ago on Mon Mar  2 19:56:15 2020.
Error: 
 Problem: package libuv-devel-1:1.23.1-1.module_el8+8246+46ee4016.x86_64 requires libuv(x86-64) = 1:1.23.1, but none of the providers can be installed
  - conflicting requests
  - package libuv-1:1.23.1-1.el8.x86_64 is excluded
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)



Expected results:

This should install correctly.


Additional info:


Relevant modulemd for libuv:

---
document: modulemd-defaults
version: 1
data:
  module: libuv
  profiles:
    epel8-buildroot: [devel]
...
---
document: modulemd
version: 2
data:
  name: libuv
  stream: epel8-buildroot
  version: 820200228154227
  context: 9edba152
  arch: x86_64
  summary: libuv-devel for EPEL 8
  description: >-
    This is a compatibility module that should be used only for developing against
    libuv for EPEL 8.
  license:
    module:
    - MIT
    content:
    - MIT and BSD and ISC
  xmd: {}
  dependencies:
  - buildrequires:
      platform: [el8]
    requires:
      platform: [el8]
  profiles:
    devel:
      rpms:
      - libuv-devel
  filter:
    rpms:
    - libuv
    - libuv-debuginfo
    - libuv-debugsource
  components:
    rpms:
      libuv:
        rationale: The libuv-devel package
        ref: epel8-buildroot
        arches: [aarch64, armv7hl, i686, ppc64le, s390x, x86_64]
  artifacts:
    rpms:
    - libuv-1:1.23.1-1.module_el8+8246+46ee4016.src
    - libuv-devel-1:1.23.1-1.module_el8+8246+46ee4016.x86_64
    - libuv-static-1:1.23.1-1.module_el8+8246+46ee4016.x86_64
...

Comment 1 Stephen Gallagher 2020-03-17 12:40:39 UTC
Moving this to the RHEL product, since it will have to be fixed there.

Comment 2 Petr Špaček 2020-04-09 11:20:28 UTC
Any ETA on this? It is effectively preventing some packages from being built in EPEL.

Comment 3 Daniel Mach 2020-04-09 14:05:54 UTC
I managed to reproduce it with ubi8:8.1 image, but it's not reproducible in my container with RHEL 8.2.
I believe it's a duplicate of another bug we've fixed already.

Comment 4 Stephen Gallagher 2020-04-09 16:26:32 UTC
(In reply to Daniel Mach from comment #3)
> I managed to reproduce it with ubi8:8.1 image, but it's not reproducible in
> my container with RHEL 8.2.
> I believe it's a duplicate of another bug we've fixed already.

I just tried it again with 8.2 packages:
[root@d543b84ae1dc /]# rpm -q dnf libdnf yum
dnf-4.2.17-6.el8.noarch
libdnf-0.39.1-5.el8.x86_64


[root@d543b84ae1dc /]# dnf  module install libuv:epel8-buildroot
Failed to set locale, defaulting to C.UTF-8
CentOS-8 - AppStream                                                                                         4.1 MB/s | 6.6 MB     00:01    
CentOS-8 - Base                                                                                              4.3 MB/s | 5.0 MB     00:01    
CentOS-8 - Extras                                                                                             13 kB/s | 4.9 kB     00:00    
Extra Packages for Enterprise Linux Modular 8 - x86_64                                                       118 kB/s | 116 kB     00:00    
Extra Packages for Enterprise Linux 8 - x86_64                                                               3.2 MB/s | 6.2 MB     00:01    
Error: 
 Problem: package libuv-devel-1:1.23.1-1.module_el8+8246+46ee4016.x86_64 requires libuv(x86-64) = 1:1.23.1, but none of the providers can be installed
  - conflicting requests
  - package libuv-1:1.23.1-1.el8.x86_64 is filtered out by modular filtering
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Comment 5 Michel Alexandre Salim 2020-04-23 02:11:34 UTC
Daniel, could we get an update on this? Thanks

Comment 6 Marek Blaha 2020-05-06 07:30:44 UTC
Yes, current behaviour of dnf is that all names from `artifacts` section are used for filtering out non-modular packages (also packages that provide any of this names are filtered out).
Do I understand it correctly - you mean that names from `filter` section should not be used for this filtering?

Comment 7 Stephen Gallagher 2020-05-06 11:27:56 UTC
(In reply to Marek Blaha from comment #6)
> Yes, current behaviour of dnf is that all names from `artifacts` section are
> used for filtering out non-modular packages (also packages that provide any
> of this names are filtered out).
> Do I understand it correctly - you mean that names from `filter` section
> should not be used for this filtering?

I don't understand what you are trying to ask with this, so I'll just try to rephrase my request.


In the case that triggered this ticket we have a case where the source package name (libuv) matches the name of one of its produced binary RPMs. However, since we're filtering this binary RPM out in `data.filter.rpms` and it doesn't appear in `data.artifacts.rpms`, DNF should not be hiding the non-modular `libuv` package from the transaction. My suspicion is that what is happening is that DNF sees `libuv-1:1.23.1-1.module_el8+8246+46ee4016.src` (note the .src suffix), takes the base package name from that and removes the non-modular libuv from view. Since no replacement is available from the module stream, the transaction cannot be resolved.


The behavior I expect from `data.filter.rpms` is that binary packages produced by the module build for this stream which have those names would neither be included in the `data.artifacts.rpms` list nor delivered to the composed modular repository (meaning Pungi would exclude them when generating the repo). To the best of my knowledge, this part is done correctly. The `data.filter.rpms` field should have *no effect* on DNF install/update transactions.


The behavior I expect from `data.artifacts.rpms` is as follows:

 * Any package NEVRA that appears in this list *for the current architecture* should cause any non-modular package (that is not in a module_hotfixes repo) to be hidden from the transaction. Right now, the problem is that if a name appears in *any* architecture, it's suppressed from all architectures. Post-processing the list and making sure to only suppress those artifacts that belong to the primary architecture (e.g. x86_64), the multilib architecture (where appropriate, e.g i686) and the noarch architecture would resolve this issue. We should *never* be including the source architecture in this calculation.

 * If two different visible module streams (any combination of enabled or default) provide artifacts with the same base name (N), they should both be part of the transaction and resolved with the usual DNF preference (higher NEVRA wins).

Comment 8 Stephen Gallagher 2020-05-15 11:32:41 UTC
Ping?

Comment 10 Marc Dequènes (Duck) 2020-10-08 07:12:06 UTC
Any news?

Comment 11 Jaroslav Mracek 2020-11-09 09:26:46 UTC
I am really sorry but what I have to say that the current behaviour is correct or according to original design of modularity. It is expected that all non-modular packages with the same name or provide as the name of modular package will be filtered out. Why it is like that - because modular rpms do not have a clear upgrade path.

Why we cannot change it - we will brake Satellite. Satellite has its own implementation of modularity therefore any changes in behaviour in dnf must be reflected in Satellite.

How to resolve it in future - new design that will not require to filter out non-modular packages.

Comment 12 Jaroslav Mracek 2020-11-09 09:30:28 UTC
The issue could be aso solved by the same way as Bug 1805260 - Old module packages still show as default. By adding of extrametadat that will tel to not filter out `libuv`.

Comment 15 Jaroslav Mracek 2020-11-09 17:21:19 UTC
I create pull-request - https://github.com/rpm-software-management/libdnf/pull/1080 that change handling of src artefacts and src filtering.

Comment 16 Jaroslav Mracek 2020-11-10 10:19:22 UTC
Additional PRs.
Test: rpm-software-management/ci-dnf-stack#908
Documentation: rpm-software-management/dnf#1681

Comment 24 errata-xmlrpc 2021-05-18 15:00:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (dnf bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1657


Note You need to log in before you can comment on or make changes to this bug.