Bug 1756382

Summary: [libdnf] PackageKit's DNF backend doesn't apply all variable substitutions.
Product: Red Hat Enterprise Linux 8 Reporter: Daniel Mach <dmach>
Component: libdnfAssignee: Jaroslav Rohel <jrohel>
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: unspecified Docs Contact:
Priority: urgent    
Version: 8.0CC: amatej, desktop-qa-list, dvolkov, emrakova, hasuzuki, jkoten, jrohel, klember, pasik, rhughes, riehecky
Target Milestone: rcKeywords: Triaged
Target Release: 8.0Flags: pm-rhel: mirror+
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: libdnf-0.38.1-1.el8 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1755424 Environment:
Last Closed: 2020-04-28 16:48:23 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1755424    
Bug Blocks:    

Description Daniel Mach 2019-09-27 13:45:33 UTC
+++ This bug was initially created as a clone of Bug #1755424 +++

Description of problem:
PackageKit isn't correctly reading in all the vars from /etc/dnf/vars/ (and /etc/yum/vars/)


Version-Release number of selected component (if applicable):
PackageKit-1.1.12-2.el8

How reproducible:
100%

Steps to Reproduce:
1. Create a repo with a local variable in the url path
  example
  http://mirror.centos.org/$contentdir/$stream/AppStream/x86_64/os/repodata/repomd.xml

2. Define the variables in /etc/dnf/vars

3. dnf makecache

4. pkcon refresh

Actual results:
Fatal error: cannot update repo 'Stream-AppStream': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried; Last error: Status code: 404 for http://mirror.centos.org/$contentdir/$stream/AppStream/x86_64/os/repodata/repomd.xml

Expected results:
Variables are expanded in PackageKit in the same manner as dnf

Additional info:
https://github.com/hughsie/PackageKit/issues/339

Comment 3 Jaroslav Rohel 2019-10-01 06:57:32 UTC
I tested the patch with microdnf.
The patch fixes only the reported problem. There are known incompatibilities with DNF that are going to be fixed later.
- Variables are read only from files. (DNF also supports reading variables from environment.)
- Variable substitutions are applied only in URLs. (DNF also supports of values substitution in other options.)
- Only "$variable" syntax is supported. (DNF also support "${var_name}" - with braces.)

Comment 4 Pat Riehecky 2019-10-01 18:52:49 UTC
I'm a bit surprised by some of these limitations....

For "Variables are read only from files. (DNF also supports reading variables from environment.)" An earlier PR seems to have the relevant functions
https://github.com/rpm-software-management/libdnf/pull/748/files#diff-1c835575752738fff3fff22f4a33fabbR622

For "Only "$variable" syntax is supported. (DNF also support "${var_name}" - with braces.)" Couldn't `ConfigParser::substitute` be used for the actual substitions?
https://github.com/rpm-software-management/libdnf/blob/master/libdnf/conf/ConfigParser.cpp#L29

I'd probably want to move `ConfigParser::substitute` into perhaps `utils` so it is a bit more universal though....

Comment 5 Jaroslav Rohel 2019-10-02 13:26:15 UTC
The problem is more complex. So, I backported only changes needed to solve reported bug.
> PackageKit isn't correctly reading in all the vars from /etc/dnf/vars/ (and /etc/yum/vars/)

There are two implementations for reading variables from files (in DNF and in libdnf) now. And there are two implementations of substitution (in libdnf and librepo). 
Our long plan is consolidate of the code (cleaning, remove duplication, ...). That is why known incompatibilities will be fixed later. We are working on it.

Comment 7 Jaroslav Rohel 2019-11-18 08:32:39 UTC
*** Bug 1767345 has been marked as a duplicate of this bug. ***

Comment 15 Jaroslav Rohel 2020-01-30 13:25:14 UTC
Notes:
- To invalidate and reload PackageKit cache after configuration changes is better to use "pkcon refresh force".
- The variables are automatically reloaded after change of any repository configuration file (*.repo) or repository directory (eg. after "touch /etc/yum.repos.d/").
- The reloading of variables can be forced by restarting PackageKit daemon (eg. "systemctl restart packagekit").

Comment 17 Jaroslav Rohel 2020-02-10 13:30:07 UTC
*** Bug 1755424 has been marked as a duplicate of this bug. ***

Comment 19 errata-xmlrpc 2020-04-28 16:48:23 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:1823