Bug 1906310

##### To build amazon-id.so:

On RHEL8.2

[ec2-user@ip-172-31-40-99 ~]$ tar xvf libdnf-plugin-amazon-id.tar 
[ec2-user@ip-172-31-40-99 ~]$ sudo dnf install cmake make gcc glib2-devel json-glib-devel
[ec2-user@ip-172-31-40-99 ~]$ sudo dnf install libdnf-devel-0.48.0-5.el8.x86_64.rpm librepo-devel-1.12.0-2.el8.x86_64.rpm libsolv-devel-0.7.11-1.el8.x86_64.rpm
[ec2-user@ip-172-31-40-99 ~]$ cmake .
-- Checking for module 'json-glib-1.0'
--   Found json-glib-1.0, version 1.4.4
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ec2-user
[ec2-user@ip-172-31-40-99 ~]$ make
Scanning dependencies of target amazon-id
[ 33%] Building C object CMakeFiles/amazon-id.dir/amazon-id.c.o
[ 66%] Building C object CMakeFiles/amazon-id.dir/util.c.o
[100%] Linking C shared library amazon-id.so
[100%] Built target amazon-id


##### Confirm that pkcon doesn't work

[ec2-user@ip-172-31-40-99 ~]$ sudo pkcon install zsh
Resolving                     [                         ] (3%)  Command failed: This tool could not find any available package: cannot update repo 'rhel-8-appstream-rhui-rpms': Cannot prepare internal mirrorlist: Curl error (6): Couldn't resolve host name for https://rhui3.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/rhel8/rhui/8/x86_64/appstream/os [Could not resolve host: rhui3.REGION.aws.ce.redhat.com]; Last error: Curl error (6): Couldn't resolve host name for https://rhui3.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/rhel8/rhui/8/x86_64/appstream/os [Could not resolve host: rhui3.REGION.aws.ce.redhat.com]


##### Apply the libdnf plugin
[ec2-user@ip-172-31-40-99 ~]$ sudo cp amazon-id.so /usr/lib64/libdnf/plugins

##### Change REGION into $REGION in the repo configuration. This is required for this libdnf plugin.
[ec2-user@ip-172-31-40-99 ~]$ sudo sed -i 's/REGION/$REGION/g' /etc/yum.repos.d/*

##### Confirm that it works.
[ec2-user@ip-172-31-40-99 ~]$ sudo pkcon install zsh
Resolving                     [=========================]         
Querying                      [=========================]         
Testing changes               [=========================]         
Finished                      [                         ] (0%)  
The following packages have to be installed:
 zsh-5.5.1-6.el8_1.2.x86_64	Powerful interactive shell
Proceed with changes? [N/y] y

                              [=========================]         
Installing                    [=========================]         
Querying                      [=========================]         
Downloading packages          [=========================]         
Testing changes               [=========================]         
Installing packages           [=========================]         
Finished                      [=========================]

> ##### Change REGION into $REGION in the repo configuration. This is required for this libdnf plugin.
> [ec2-user@ip-172-31-40-99 ~]$ sudo sed -i 's/REGION/$REGION/g' /etc/yum.repos.d/*

After this change, the dnf command doesn't work any more. It's required to change REGION into $REGION even in the dnf python plugin, /usr/lib/python3.6/site-packages/dnf-plugins/amazon-id.py.

--- ./amazon-id.py.original	2020-12-10 09:32:45.485443577 +0000
+++ ./amazon-id.py	2020-12-10 09:33:08.710339660 +0000
@@ -123,10 +123,10 @@ class AmazonID(dnf.Plugin):
 
         if repo.baseurl:
             repo.baseurl = tuple(
-                url.replace('REGION', region, 1) for url in repo.baseurl
+                url.replace('$REGION', region, 1) for url in repo.baseurl
             )
         elif repo.mirrorlist:
-            repo.mirrorlist = repo.mirrorlist.replace('REGION', region, 1)
+            repo.mirrorlist = repo.mirrorlist.replace('$REGION', region, 1)
         else:
             raise dnf.exceptions.RepoError("RHUI repository %s does not have an url" % repo.name)

I have a good news and bad news - The good news is that the support of the plugins will be fixed in LIBDNF5. The bad news is that the component is planned for RHEL10.

The implementation of the concept of shared plugins in current libdnf would be quite difficult due to compatibility. If we will allow to use the current libdnf plugins with `dnf` it will create at least change in behavior that can break thinks. But this is only a minor thing - It would be technically nearly impossible because LIBDNF does not have an access to a lot of information stored in dnf-python side. It is one of the reasons why we developed a new library because such a changes cannot be achieved without braking API and a huge restructuralisation of the project.

I am curious what is a valid path substitution in AMAZON cloud - `https://something/$REGION/something` or `https://something/REGION/something`.

If `$REGION` is uniquely used then DNF, microdnf, and packagekit can use substitutions, a general mechanism supported in DNF and LIBDNF. VARS are stored in `/etc/dnf/vars/` and it only requires to create a script that will store properly or update variables in `/etc/dnf/vars/`.

We use REGION in the URL and the dnf plugin replaces this string with the region of the given instance. The script does much more than that, though. See /usr/lib/python3.6/site-packages/dnf-plugins/amazon-id.py on a RHEL 8 AWS instance.

Just on this note, there is another consequence of this issue. 
Image builder doesn't work as well, as packagekit tries to resolve URL from repo.yaml

Sep 13 12:52:42 ip-172-31-45-154.ap-southeast-2.compute.internal packagekitd[5186]: LRO_MIRRORLISTURL processing failed: Curl error (6): Couldn't resolve host name for https://rhui.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/rhel8/rhui/8/aarch64/appstream/os [Could not resolve host: rhui.REGION.aws.ce.redhat.com]
Sep 13 12:55:09 ip-172-31-45-154.ap-southeast-2.compute.internal packagekitd[5186]: LRO_MIRRORLISTURL processing failed: Curl error (6): Couldn't resolve host name for https://rhui.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/rhel8/rhui/8/aarch64/appstream/os [Could not resolve host: rhui.REGION.aws.ce.redhat.com]
Sep 13 12:55:09 ip-172-31-45-154.ap-southeast-2.compute.internal packagekitd[5186]: LRO_MIRRORLISTURL processing failed: Curl error (6): Couldn't resolve host name for https://rhui.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/rhel8/rhui/8/aarch64/baseos/os [Could not resolve host: rhui.REGION.aws.ce.redhat.com]
Sep 13 12:55:10 ip-172-31-45-154.ap-southeast-2.compute.internal packagekitd[5186]: LRO_MIRRORLISTURL processing failed: Curl error (6): Couldn't resolve host name for https://rhui.REGION.aws.ce.redhat.com/pulp/mirror/content/dist/layered/rhui/rhel8/aarch64/ansible/2/os [Could not resolve host: rhui.REGION.aws.ce.redhat.com]
Sep 13 12:55:10 ip-172-31-45-154.ap-southeast-2.compute.internal packagekitd[5186]: LRO_MIRRORLISTURL processing failed: Curl error (6):

Dear Masahiro-san and other followers of this bug report,

There's a new package in all RHUI client configuration repos for RHEL 8 and 9, both architectures: amazon-libdnf-plugin.

It can be installed manually, or it will be installed automatically when rh-amazon-rhui-client<any flavor> is updated. This can happen manually, or using the update-client-config-packages daily cron job, if a recent rh-amazon-rhui-client version is already installed.

Note that if you've installed PackageKit before, you will have to restart service packagekit for this package manager to reload libdnf plugins and thereby notice the new Amazon ID plugin), allowing pkcon and/or cockpit to work in AWS. Microdnf doesn't need this post-installation step.

Kind regards,
Radek