Bug 1768206 - DNF prompts for GPG key import for "repo_gpgcheck=1"-repositories despite "rpm --import"-ing the keys first
Summary: DNF prompts for GPG key import for "repo_gpgcheck=1"-repositories despite "rp...
Keywords:
Status: ASSIGNED
Alias: None
Product: Fedora
Classification: Fedora
Component: dnf
Version: rawhide
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Jaroslav Rohel
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-11-03 12:08 UTC by Eveline Raine
Modified: 2024-03-10 17:47 UTC (History)
44 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed:
Type: Bug
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Fedora Pagure releng issue 9391 0 None None None 2020-04-08 13:08:18 UTC
Github containers/libpod#4431 0 None None None 2020-09-21 10:37:31 UTC
Github containers libpod issues 4431 0 'None' closed `dnf install -y` in a container gets `SIGPIPE` on GPG key import prompt 2021-01-27 18:28:36 UTC

Description Eveline Raine 2019-11-03 12:08:09 UTC
Description of problem:

Trying to refresh repo metadata for the first time after adding a new repository with "repo_gpgcheck=1" prompts for importing the signing keys, regardless of those keys being already imported with "rpm --import".

Version-Release number of selected component (if applicable):

dnf-4.2.9-5.fc31.x86_64

How reproducible: always

Steps to Reproduce:
1. Get a .repo, for a repository with signed metadata, like this one:

[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

2. Create a fresh Fedora container with this repo installed:
(Or instead you can try on your host, if you don't have this repo already)

$ podman run -it -v ./kubernetes.repo:/etc/yum.repos.d/kubernetes.repo:Z fedora:31 bash

3. Import all signing keys for this repo:

[root@8493db416f4a /]# rpm --import https://packages.cloud.google.com/yum/doc/yum-key.gpg
[root@8493db416f4a /]# rpm --import https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
[root@8493db416f4a /]# rpm -qa --qf '%{VERSION}-%{RELEASE} %{SUMMARY}\n' *pubkey*
3c3359c4-5c6ae44d Fedora (31) <fedora-31-primary> public key
ba07f4fb-5ac168db Google Cloud Packages Automatic Signing Key <gc-team> public key
a7317b0f-551deab2 Google Cloud Packages Automatic Signing Key <gc-team> public key
3e1ba8d5-558ab6a8 Google Cloud Packages RPM Signing Key <gc-team> public key

4. Try refreshing the metadata:

[root@8493db416f4a /]# dnf updateinfo
Fedora Modular 31 - x86_64                                                                                                    438 kB/s | 5.2 MB     00:12    
Fedora Modular 31 - x86_64 - Updates                                                                                          112 kB/s | 1.2 MB     00:11    
Fedora 31 - x86_64 - Updates                                                                                                  2.3 MB/s | 5.8 MB     00:02    
Fedora 31 - x86_64                                                                                                            2.9 MB/s |  71 MB     00:24    
Kubernetes                                                                                                                    267  B/s | 454  B     00:01    
Kubernetes                                                                                                                    5.8 kB/s | 1.8 kB     00:00    
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team>"
 Fingerprint: D0BC 747F D8CA F711 7500 D6FA 3746 C208 A731 7B0F
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Is this ok [y/N]: ^C
Importing GPG key 0xBA07F4FB:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team>"
 Fingerprint: 54A6 47F9 048D 5688 D7DA 2ABE 6A03 0B21 BA07 F4FB
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Is this ok [y/N]: ^C
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key <gc-team>"
 Fingerprint: 3749 E1BA 95A8 6CE0 5454 6ED2 F09C 394C 3E1B A8D5
 From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Is this ok [y/N]: ^C   
Failed to download metadata for repo 'kubernetes'
Ignoring repositories: kubernetes
Last metadata expiration check: 0:00:03 ago on Sun Nov  3 11:40:16 2019.
...



Actual results:

"dnf updateinfo" prompts to accept Kubernetes GPG keys to validate metadata.


Expected results:

"dnf updateinfo" uses the already-imported GPG keys and validates metadata without prompting for key import.


Additional info:

DNF does make a use of pre-imported GPG keys for package signature checks. So if "repo_gpgcheck" is set to "0", running "dnf install -y kubectl" does not show any prompts.\

See the link to GitHub Issue with a scenario how this prompt leads to failed container builds / exec-s.

Comment 1 Martial Rioux 2020-02-08 05:15:13 UTC
Hi,

As per the solution ID 3720351

https://access.redhat.com/solutions/3720351

You need to perform both operation rpm --import and rpmkeys --import

# Import k8S Key
rpm --import https://packages.cloud.google.com/apt/doc/rpm-package-key.gpg
rpmkeys --import https://packages.cloud.google.com/apt/doc/rpm-package-key.gpg
rpm --import https://packages.cloud.google.com/apt/doc/yum-key.gpg
rpmkeys --import https://packages.cloud.google.com/apt/doc/yum-key.gpg

# Install K8s
dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

# Enable K8s
systemctl enable kubelet

Comment 2 Antoine TRAN 2020-02-11 15:27:01 UTC
Hi, I have the same issue in Redhat 7.7. I tried to do these commands:

rpm --import https://packages.cloud.google.com/apt/doc/rpm-package-key.gpg
rpmkeys --import https://packages.cloud.google.com/apt/doc/rpm-package-key.gpg
rpm --import https://packages.cloud.google.com/apt/doc/yum-key.gpg
rpmkeys --import https://packages.cloud.google.com/apt/doc/yum-key.gpg

and it changed nothing. It seems to me that the keys are read for package signature check, but not for repository signature check. Here is however a workaround:
        # https://unix.stackexchange.com/questions/337257/how-to-get-yum-to-use-the-already-imported-gpg-keys
        yum -q makecache -y --disablerepo='*' --enablerepo=kubernetes

Then yum will import itself the gpg keys.

Comment 3 Miroslav Suchý 2020-03-13 20:19:38 UTC
I discovered this when using fedora-upgrade and this is quite painful as it blocked the whole process. Fortunately, after the upgrade is finished.

Reproducer of F32:
1) rpm --import /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-32-primary
2) /usr/bin/package-cleanup --orphans

The output is:
Importing GPG key 0x12C944D0:
 Userid     : "Fedora (32) <fedora-32-primary>"
 Fingerprint: 97A1 AE57 C3A2 372C CA3A 4ABA 6C13 026D 12C9 44D0
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-32-x86_64
Is this ok [y/N]:

I Ctrl+C this and as you can see

# rpm -q gpg-pubkey
gpg-pubkey-81b46521-55b3ca9a
gpg-pubkey-5250aef3-573f58da
gpg-pubkey-b7546f06-5544d2dc
gpg-pubkey-96ca6280-5544d430
gpg-pubkey-fdb19c98-56fd6333
gpg-pubkey-6806a9cb-562bce39
gpg-pubkey-fa7a179a-562bcd6e
gpg-pubkey-7fac5991-4615767f
gpg-pubkey-64dab85d-57d33e22
gpg-pubkey-f5282ee4-58ac92a3
gpg-pubkey-7d838377-582f2494
gpg-pubkey-b9c13282-582f265a
gpg-pubkey-9db62fb1-59920156
gpg-pubkey-09eab3f2-595fbba3
gpg-pubkey-7f858107-595fbac5
gpg-pubkey-429476b4-5a886537
gpg-pubkey-42f19ed0-5a09d645
gpg-pubkey-d6841af8-5ac3b505
gpg-pubkey-62c29d00-5be4887a
gpg-pubkey-cfc659b9-5b6eac67
gpg-pubkey-c0aeda6e-5ad45327
gpg-pubkey-1d14a795-5ad4532e
gpg-pubkey-3c3359c4-5c6ae44d
gpg-pubkey-c481937a-5bc4662d
gpg-pubkey-54a86092-5bc4663b
gpg-pubkey-d38b4796-570c8cd3
gpg-pubkey-faa19ecc-5cc09115
gpg-pubkey-12c944d0-5d5156ab  <---- here
gpg-pubkey-100bcd92-5ca9b382
gpg-pubkey-6dc1be18-5ca9b41f

The key is already imported.
Note that this is for normal fedora.repo which has 
  repo_gpgcheck=0


Additionally, when I answer N, then the DNF proceed without any error.

For the record, I even tried 

   # rpmkeys --import /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-32-primary

but it makes no difference

Comment 4 Dennis Gilmore 2020-03-18 18:43:12 UTC
the key needs to be imported into dnf's databases, which is seperate from rpm's this is by design

Comment 5 Dusty Mabe 2020-03-20 16:14:41 UTC
I'm booting Fedora 32 beta Cloud base instances (dnf-4.2.19-1.fc32.noarch) for the Fedora Cloud test day today. The dnf-makecache.service that runs 10 minutes after bootup fails because dnf prompts the user [y/N] to import the gpg key:

```
[fedora@ip-10-0-1-54 ~]$ journalctl -u dnf-makecache -o cat 
Starting dnf makecache...
Failed determining last makecache time.
Fedora 32 openh264 (From Cisco) - aarch64       314  B/s | 543  B     00:01
Fedora 32 openh264 (From Cisco) - aarch64       1.6 MB/s | 1.6 kB     00:00
Importing GPG key 0x12C944D0:
 Userid     : "Fedora (32) <fedora-32-primary>"
 Fingerprint: 97A1 AE57 C3A2 372C CA3A 4ABA 6C13 026D 12C9 44D0
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-32-aarch64
Is this ok [y/N]: Fedora 32 openh264 (From Cisco) - aarch64       1.9 kB/s | 543  B     00:00
Error: Failed to download metadata for repo 'fedora-cisco-openh264': repomd.xml GPG signature verification error: Bad GPG signature
Fedora Modular 32 - aarch64                      12 kB/s | 6.5 kB     00:00
Errors during downloading metadata for repository 'fedora-modular':
  - Downloading successful, but checksum doesn't match. Calculated: 2fccbe160a06829db3b1d7445e6a38b75b006f3934cb6f7d440dfb6459f58dce5eaf37bc0275d3e3d1dd80206a2565edc30624ccb9c7d8f090d84e42c1e0b058(sha512) 2fccbe160a06829db3b1d7445e6a38b75b006f3934cb6f7d440dfb6459f58dce5eaf37bc0275d3e3d1dd80206a2>
Error: Failed to download metadata for repo 'fedora-modular': Cannot download repomd.xml: Downloading successful, but checksum doesn't match. Calculated: 2fccbe160a06829db3b1d7445e6a38b75b006f3934cb6f7d440dfb6459f58dce5eaf37bc0275d3e3d1dd80206a2565edc30624ccb9c7d8f090d84e42c1e0b058(sha512) 2fccb>
dnf-makecache.service: Main process exited, code=exited, status=1/FAILURE
dnf-makecache.service: Failed with result 'exit-code'.
Failed to start dnf makecache.
```


I've been told this is happening because of this bug where dnf doesn't properly detect the key that signed the repodata is already imported. Which sounds right because you can see the key it asks us to import is the "Fedora (32) <fedora-32-primary>".

Do you agree that the problem I'm seeing is because of this bug? Can we get this bug fixed?

Comment 6 Fedora Blocker Bugs Application 2020-03-20 16:20:41 UTC
Proposed as a Blocker for 32-final by Fedora user dustymabe using the blocker tracking app because:

 https://fedoraproject.org/wiki/Fedora_32_Final_Release_Criteria#System_services

All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present. 

dnf-makecache.service fails: https://bugzilla.redhat.com/show_bug.cgi?id=1768206#c5

Comment 7 Geoffrey Marr 2020-03-23 19:28:11 UTC
Discussed during the 2020-03-23 blocker review meeting: [0]

The decision to classify this bug as an "AcceptedBlocker" was made as it violates the following criterion:

"All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present" for the Cloud default install.

Note blocker aspect can be resolved by adjusting the service, without fixing the underlying dnf issue

[0] https://meetbot.fedoraproject.org/fedora-blocker-review/2020-03-23/f32-blocker-review.2020-03-23-16.00.txt

Comment 8 Dennis Gilmore 2020-03-26 17:34:56 UTC
(In reply to Dusty Mabe from comment #5)

> 
> I've been told this is happening because of this bug where dnf doesn't
> properly detect the key that signed the repodata is already imported. Which
> sounds right because you can see the key it asks us to import is the "Fedora
> (32) <fedora-32-primary>".
> 
> Do you agree that the problem I'm seeing is because of this bug? Can we get
> this bug fixed?

Dusty,

The issue is not that dnf does not detect that the gpg key is not already imported, the issue is that dnf uses its own databases for gpg keys and because we have enabled for security reasons repo gpgchecking for the cisco repo it does not have the gpg key imported into its own database. a workaround for you make be to run "dnf -y makecache&& dnf clean all" in the kickstart

Comment 9 Kalev Lember 2020-03-26 17:56:41 UTC
Daniel, would it be possible to change dnf so that it uses all the keys that are imported into the rpm database for checking repo_gpgcheck=1 enabled repos? dnf wouldn't have to import the keys into the rpm database itself, just use what's already there (the fedora-cisco-openh264 repo metadata is signed with the main Fedora key).

Comment 10 Dusty Mabe 2020-03-28 15:42:56 UTC
(In reply to Dennis Gilmore from comment #8)

IIUC there was a change in behavior here between F31 and F32 to to enable the fedora-cisco-openh264 repo by default, which happens to have exposed this. I'd argue that the current behavior where the user has not installed anything outside of the system (i.e. they didn't add the fedora-cisco-openh264 repo, the Fedora install did) but is prompted to trust a key that is **already** installed on the system, is not what a user would expect. It also happens to make the dnf makecache service fail.

Some options:

- make dnf learn about the keys already installed on the system
- make dnf makecache not fail somehow
    - probably not good if the answer is having the automated service run dnf -y makecache
- go back to making the cisco repo disabled by default



Note that dnf makecache doesn't always seem to fail (I assume this is because it has `skip_if_unavailable=True`) but I did just verify I do see it today in the latest 20200328.n.0 f32 cloud vagrant box:


```
[vagrant@vanilla-f32 ~]$ sudo journalctl -u dnf-makecache.service -o cat
Starting dnf makecache...
Failed determining last makecache time.
Fedora 32 openh264 (From Cisco) - x86_64        1.3 kB/s | 543  B     00:00
Fedora 32 openh264 (From Cisco) - x86_64        1.6 MB/s | 1.6 kB     00:00
Importing GPG key 0x12C944D0:
 Userid     : "Fedora (32) <fedora-32-primary>"
 Fingerprint: 97A1 AE57 C3A2 372C CA3A 4ABA 6C13 026D 12C9 44D0
 From       : /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-32-x86_64
Is this ok [y/N]: Fedora 32 openh264 (From Cisco) - x86_64        1.5 kB/s | 543  B     00:00
Error: Failed to download metadata for repo 'fedora-cisco-openh264': repomd.xml GPG signature verification error: Bad GPG signature
Fedora Modular 32 - x86_64                      7.8 MB/s | 4.9 MB     00:00
Fedora Modular 32 - x86_64 - Updates            497  B/s | 257  B     00:00
Fedora Modular 32 - x86_64 - Test Updates       889 kB/s | 576 kB     00:00
Fedora 32 - x86_64 - Test Updates               6.0 MB/s |  15 MB     00:02
Fedora 32 - x86_64 - Updates                    592  B/s | 257  B     00:00
Fedora 32 - x86_64                               34 kB/s | 6.1 kB     00:00
Errors during downloading metadata for repository 'fedora':
  - Downloading successful, but checksum doesn't match. Calculated: 6f5de2787e33e8a282cc3c164ffe3e1661e3e33128a9b59b605e63b6085dc80d4ea1e522ef6c36dca4294e55c0ae3c4ebb7700d93e50f9ca61065b080cf9da0b(sha512) 6f5de2787e33e8a282cc3c164ffe3e1661e3e33128a9b59b605e63b6085dc80d4ea1e522ef6c36dca4294e55c0a>
Error: Failed to download metadata for repo 'fedora': Cannot download repomd.xml: Downloading successful, but checksum doesn't match. Calculated: 6f5de2787e33e8a282cc3c164ffe3e1661e3e33128a9b59b605e63b6085dc80d4ea1e522ef6c36dca4294e55c0ae3c4ebb7700d93e50f9ca61065b080cf9da0b(sha512) 6f5de2787e33e>
dnf-makecache.service: Main process exited, code=exited, status=1/FAILURE
dnf-makecache.service: Failed with result 'exit-code'.
Failed to start dnf makecache.
dnf-makecache.service: Consumed 8.468s CPU time.
```

Comment 11 Zbigniew Jędrzejewski-Szmek 2020-03-29 20:29:55 UTC
Asking again for a key is certainly contributing to security-question-fatigue.
I'd vote for option 1 (make dnf learn about the keys already installed on the system)
if at all possible.

Certainly +1 FE. A blocker if making dnf-makecache.service is covered by some blocker rule.

Comment 12 Dusty Mabe 2020-03-29 20:46:35 UTC
(In reply to Zbigniew Jędrzejewski-Szmek from comment #11)
> Certainly +1 FE. A blocker if making dnf-makecache.service is covered by
> some blocker rule.

I cited a release criteria that I think this violates in https://bugzilla.redhat.com/show_bug.cgi?id=1768206#c6

Comment 13 Miroslav Suchý 2020-03-30 07:15:53 UTC
> - go back to making the cisco repo disabled by default

This is not an option. As I shown in comment #3 this cause problem during upgrade from Fedora n to n+1 too.

Comment 14 Daniel Mach 2020-03-30 07:36:40 UTC
I don't believe it's a new bug or a regression. Could someone confirm that?
If it's not a regression, do you really want us to fix this bug by implementing an RFE?

1) it will take some time, I cannot guarantee that we'll deliver a working fix in time
   (it can be an easy fix or it can turn in a big chunk of work; gpg handling in dnf definitely needs to be reworked, but I was hoping we'll do it during regular development, not by fixing a blocker)
2) it may break more things that it fixes

Comment 15 Miroslav Suchý 2020-03-30 08:09:33 UTC
@dmach is there some workaround? Can I tell DNF to import that key?

Comment 16 Dusty Mabe 2020-03-30 12:49:31 UTC
(In reply to Daniel Mach from comment #14)
> I don't believe it's a new bug or a regression. Could someone confirm that?

Daniel, you are correct that the behavior is most likely not a regression in DNF itself. When I was referring to regressing behavior I was referring to the distro as a whole. Something changed to introduce this new behavior that the user now sees. The regression is from the user perspective and not necessarily in dnf itself. We must all work together to figure out the best short term and long term solution to the problem.

Comment 17 Zbigniew Jędrzejewski-Szmek 2020-03-30 16:42:46 UTC
Adding some Workstation SIG members to CC (sorry, not all, bugzilla doesn't make this easy).

Comment 18 Adam Williamson 2020-03-30 17:26:47 UTC
To be clear:

* This is already an accepted blocker. It needs to be addressed, somehow, for release. We do not need more blocker votes.

* As Dusty says, from dnf perspective this is not a 'new' bug, but from the distribution perspective it is, because before Fedora 32 we did not have a default-enabled repository with "repo_gpgcheck=1", but now we do.

Comment 19 Chris Murphy 2020-03-30 17:39:34 UTC
a) assess the scope, risk, and time frame for a dnf fix;
b) revert 'enable the fedora-cisco-openh264 repo by default'.

Is there another option? b) would be really unfortunate, but is straightforward, and no worse off than we were in F31.

Comment 20 Adam Williamson 2020-03-30 17:58:54 UTC
Well, theoretically there's "c) change cisco repo to repo_gpgcheck=0", but AIUI that's impractical as it's the only way we have to ensure the integrity of that repo :/

Comment 21 Kalev Lember 2020-03-30 18:14:41 UTC
d) switch to mirrormanager for the cisco repo metadata, which would allow us to change repo_gpgcheck to 0

I'd prefer either option (a) or (d).

Comment 22 Adrian Reber 2020-04-06 18:42:45 UTC
(In reply to Kalev Lember from comment #21)
> d) switch to mirrormanager for the cisco repo metadata, which would allow us
> to change repo_gpgcheck to 0

We discussed this on IRC a bit and I think it is possible for MirrorManager to handle this if we can get access to the directory structure and repodata/repomd.xml and a trigger when we would need to check for a new repomd.xml file. It should be doable with what MirrorManager currently can do.

Comment 23 Ben Cotton 2020-04-08 12:10:16 UTC
I discussed this with the DNF team in their meeting today. They agree that the MirrorManager approach is the best option. Adrian, where do we stand on this?

Comment 24 Adrian Reber 2020-04-08 12:58:16 UTC
Someone (I guess from releng) needs to rsync the directory tree to mm-backend01 in Fedora infrastructure. Once that is synced I can start to look at configuring MirrorManager to handle this.

The goal would be to only provide a metalink for this repository. All other functionality of MirrorManager would not be available. For MirrorManager to be able to create the metalink only the directory structure and the repomd.xml files are necessary. RPMs do not need to be synced to mm-backend01.

The good thing is that this can all be set up in parallel to the existing structure so we do not need to do the switch on one day. We can do the change, if we want, for each release individually.

Looking at the current setup most of it could be reused.

We would create the repositories in MirrorManager. The client would then connect to mirrors.fedoraproject.org to download the metalink for fedora-cisco-openh264-31.
Maybe something like this: https://mirrors.fedoraproject.org/metalink?repo=fedora-cisco-openh264-$releasever&arch=$basearch

This would return a metalink pointing to codecs.fedoraproject.org (which is served by all Fedora proxies as far as I understood it), from there the client downloads the repomd.xml and can verify the checksum against the metalink and then everything would continue as it works right now.

Comment 25 Ben Cotton 2020-04-08 13:08:18 UTC
Fedora Release Engineering ticket opened: https://pagure.io/releng/issue/9391

Comment 26 Kalev Lember 2020-04-08 13:19:08 UTC
Another instance of this is https://github.com/fedora-silverblue/issue-tracker/issues/36#issuecomment-610927905
 where the libdnf repo_gpgcheck=1 issues are breaking gnome-software on Silverblue.

Switching to mirrormanager should nicely work this around as well so +1 from me to go down that route.

Comment 27 Kalev Lember 2020-04-09 12:39:33 UTC
fedora-repos pull requests to switch to metalink:

https://src.fedoraproject.org/rpms/fedora-repos/pull-request/57 (rawhide)
https://src.fedoraproject.org/rpms/fedora-repos/pull-request/58 (f32)

Comment 28 Jerry James 2020-04-09 21:12:13 UTC
Is the change to fedora-cisco-openh264 going to make bug 1350930 affect all Fedora packagers, or does that bug have the same underlying cause as this one?

Comment 29 Adam Williamson 2020-04-09 21:35:01 UTC
So, the fedora-repos change doesn't really fix the underlying dnf bug here: there's still a bug for any repos which do have repo_gpgcheck=1 . We just fixed things so we don't need repo_gpgcheck=1 for the cisco repo any more.

So the update shouldn't close this bug. But it *does* resolve the release blocker-y problem here, because we once again have no default repos with repo_gpgcheck=1, so this goes back to being an obscure bug only a few people run into.

So, mohan and I agreed the 'AcceptedBlocker' status should be transferred to https://bugzilla.redhat.com/show_bug.cgi?id=1814671 , which is closer to being a report of the release blocker issue as opposed to the underlying DNF bug. We'll mark the update as fixing that bug, and leave this one open.

Comment 30 Dusty Mabe 2020-04-10 02:32:26 UTC
+1

Comment 31 Neal Gompa 2020-11-03 00:44:33 UTC
There is currently no fix for this, and the underlying issue still persists.

Comment 32 Demi Marie Obenour 2021-05-14 03:07:29 UTC
Potential workaround: run `rpm -qi gpg-pubkey|gpg --import <keyring>`

Comment 33 Brian J. Murrell 2021-11-19 12:02:44 UTC
(In reply to Neal Gompa from comment #31)
> There is currently no fix for this, and the underlying issue still persists.

Indeed.  Is there any plan to fix the actual underlying issue?

Comment 34 Daniel Farina 2022-07-06 00:23:03 UTC
I'd like to re-iterate that it's important to have a fix for this. I itemize all the public keys used by RPMs I rely on and save them: any signing key rotations I want to inspect for myself. This works as expected for rpms, but for the repository signatures themselves, I don't see an equivalent way to import specific keys.  My apparently-best non-interactive option is to accept all keys without question at least once. That's fine for my laptop, it's less fine for servers.

Between non-interactively accepting arbitrary keys even once, and only being able to check rpms with keys I have itemized, I opt to turn off repository metadata checking. Seems a shame.

Comment 35 Irwin.Sobel 2022-09-29 16:50:10 UTC
On Fedora 36:
# uname -a 
Linux isobel1 5.18.18-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Aug 17 16:02:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
# dnf update
Fedora 36 openh264 (From Cisco) - x86_64                                 345  B/s | 566  B     00:01    
Fedora 36 openh264 (From Cisco) - x86_64                                 1.6 MB/s | 1.6 kB     00:00    
Fedora 36 openh264 (From Cisco) - x86_64                                 851  B/s | 566  B     00:00    
Error: Failed to download metadata for repo 'fedora-cisco-openh264': repomd.xml GPG signature verification error: Bad GPG signature

Similarly trying to install a downloaded Zoom update:
# dnf install zoom_x86_64.rpm
Fedora 36 openh264 (From Cisco) - x86_64                                 312  B/s | 566  B     00:01    
Fedora 36 openh264 (From Cisco) - x86_64                                 1.6 MB/s | 1.6 kB     00:00    
Fedora 36 openh264 (From Cisco) - x86_64                                 824  B/s | 566  B     00:00    
Error: Failed to download metadata for repo 'fedora-cisco-openh264': repomd.xml GPG signature verification error: Bad GPG signature

Comment 36 Allen S. Rout 2022-10-10 19:19:44 UTC
RHEL customer weighing in on this.

I'm currently in the slightly ridiculous state of, first importing an RPM key, and then running this bit of foolishnesss:

```
        exec {'/usr/bin/dnf -y module list':
          subscribe => Exec['/usr/bin/rpm --import  /etc/pki/rpm-gpg/RPM-GPG-KEY-remi.el8'],
          refreshonly => true,
          require => File['/etc/pki/rpm-gpg/RPM-GPG-KEY-remi.el8']
        }

```

so that yum accepts the key for the repo interactions.

Then I can go on and assert my package requirements. 

This is just silly.   If dnf has decided it's got good reasons to squirrel away its own key database, I've got no problem with that.  But we need the same entry points that we've got for the RPM key database:  list keys, add key, delete key.    This workaround is embarrasing.

Comment 37 Edward Valley 2023-04-12 03:29:38 UTC
As some users have pointed out, DNF and RPM keep separate databases to store trusted GPG keys. By reading DNF configuration reference, we know that in DNF's main configuration file (/etc/dnf/dnf.conf) and repository configuration files (/etc/yum.repos.d/*.repo), we can use:

gpgcheck=1 to make RPM perform GPG signature check on packages (RPM files)
repo_gpgcheck=1 to make DNF perform GPG signature check on repository metadata (repomd.xml)

So, in order to automate any task involving package installation or repository mirroring (using dnf reposync), while avoiding to pass --assumeyes to dnf commands, we need to manually import corresponding GPG keys to each database.

RPM keeps its database at /var/lib/rpm by default, but if we need to use one in any other location, we can initialize it with the following command:

rpmdb --dbpath "/path/to/rpm/database" --initdb

Then, as some users have also stated, we need to import GPG keys into the RPM database by using:

rpmkeys --dbpath "/path/to/rpm/database" --import "/path/to/gpg/key"

Note that if we are using dnf reposync for repository mirroring, and not passing --gpgcheck to it, we don't need the steps just above. If we indeed are passing --gpgcheck to dnf reposync, then in order to avoid mirrored packages to be deleted, we must import GPG keys to the RPM database being used, which we probably don't want to be the default one.

Now, we need to import GPG keys to the database used by DNF, and this is maybe the harder part, just because the path to it, have a part that is a hash we need to compute.

DNF keeps a metadata cache directory for each configured and enabled repository, which by default, is under /var/cache/dnf. This directory is named {repo-id}-{url-hash}, where {repo-id} is the repository identifier reference (the one between square brackets in the .repo file), and {url-hash} match the first 16 hexadecimal lowercase characters of the SHA256 hash of the repository URL (the metalink/mirrorlist/baseurl in the .repo file) after variables (basearch, releasever, etc...) are substituted.

We can find the hash related code here:
https://github.com/rpm-software-management/libdnf/blob/7cdc7a387b1ed47850455761699c44cec62c9185/libdnf/repo/Repo.cpp#L1376

For example, if we have the following repository configuration file:

[epel]
name=Extra Packages for EL$releasever
baseurl=http://mirror.example.com/epel/$releasever/Everything/$basearch
gpgkey=file:///etc/pki/rpm-gpg/21EA45AB2F86D6A1.gpg
gpgcheck=1
enabled=1

Assuming basearch=x86_64 and releasever=8, we can compute the {url-hash} part by using this bash shell command:

echo -n "http://mirror.example.com/epel/8/Everything/x86_64" | sha256sum | head -c 16

Then, the repository metadata cache directory would be:

/var/cache/dnf/epel-76210325254b5510

Under this directory, for a repository without repo_gpgcheck=1, we can find two directories where dnf keeps cached metadata, named "repodata" and "packages"

For a repository with repo_gpgcheck=1, once we have accepted the GPG keys used to verify repository metadata, we can find an additional directory named "pubring". That directory, is a GPG keybox database where such keys are stored.

Now, in order our automated package installation or repository mirroring to securely succeed, without DNF prompting us to confirm we trust GPG keys used to verify repository metadata, and without passing --assumeyes as an argument to dnf, we need to create that GPG keybox beforehand and import the keys into it.

Taking the repository configuration file from the previous example, we can use the following bash shell snippet (that have NO error checking code) for this purpose:

repo_id="epel"
basearch="x86_64"
releasever="8"
baseurl="http://mirror.example.com/epel/$releasever/Everything/$basearch"
url_hash="$(echo -n "${baseurl}" | sha256sum | head -c 16)"
pubring_path="/var/cache/dnf/${repo_id}-${url_hash}/pubring"
mkdir -p "${pubring_path}"
chmod 700 "${pubring_path}"
gpg --homedir "${pubring_path}" --import "/path/to/gpg/key/file"

Again, if we are using dnf reposync for repository mirroring, we probably don't want to use the default DNF cache directory. See the installroot dnf configuration option and the --installroot dnf command line parameter.

Of course, if we are not running dnf using root (it's unnecessary and more secure for repository mirroring), default paths slightly changes.

In my personal opinion, a straightforward solution to this, is a missing part of DNF. Maybe DNF should automatically trust/import keys that the operating system provider or system administrator has decided to put in the repository configuration files, maybe subject to file permission restrictions or the method used to access a remote key.

I hope this be of some help for anyone facing issues with the subject.

Thanks


Note You need to log in before you can comment on or make changes to this bug.