Bug 2270397

A stable rpm version doesn't suddenly forget where to look for its own signatures. If rpm says there is no signature then there isn't one. 

The corresponding packages in the rawhide repositories *are* signed so, it means the rawhide container images are being built from packages prior to signing.

I believe I've got a fix submitted to Koji for this: https://pagure.io/koji/pull-request/4061

Proposing as a Beta FE to decide if we think this is important enough to fix for Beta (otherwise it would just get fixed with post-Beta nightlies which will be published to the registries).

+3 in https://pagure.io/fedora-qa/blocker-review/issue/1538 , marking accepted FE.

I've updated koji with the patch so hopefully this will be fixed in rc10.

ok, this was not actually fixed in rc10, but... is now fixed in both branched 40 and rawhide. ;) 

If anyone could doublecheck that would be great.

Pulling fresh images from the registry, I see

$ podman run --rm -ti registry.fedoraproject.org/fedora:40 rpm -qi rpm | grep Signature
Signature   : RSA/SHA256, Wed Feb  7 16:03:09 2024, Key ID 0727707ea15b79cc
$ podman run --rm -ti registry.fedoraproject.org/fedora:rawhide rpm -qi rpm | grep Signature
Signature   : RSA/SHA256, Sun Feb 11 08:19:17 2024, Key ID d0622462e99d6ad1

So I'm happy.

Let's call it fixed, then, there is no update to push here, we fixed it in the koji deployment.

For the record, the problem is back on registry.fedoraproject.org/fedora:40:

$ podman pull registry.fedoraproject.org/fedora:40
Trying to pull registry.fedoraproject.org/fedora:40...
Getting image source signatures
Copying blob c6405a39eed1 skipped: already exists  
Copying config 71a9aee84c done   | 
Writing manifest to image destination
71a9aee84c30bdd96c84be43693f6d74ba56f1d71c278f383ef1524f0ec86903
$ podman run --rm registry.fedoraproject.org/fedora:40 rpm -qi rpm | grep Signature
Signature   : (none)

Reopening the bug and proposing for a Final freeze exception.

I think our patches on Koji got dropped accidentally somehow, because not only is this back, but the verbose logging is missing from the kiwi image builds again.

Are we sure the https://pagure.io/koji/pull-request/4061 mentioned in comment 2 ever got merged? It is still listed as open and the https://pagure.io/koji/history/plugins/builder/kiwi.py?identifier=master does not show any change in the past year.

Do we need some tests added to the pipeline first, to make sure the result is sanity-checked and pipeline stopped with a loud bang?

It was cherry-picked into an infra koji build, but because they weren't committed to the koji package, I'm guessing it got overwritten and dropped accidentally.

New ticket: https://pagure.io/fedora-qa/blocker-review/issue/1573

if the change goes into an f39 infra build, it doesn't really need an f40 final freeze exception (both because it's f39 not f40, and because infra builds aren't subject to the freeze for 'regular' builds).

Discussed during the 2024-04-08 blocker review meeting: [1]

The decision to classify this bug as a RejectedFreezeException (Final) was made:

"This is clearly an important problem and we want to fix it, but an F40 Final freeze exception is no use in doing that, as the fix needs to be applied to the builders, which run Fedora 39. What we need is an infrastructure freeze break request, nirik has proposed one on the mailing list, we need releng/sysadmin-main members to vote on that at https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org/thread/XFJATU5DDOUDLF7DCBY6JG5CNLKQ4MAT/ ."

[1] https://meetbot.fedoraproject.org/blocker-review_matrix_fedoraproject-org/2024-04-08/f40-blocker-review.2024-04-08-16.00.html

The infrastructure fix is in, so hopefully tonights nightlys will be fixed.

The images were fixed for a while:

$ podman images registry.fedoraproject.org/fedora:40
REPOSITORY                         TAG         IMAGE ID      CREATED       SIZE
registry.fedoraproject.org/fedora  40          f9754cf18f83  7 days ago  229 MB
$ podman run --rm registry.fedoraproject.org/fedora:40 rpm -qi rpm | grep Signature
Signature   : RSA/SHA256, Wed Feb  7 16:03:09 2024, Key ID 0727707ea15b79cc

And now they are broken again:

$ podman images registry.fedoraproject.org/fedora:40
REPOSITORY                         TAG         IMAGE ID      CREATED       SIZE
registry.fedoraproject.org/fedora  40          895a2e2fc547  18 hours ago  227 MB
$ podman run --rm registry.fedoraproject.org/fedora:40 rpm -qi rpm | grep Signature
Signature   : (none)

Yeah, I applied a updated upstream patch and... it's not defaulting to what I expect. 

Asking about it upstream and will try and get it fixed asap.

% podman images registry.fedoraproject.org/fedora:40
REPOSITORY                         TAG         IMAGE ID      CREATED      SIZE
registry.fedoraproject.org/fedora  40          868a0de6755b  8 hours ago  222 MB
% podman run --rm registry.fedoraproject.org/fedora:40 rpm -qi rpm | grep Signature
Signature   : RSA/SHA256, Wed Feb  7 16:03:09 2024, Key ID 0727707ea15b79cc

Please confirm. :)

Confirming that on both

REPOSITORY                         TAG         IMAGE ID      CREATED       SIZE
registry.fedoraproject.org/fedora  rawhide     e4352707d741  27 hours ago  229 MB
registry.fedoraproject.org/fedora  40          868a0de6755b  3 days ago    229 MB

I currently see the signature. Thank you.

The issue is back, now on registry.fedoraproject.org/fedora:40 and registry.fedoraproject.org/fedora:41: bug 2308287.

This bug appears to have been reported against 'rawhide' during the Fedora Linux 42 development cycle.
Changing version to 42.