Bug 1633482

This is not limited to image signatures, it also means that with the default config skopeo and podman does not verify the CA issuer of the remote registry.
This can easily be demonstrated by installing docker-distribution and configure it to use a self signed certificate.

(In reply to Johan Swensson from comment #2)
> This is not limited to image signatures, it also means that with the default
> config skopeo and podman does not verify the CA issuer of the remote
> registry.
> This can easily be demonstrated by installing docker-distribution and
> configure it to use a self signed certificate.

This only seems to be the case if podman/skopeo is run on the same machine as docker-distribution. Which is strange since there are no configured insecure registries by default, but there are in docker.

# podman info
<...>
insecure registries:
  registries: []
<...>

# docker info
<...>
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Registries: registry.access.redhat.com (secure), docker.io (secure)

Lokesh, do you know if RHEL registries are signed now?

(In reply to Johan Swensson from comment #3)
[...]
> This only seems to be the case if podman/skopeo is run on the same machine
> as docker-distribution. Which is strange since there are no configured
> insecure registries by default, but there are in docker.
> 
> # podman info
> <...>
> insecure registries:
>   registries: []
> <...>
> 
> # docker info
> <...>
> Insecure Registries:
>  127.0.0.0/8
> Live Restore Enabled: false
> Registries: registry.access.redhat.com (secure), docker.io (secure)

We are currently working on marking localhost to be insecure by default. In the meantime, we need to set this manually in the `/etc/containers/registries.conf` by adding the specific registry to the insecure list.

(In reply to Johan Swensson from comment #2)
> This is not limited to image signatures, it also means that with the default
> config skopeo and podman does not verify the CA issuer of the remote
> registry.

I can quite authoritatively say that nothing in policy.json has any connection with TLS or CA verification at all.  That’s not to say that there can’t be a problem with CA verification, but…

> This can easily be demonstrated by installing docker-distribution and
> configure it to use a self signed certificate.

… please provide detailed and comprehensive steps to reproduce, how an untrusted CA is accepted (and how it is not accepted with different policy.json).

(In reply to Miloslav Trmač from comment #6)
> … please provide detailed and comprehensive steps to reproduce, how an
> untrusted CA is accepted (and how it is not accepted with different
> policy.json).

I have not been able to reproduce it my self either, sorry about the noise.

To be more specific about what is required:

$ cat /etc/containers/policy.json
{
    "default": [
        {
            "type":    "signedBy",
            "keyType": "GPGKeys",
            "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
            "signedIdentity": {"type":"matchExact"}
        }
    ]
}

$ cat /etc/containers/registries.d/redhat.yaml
docker:
    registry.access.redhat.com:
        sigstore: https://access.redhat.com/webassets/docker/content/sigstore

With this configuration container image pulled from registry.access.redhat.com will be verified. For images mirrored on registry.redhat.io Red Hat only started generating valid signatures for them from April 1st, 2019. Until Red Hat publish signatures for images distributed on registry.redhat.io prior to April 1st, 2019 signature validation on images pulled from there won't work.

If you expect to only be pulling images distributed after that date, you can modify registries.d/redhat.yaml like so:

$ cat /etc/containers/registries.d/redhat.yaml
docker:
    registry.access.redhat.com:
        sigstore: https://access.redhat.com/webassets/docker/content/sigstore
    registry.redhat.io:
        sigstore: https://access.redhat.com/webassets/docker/content/sigstore

With this enabled trying to pull an unsigned image from another registry fails with an error such as:

podman pull docker.io/myrepo/depcheck:0.1
Trying to pull docker.io/myrepo/depcheck:0.1...Failed
error pulling image "docker.io/myrepo/depcheck:0.1": unable to pull docker.io/myrepo/depcheck:0.1: 2 errors occurred:

* Source image rejected: A signature was required, but no signature exists 

To workaround this issue, we can add an insecure policy for docker.io. That would cover everything in the default search path from registries.conf.

$ cat /etc/containers/policy.json 
{
    "default": [
        {
            "type":    "signedBy",
            "keyType": "GPGKeys",
            "keyPath": "/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release",
            "signedIdentity": {"type":"matchExact"}
        }
    ],
    "transports": {
                      "docker": {
                          "docker.io": [{"type": "insecureAcceptAnything"}]
                      }
    }
}

(In reply to Jason Shepherd from comment #17)
> With this enabled trying to pull an unsigned image from another registry
> fails with an error such as:
> 
> podman pull docker.io/myrepo/depcheck:0.1
> Trying to pull docker.io/myrepo/depcheck:0.1...Failed
> error pulling image "docker.io/myrepo/depcheck:0.1": unable to pull
> docker.io/myrepo/depcheck:0.1: 2 errors occurred:
> 
> * Source image rejected: A signature was required, but no signature exists 
> 
> To workaround this issue, we can add an insecure policy for docker.io. That
> would cover everything in the default search path from registries.conf.

No, it wouldn’t.  What about my-private-registry.my-company.example.com?  I can’t see why we would treat that registry and docker.io differently in the default configuration.

The RH keys just have no place being in the global default scope; define them specifically for "transports"."docker"."registry.access.redhat.com" (and "transports"."docker"."redhat.io", eventually); leave the insecure "transports"."docker-daemon" configuration as is.  And then, independently, decide what should be the global default. In order not to break customers’ systems, I think the shipped default almost certainly has to be insecureAcceptAnything as well, but we should prominently document how to set default: reject for customers who want that kind of lock down.

See this KB article for details on turning on image signature verification for Red Hat repositories in RHEL-7:

https://access.redhat.com/articles/3116561

Do we know if all images from Red Hat are signed?  Are we prepared to turn this on by default?

After evaluating this issue, there are no plans to address it further or fix it in an upcoming release.  Therefore, it is being closed.  If plans change such that this issue will be fixed in an upcoming release, then the bug can be reopened.