Bug 1393545 - include F25 Primary GPG Key in /usr/share/ostree/trusted.gpg.d/ for Atomic Host
Summary: include F25 Primary GPG Key in /usr/share/ostree/trusted.gpg.d/ for Atomic Host
Keywords:
Status: CLOSED EOL
Alias: None
Product: Fedora
Classification: Fedora
Component: fedora-repos
Version: 28
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: Dennis Gilmore
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-11-09 19:32 UTC by Micah Abbott
Modified: 2019-05-28 19:26 UTC (History)
9 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-05-28 19:26:23 UTC
Type: Bug


Attachments (Terms of Use)

Description Micah Abbott 2016-11-09 19:32:54 UTC
The F25 ostree composes are signed with the F25 Primary GPG Key, but that key is not installed in /usr/share/ostree/trusted.gpg.d/ where it can be used by 'ostree show'.

# rpm-ostree status
State: idle
Deployments:
● fedora-atomic:fedora-atomic/25/x86_64/docker-host
       Version: 25.40 (2016-11-08 10:20:02)
        Commit: b02be04acde2a04a96d9e969edf3b73753967c06811fc51d65c2b2b4985dff4a
        OSName: fedora-atomic
# ostree show b02be04acde2a04a96d9e969edf3b73753967c06811fc51d65c2b2b4985dff4a
commit b02be04acde2a04a96d9e969edf3b73753967c06811fc51d65c2b2b4985dff4a
Date:  2016-11-08 10:20:02 +0000
Version: 25.40
(no subject)
 
Found 1 signature:
 
  Signature made Tue 08 Nov 2016 10:20:33 AM UTC using RSA key ID 4089D8F2FDB19C98
  Can't check signature: public key not found
# ls /usr/share/ostree/trusted.gpg.d/
README-gpg


Compared to CentOS Atomic Host:

# rpm-ostree status
State: idle
Deployments:
● centos-atomic-host:centos-atomic-host/7/x86_64/standard
       Version: 7.20161104 (2016-11-04 22:53:08)
        Commit: daf53c718bd1cd9b84b1b5d21843cd9b981d3634c2444c2bcb09cccee771058b
        OSName: centos-atomic-host
  GPGSignature: 1 signature
                Signature made Fri 04 Nov 2016 10:56:08 PM UTC using RSA key ID F17E745691BA8335
                Good signature from "CentOS Atomic SIG <security@centos.org>"
# ostree show daf53c718bd1cd9b84b1b5d21843cd9b981d3634c2444c2bcb09cccee771058b
commit daf53c718bd1cd9b84b1b5d21843cd9b981d3634c2444c2bcb09cccee771058b
Date:  2016-11-04 22:53:08 +0000
Version: 7.20161104
(no subject)

Found 1 signature:

  Signature made Fri 04 Nov 2016 10:56:08 PM UTC using RSA key ID F17E745691BA8335
  Good signature from "CentOS Atomic SIG <security@centos.org>"
# ls /usr/share/ostree/trusted.gpg.d/
pubring.gpg  pubring.gpg~  README-gpg  secring.gpg  trustdb.gpg
# ostree admin unlock
Development mode enabled.  A writable overlayfs is now mounted on /usr.
All changes there will be discarded on reboot.
# gpg --homedir /usr/share/ostree/trusted.gpg.d/ -k
gpg: WARNING: unsafe permissions on homedir `/usr/share/ostree/trusted.gpg.d/'
/usr/share/ostree/trusted.gpg.d//pubring.gpg
--------------------------------------------
pub   2048R/91BA8335 2015-06-10
uid                  CentOS Atomic SIG (http://wiki.centos.org/SpecialInterestGroup/Atomic) <security@centos.org>



The CentOS spec file looks pretty straight-forward:

https://git.centos.org/blob/rpms!centos-release-atomic/HEAD/SPECS!centos-release-atomic.spec

Comment 1 Dennis Gilmore 2016-11-10 14:04:39 UTC
fedora-repos has all of the gpg keys ever used in Fedora. 

how does the verification work?
can a commit be verified before a update is pulled down?
do we need to have all possible keys available? or only the one for the commit.

Comment 2 Micah Abbott 2016-11-10 14:38:00 UTC
From 'man ostree':

GPG VERIFICATION
       OSTree supports signing commits with GPG. Operations on the system repository by default use keyring files in /usr/share/ostree/trusted.gpg.d. Any public key in a keyring file in that directory will be trusted by the client. No private keys should be present in this directory.

       In addition to the system repository, OSTree supports a per-remote remotename.trustedkeys.gpg file stored in the toplevel of the repository (alongside objects/ and such). This is particularly useful when downloading content that may not be fully trusted (e.g. you want to inspect it but not deploy it as an OS), or use it for containers. This file is written via ostree remote add --gpg-import.

> can a commit be verified before a update is pulled down?

You can pull the commit metadata, which includes the signature, and verify that commit before pulling all of the file objects.  

In the example below, I have pulled down all the commit history for the branch, but not the file objects.  Then I use 'ostree show' to verify the signature of the HEAD-1 commit.

# rpm-ostree status
State: idle
Deployments:
● fedora-atomic:fedora-atomic/25/x86_64/docker-host
       Version: 25.40 (2016-11-08 10:20:02)
        Commit: b02be04acde2a04a96d9e969edf3b73753967c06811fc51d65c2b2b4985dff4a
        OSName: fedora-atomic
# ostree pull --commit-metadata-only --depth=-1 fedora-atomic:fedora-atomic/25/x86_64/docker-host

80 metadata, 0 content objects fetched; 14 KiB transferred in 31 seconds
# ostree log fedora-atomic/25/x86_64/docker-host
commit b02be04acde2a04a96d9e969edf3b73753967c06811fc51d65c2b2b4985dff4a
Date:  2016-11-08 10:20:02 +0000
Version: 25.40
(no subject)

commit c8379fb28b7102367d1f4dbdb8f8082628d098aba8969f4c98ee7677b068a2c7
Date:  2016-11-06 10:22:13 +0000
Version: 25.39
(no subject)

commit 7f19ed0a253b9023dd1d41acd9043deadb08a70d379f424b14fff61703f47827
Date:  2016-11-01 10:26:04 +0000
Version: 25.38
(no subject)

...
# ostree show c8379fb28b7102367d1f4dbdb8f8082628d098aba8969f4c98ee7677b068a2c7
commit c8379fb28b7102367d1f4dbdb8f8082628d098aba8969f4c98ee7677b068a2c7
Date:  2016-11-06 10:22:13 +0000
Version: 25.39
(no subject)

Found 1 signature:

  Signature made Sun 06 Nov 2016 10:22:22 AM UTC using RSA key ID 4089D8F2FDB19C98
  Can't check signature: public key not found

> do we need to have all possible keys available? or only the one for the
> commit.

We would only need a keyring that has the GPG key used to sign the commit.

Comment 3 Colin Walters 2016-11-10 14:45:14 UTC
See https://github.com/projectatomic/centos-release-atomic-host-devel/blob/master/centos-devel-atomic-host-release.spec#L88

The problem is right now doing that would require fedora-repos to depend on ostree.  (Although just for file path ownership, which honestly, I'm not sure if we really care right now if someone removes fedora-repos and /usr/share/ostree/trusted.gpg.d stays around).

Comment 4 Stephen Gallagher 2016-11-10 14:47:00 UTC
(In reply to Colin Walters from comment #3)
> See
> https://github.com/projectatomic/centos-release-atomic-host-devel/blob/
> master/centos-devel-atomic-host-release.spec#L88
> 
> The problem is right now doing that would require fedora-repos to depend on
> ostree.  (Although just for file path ownership, which honestly, I'm not
> sure if we really care right now if someone removes fedora-repos and
> /usr/share/ostree/trusted.gpg.d stays around).

Is there a reason that couldn't just be moved into the `filesystem` package?

Comment 5 Colin Walters 2016-11-10 14:52:21 UTC
A root problem is the split between the fedora-release packages and fedora-repos only works for rpms.  If we wanted default Docker registries or ostree remotes, there's no mechanism to have those only apply to one product.

Comment 6 Stephen Gallagher 2016-11-10 15:00:42 UTC
(In reply to Colin Walters from comment #5)
> A root problem is the split between the fedora-release packages and
> fedora-repos only works for rpms.  If we wanted default Docker registries or
> ostree remotes, there's no mechanism to have those only apply to one product.

Sure there is. Each of the Editions has their own fedora-release-$EDITION subpackage that we could add a Requires: to.

Comment 7 Colin Walters 2016-11-10 15:02:32 UTC
Here's a patch I started on which should illustrate what I'm talking about; there may be a better approach, but:

--- a/fedora-release.spec
+++ b/fedora-release.spec
@@ -28,6 +28,7 @@ Obsoletes:      fedora-release-standard < 22-0.8
 
 
 Requires:       fedora-repos(%{version})
+BuildRequires:  fedora-repos(%{version})
 BuildArch:      noarch
 
 %description
@@ -43,6 +44,7 @@ Requires:       fedora-release = %{version}-%{release}
 %description atomichost
 Provides a base package for Fedora Atomic Host-specific configuration files to
 depend on.
+Requires:       ostree
 
 %package cloud
 Summary:        Base package for Fedora Cloud-specific default configurations
@@ -218,6 +220,13 @@ install -m 0644 org.gnome.shell.gschema.override $RPM_BUILD_ROOT%{_datadir}/glib
 mkdir -p $RPM_BUILD_ROOT/%{_prefix}/sbin/
 install -m 0755 convert-to-edition $RPM_BUILD_ROOT/%{_prefix}/sbin/
 
+# This should likely go into fedora-repos, though that'd be more awkward
+# as it'd require a dependency on ostree there, or a subpackage
+mkdir -p -m 755 %{buildroot}/%{_datadir}/ostree/trusted.gpg.d
+for file in /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-%{dist_version}-primary; do
+    gpg --dearmor < $file > %{buildroot}/%{_datadir}/ostree/trusted.gpg.d/$(basename $file).gpg
+done
+
 %post -p <lua>
 %include %{_sourcedir}/convert-to-edition.lua
 -- On initial installation, we'll at least temporarily put the non-product
@@ -316,6 +325,7 @@ glib-compile-schemas %{_datadir}/glib-2.0/schemas &> /dev/null || :
 %{!?_licensedir:%global license %%doc}
 %license LICENSE
 %config %attr(0644,root,root) /usr/lib/os.release.d/os-release-atomichost
+%{_datadir}/ostree/trusted.gpg.d/*
 
 
 %files cloud

Comment 8 Dennis Gilmore 2016-11-10 19:18:35 UTC
(In reply to Colin Walters from comment #7)
> Here's a patch I started on which should illustrate what I'm talking about;
> there may be a better approach, but:
> 
> --- a/fedora-release.spec
> +++ b/fedora-release.spec
> @@ -28,6 +28,7 @@ Obsoletes:      fedora-release-standard < 22-0.8
>  
>  
>  Requires:       fedora-repos(%{version})
> +BuildRequires:  fedora-repos(%{version})
This is a non starter, you have a chicken and egg problem fedora-repos and fedora-release both Require each other and require that they be the same version, it  could be bootstrapped though 

>  BuildArch:      noarch
>  
>  %description
> @@ -43,6 +44,7 @@ Requires:       fedora-release = %{version}-%{release}
>  %description atomichost
>  Provides a base package for Fedora Atomic Host-specific configuration files
> to
>  depend on.
> +Requires:       ostree
>  
>  %package cloud
>  Summary:        Base package for Fedora Cloud-specific default
> configurations
> @@ -218,6 +220,13 @@ install -m 0644 org.gnome.shell.gschema.override
> $RPM_BUILD_ROOT%{_datadir}/glib
>  mkdir -p $RPM_BUILD_ROOT/%{_prefix}/sbin/
>  install -m 0755 convert-to-edition $RPM_BUILD_ROOT/%{_prefix}/sbin/
>  
> +# This should likely go into fedora-repos, though that'd be more awkward
> +# as it'd require a dependency on ostree there, or a subpackage
> +mkdir -p -m 755 %{buildroot}/%{_datadir}/ostree/trusted.gpg.d
> +for file in /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-%{dist_version}-primary; do
> +    gpg --dearmor < $file >
> %{buildroot}/%{_datadir}/ostree/trusted.gpg.d/$(basename $file).gpg
> +done
> +
>  %post -p <lua>
>  %include %{_sourcedir}/convert-to-edition.lua
>  -- On initial installation, we'll at least temporarily put the non-product
> @@ -316,6 +325,7 @@ glib-compile-schemas %{_datadir}/glib-2.0/schemas &>
> /dev/null || :
>  %{!?_licensedir:%global license %%doc}
>  %license LICENSE
>  %config %attr(0644,root,root) /usr/lib/os.release.d/os-release-atomichost
> +%{_datadir}/ostree/trusted.gpg.d/*
>  
>  
>  %files cloud
the problem should be solved entirely in fedora-repos.  we could easily make a fedora-repos-atomic sub package. that has the bits in it. I just need to know which of the gpog keys we need to pull into the keyring

Comment 9 Colin Walters 2016-11-10 19:28:36 UTC
Conceptually I think it should be fedora-repos-ostree, and we might as well also take the next step of including the remote config there, like we do for CentOS.  As far as the keys, each branch should be signed by the same key that's signing RPMs, right?

Comment 10 Dennis Gilmore 2016-11-10 19:55:44 UTC
yes each release has its own keys. It comes down to is there value in having the f26 key installed on a f25 system, or f25 on f24 to facilitate transitioning from one release to the next. or is it all self contained within a release and we only need the key for the release in question.

Comment 11 Colin Walters 2016-11-10 20:48:14 UTC
Unfortunately, right now each release is a separate ostree repository.  That means we need separate remotes, and hence having separate keyrings is forced (if we use per-remote keyrings).

If we choose the /usr/share/ostree/trusted.gpg.d route, then indeed it'd be useful to have future release keyrings in there.

But we should try to do the per-remote keyring I think.  Hm...this would likely be a lot simpler if we could refer to an existing file path in the remote spec, then we could just point it at the rpm-md key.

Comment 12 Colin Walters 2016-11-10 20:49:00 UTC
Filed that as https://github.com/ostreedev/ostree/issues/573

Comment 13 Colin Walters 2017-01-23 20:57:12 UTC
I started working on this, but it blocks on https://pagure.io/releng/issue/6598

Comment 14 Colin Walters 2017-02-03 14:24:05 UTC
Ok, so now we'd need to rebuild rpm-ostree with the current ostree, so we pick up `ostree_repo_verify_commit_for_remote()`.  See https://kojipkgs.fedoraproject.org//packages/rpm-ostree/2016.13/1.fc25/data/logs/x86_64/build.log for where it's too old.

That said I'd like to try to bump to 2017.1.  But that blocks on https://pagure.io/releng/issue/6602

Comment 15 Colin Walters 2017-02-14 20:46:23 UTC
This should be ready for more work now.  I'd like to finally add the remote config to a package so that we can change it on upgrades.

A conceptual issue remains in that fedora-repos is currently RPM specific, and we still need to have an ostree remote config.  If we add /etc/ostree/remotes.d/fedora-atomic.conf into this package, it'd show up in the container base image, Server etc.  That doesn't seem like a huge problem, but we might as well create a subpackage fedora-repos-ostree.  Agree?

Comment 16 Colin Walters 2017-02-14 20:47:52 UTC
Alternatively, create fedora-repos-atomic, which could have a future configuration for registry.fedoraproject.org for container images too.  Then it could be inherited by Server/Workstation too.  Yes, the ostree remote would show up there but I doubt anyone's going to really care about a new file in either of those; they're both already huge.

Comment 17 Colin Walters 2017-03-03 21:52:13 UTC
I don't mind doing the work on this to create a `fedora-repos-atomic` subpackage if that's fine with everyone else.  Open to other ideas as well.

Comment 18 Dusty Mabe 2017-07-07 14:52:09 UTC
i'm +1 to another rpm but maybe would prefer we don't add container image registries to it. Seems like it would be best if we don't intertwine the two. i.e. there is a case where you want container registry information on a non-atomic host system. WDYT?

If we want to create a `fedora-repos-ostree` or `fedora-repos-atomic` package then let's create a work item for it on pagure.io/atomic-wg.

Comment 20 Colin Walters 2017-07-14 17:31:04 UTC
Hi @ausil, thanks for working on this.  However there's a bit of confusion in this bug; originally we were talking about needing a separate *keys* subpackage. 

But in https://bugzilla.redhat.com/show_bug.cgi?id=1393545#c12 everything changed, since ostree learned how to use the keys from /etc/pki/rpm-gpg like the libdnf side does.

Then the conversation turned to having the *ostree remote files* in a package, which can just reference the rpm-gpg keys.

I think a variant of your patch which adds a -repos subpackage and includes content like:

```
$ cat /etc/ostree/remotes.d/fedora-atomic-27.conf 
[remote "fedora-atomic-27"]
url=https://kojipkgs.fedoraproject.org/atomic/rawhide/
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-primary
```

is what we're talking about.

However...I have to say again everything would be dramatically simpler if there was just one ostree repo, or at least one for FAH and one for FWS.  But that does run up against the key-per-release issue.

Comment 21 Dusty Mabe 2017-07-14 17:38:10 UTC
(In reply to Colin Walters from comment #20)

> But in https://bugzilla.redhat.com/show_bug.cgi?id=1393545#c12 everything changed, since ostree learned how to use the keys from /etc/pki/rpm-gpg like the libdnf side does.

So we don't need this work at all now, right? I like the gpgkeypath approach personally. 

> But that does run up against the key-per-release issue.

doesn't gpgkeypath=key1,key2 work?

Comment 22 Colin Walters 2017-07-14 18:00:58 UTC
That's https://github.com/ostreedev/ostree/issues/773
Which would be pretty easy to implement, and I'm happy to do so but we need to be sure how we're going to use it first.

Say we had a unified repo, it might look like this:

```
[remote "fedora-atomic"]
url=https://kojipkgs.fedoraproject.org/atomic/
gpg-verify=true
gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-primary,/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-27-primary
```

But what happens when f28 comes out?  Do we rotate that key list, and it'd end up in the new tree?  But the problem is that the way ostree works we'll be reading the remote config from the *old* tree.  We could just always have the file specify "N-1,N,N+1" keys?

Or perhaps it's simpler to have something like a directory:
/etc/pki/rpm-gpg/fedora-supported/
  RPM-GPG-KEY-fedora-26-primary
  RPM-GPG-KEY-fedora-27-primary

Then we could say:
gpgkeypath=/etc/pki/rpm-gpg/fedora-supported

Note that currently libdnf actually auto-imports everything in /etc/pki/rpm-gpg...which means it trusts RPM-GPG-KEY-fedora-10-primary too which it likely shouldn't.

Comment 23 Dusty Mabe 2017-07-14 19:22:24 UTC
(In reply to Colin Walters from comment #22)
> 
> Say we had a unified repo, it might look like this:
> 
> ```
> [remote "fedora-atomic"]
> url=https://kojipkgs.fedoraproject.org/atomic/
> gpg-verify=true
> gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-primary,/etc/pki/rpm-gpg/
> RPM-GPG-KEY-fedora-27-primary
> ```

Exactly.

> 
> But what happens when f28 comes out?  Do we rotate that key list, and it'd
> end up in the new tree?  But the problem is that the way ostree works we'll
> be reading the remote config from the *old* tree.  We could just always have
> the file specify "N-1,N,N+1" keys?

Why not just N,N+1 ?

basically if we start delivering the repo file via an rpm then we have
this:

I'm on Fedora 26 /etc/ostree/remotes.d/fedora-atomic.conf is provided
by rpm foo-fc26.rpm. and has gpgkeypath=f26key,f27key. 

rpm-ostree upgrade from f26 to f27. Now /etc/ostree/remotes.d/fedora-atomic.conf
is provided  by foo-fc27.rpm and has gpgkeypath=f27key,f28key.

Thoughts?

Comment 24 Colin Walters 2017-07-16 19:02:08 UTC
I think that'd work, yep.

Comment 25 Colin Walters 2017-07-16 19:02:29 UTC
What do you think, Dennis?

Comment 26 Dennis Gilmore 2017-07-31 22:29:36 UTC
We probably should make a single key that we sign ostree commits with, like we have a single key for all modules.

Comment 27 Jan Kurik 2017-08-15 07:34:40 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 27 development cycle.
Changing version to '27'.

Comment 28 Dusty Mabe 2017-09-25 18:13:55 UTC
(In reply to Dennis Gilmore from comment #26)
> We probably should make a single key that we sign ostree commits with, like
> we have a single key for all modules.

Not sure how likely this is to happen anytime soon (or even if it's a good idea). Can we go with proposal in https://bugzilla.redhat.com/show_bug.cgi?id=1393545#c23 until we figure that out?

Comment 29 Fedora End Of Life 2018-02-20 15:21:56 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 28 development cycle.
Changing version to '28'.

Comment 30 Ben Cotton 2019-05-02 22:08:31 UTC
This message is a reminder that Fedora 28 is nearing its end of life.
On 2019-May-28 Fedora will stop maintaining and issuing updates for
Fedora 28. It is Fedora's policy to close all bug reports from releases
that are no longer maintained. At that time this bug will be closed as
EOL if it remains open with a Fedora 'version' of '28'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 28 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 31 Ben Cotton 2019-05-28 19:26:23 UTC
Fedora 28 changed to end-of-life (EOL) status on 2019-05-28. Fedora 28 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.