Bug 1587869 - Can't import image signature from the RedHat registry
Summary: Can't import image signature from the RedHat registry
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Image
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 3.10.0
Assignee: Ben Parees
QA Contact: Dongbo Yan
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-06-06 08:26 UTC by zhou ying
Modified: 2018-07-30 19:17 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
undefined
Clone Of:
Environment:
Last Closed: 2018-07-30 19:17:19 UTC
Target Upstream Version:


Attachments (Terms of Use)
controllers pod logs (1.33 MB, text/plain)
2018-06-07 02:54 UTC, zhou ying
no flags Details
new logs from controller (3.80 MB, text/plain)
2018-06-11 03:12 UTC, zhou ying
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1816 None None None 2018-07-30 19:17:58 UTC

Description zhou ying 2018-06-06 08:26:12 UTC
Description of problem:
After config the sigstore, can't import image signatrue from the redhat registry.

Version-Release number of selected component (if applicable):
openshift v3.10.0-0.60.0

How reproducible:
always

Steps to Reproduce:
1. Config the sigstore for the master host, and restart api and controllers;
2. Try to import image from redhat registry:
`oc import-image jboss-webserver31-tomcat8-openshift:latest --from=registry.access.redhat.com/jboss-webserver-3/webserver31-tomcat8-openshift:latest --confirm -n install-test`
3. Check the image
`oc describe istag jboss-webserver31-tomcat8-openshift:latest`


Actual results:
3.Can't import the image signature from the registry.


Expected results:
3. Should import the image signature from the registry.

Additional info:

Comment 1 Ben Parees 2018-06-06 14:08:09 UTC
can you share the contents of your /etc/containers/registries.d directory?  (filenames + contents)

w/ a redhat.yaml file in /etc/containers/registries.d with the following content:

docker:
  registry.access.redhat.com:
    sigstore: https://access.redhat.com/webassets/docker/content/sigstore


I was able to import the signatures:

Image Name:		sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e
Docker Image:		registry.access.redhat.com/jboss-webserver-3/webserver31-tomcat8-openshift@sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e
Name:			sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e
Created:		30 seconds ago
Annotations:		image.openshift.io/dockerLayersOrder=ascending
Image Size:		204.8MB (first layer 74.92MB, last binary layer 44.87MB)
Image Signatures:	 
			Name:	sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e@7955e42da12ab29746d0b18a6e92fd1f4fe68f39842dc377a86623e677149016
			Type:	AtomicImageV1
			Status:	Unverified
Image Signatures:	 
			Name:	sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e@aa244d390f5611242b06e3dc25a3e4e9fd3f59ff9a0dfb64f3cf7d3ed587f4de
			Type:	AtomicImageV1
			Status:	Unverified
Image Signatures:	 
			Name:	sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e@47763d5c2b59a8ced4548690aeb7ca252ebdfb043be9fc479436b77cef17f99b
			Type:	AtomicImageV1
			Status:	Unverified


If you can share level 5 master logs that would also help.

Comment 3 zhou ying 2018-06-07 02:54:24 UTC
Created attachment 1448570 [details]
controllers pod logs

Comment 4 zhou ying 2018-06-07 02:55:21 UTC
[root@qe-yinzhou-310-master-etcd-1 registries.d]# ll /etc/containers/registries.d/redhat.yaml 
-rw-r--r--. 1 root root 112 Jun  6 22:21 /etc/containers/registries.d/redhat.yaml
[root@qe-yinzhou-310-master-etcd-1 registries.d]# cat /etc/containers/registries.d/redhat.yaml 
docker:
  registry.access.redhat.com:
    sigstore: https://access.redhat.com/webassets/docker/content/sigstore

Comment 5 Ben Parees 2018-06-07 15:10:27 UTC
No other files in /etc/containers/registries.d ?

Comment 6 zhou ying 2018-06-08 02:49:17 UTC
With the default file /etc/containers/registries.d/default.yaml

Comment 8 zhou ying 2018-06-11 03:12:08 UTC
Created attachment 1449843 [details]
new logs from controller

Comment 9 zhou ying 2018-06-11 03:13:50 UTC
Ben :

   I create the env by jenkins , not sure is the same of https://bugzilla.redhat.com/show_bug.cgi?id=1506066?

Comment 10 zhou ying 2018-06-11 03:14:49 UTC
And I can import the signature with 3.9.30.

Comment 11 Ben Parees 2018-06-11 03:18:20 UTC
if your master is running inside a container, then the 
 /etc/containers/registries.d/redhat.yaml file needs to be present inside that container also.

So i'm not sure what the jenkins job does when creating an env, but it certainly sounds possible that the master controller process is not finding your redhat.yaml file which contains the lookaside information for the signature retrieval.

also I did not see the expected output in your logs, which makes me think that whatever openshift binary you replaced, is not the one that is running the master (which again would make sense if your master is running inside a container).

Comment 12 Ben Parees 2018-06-11 03:21:15 UTC
oc cluster up was very different in 3.9, so if your cluster is based on oc cluster up, it may be that the configuration process has changed for this.

do you have access to a "traditional" 3.10 cluster where you can attempt signature imports?  I suspect getting the correct lookaside configuration into a master running inside a container is going to be a bit tricky, unless the new cluster up rewrite supports mounting directories into the master container.

Comment 14 Ben Parees 2018-06-11 20:39:08 UTC
I tried to patch your cluster to mount the necessary config but i think it's not starting now.

the PR to fix this is here:
https://github.com/openshift/openshift-ansible/pull/8719

if you want to try to manually fix your cluster, you need to patch your /etc/origin/node/pods/controller.yaml w/ the hostpath mounts defined in that PR so that the /etc/containers/registries.d dir is mounted into the controller pod at the same path.

Comment 16 zhou ying 2018-06-14 05:27:43 UTC
Confirmed with openshift ,the issue has fixed:
openshift v3.10.0-0.67.0

[root@qe-yinzhou-310-master-etcd-1 home]# oc describe istag jboss-webserver31-tomcat8-openshift:latest -n install-test
Image Name:		sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e
Docker Image:		registry.access.redhat.com/jboss-webserver-3/webserver31-tomcat8-openshift@sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e
Name:			sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e
Created:		14 minutes ago
Annotations:		image.openshift.io/dockerLayersOrder=ascending
Image Size:		204.8MB (first layer 74.92MB, last binary layer 44.87MB)
Image Signatures:	 
			Name:	sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e@7955e42da12ab29746d0b18a6e92fd1f4fe68f39842dc377a86623e677149016
			Type:	AtomicImageV1
			Status:	Unverified
Image Signatures:	 
			Name:	sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e@aa244d390f5611242b06e3dc25a3e4e9fd3f59ff9a0dfb64f3cf7d3ed587f4de
			Type:	AtomicImageV1
			Status:	Unverified
Image Signatures:	 
			Name:	sha256:29470583ce1511dbbe8e2552b7f0278d0d74595c42a00632cd23fda1ffab361e@47763d5c2b59a8ced4548690aeb7ca252ebdfb043be9fc479436b77cef17f99b
			Type:	AtomicImageV1
			Status:	Unverified

Comment 18 errata-xmlrpc 2018-07-30 19:17:19 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816


Note You need to log in before you can comment on or make changes to this bug.