Bug 1564830

Summary: [Satellite 6.3]Unable to sync docker images from 'quay.io' registry.
Product: Red Hat Satellite Reporter: Amar Huchchanavar <ahuchcha>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED DUPLICATE QA Contact: Katello QA List <katello-qa-list>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.3.0CC: austin, ipanova, mmithaiw, sgraessl, tomckay, ttereshc
Target Milestone: UnspecifiedKeywords: Reopened, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-05-09 14:26:28 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Amar Huchchanavar 2018-04-08 05:19:30 UTC
Description of problem:
Unable to sync docker images from quay.io registry.
It fails with an error 'Could not find registry API at https://quay.io/api/v1/repository/'


Version-Release number of selected component (if applicable):
Satellite 6.3

How reproducible:
Always 

Steps to Reproduce:
1.Configure docker repository with url https://quay.io/api/v1/repository/  or  https://quay.io/
2.Try to sync the repository.


Actual results:
Fails with an error : 
Could not find registry API at https://quay.io/

Expected results:
It should sync the images from the Quay

Additional info:
Quay allows access via api when providing authentication token.
With satellite, I do not see any possible way to do it.

Comment 3 Tom McKay 2018-04-26 19:12:49 UTC
This works fine, as far as I can tell. Note that to sync from quay.io a username and token password must be specified in the repo details. This can be found on the quay.io user settings page.

Please let me know if there are any questions. Thanks!

Comment 4 Tom McKay 2018-05-07 13:59:44 UTC
Reopening... this works upstream but not in 6.3.0. Pulp is not able to find a registry at https://quay.io even w/ username/password.

Comment 5 amacdona@redhat.com 2018-05-07 18:28:06 UTC
I was able to get this to work with current master on pulp and pulp_docker. It is unclear if there is a bug or if the reproduction steps were incorrect.

1. I made a public repository on quay.io, https://quay.io/repository/asmacdo/busybox

2. Create a pulp repo

$ pulp-admin docker repo create --repo-id myquay --feed https://quay.io/repository/ --upstream-name asmacdo/busybox

3. Sync 

$ pulp-admin docker repo sync run --repo-id myquay

There could be an auth issue as well, but please make sure that the feed and upstream name are both correct.

Comment 6 Tom McKay 2018-05-07 20:39:13 UTC
@amacdona - Does it work in either of these versions?
pulp-server-2.13.4.6-1.el7sat.noarch.rpm 
pulp-server-2.13.4.9-1.el7sat.noarch.rpm

Comment 7 Ina Panova 2018-05-09 12:59:41 UTC
i did not look very deep in here but this looks like same issue with tokens and scope specification like in google registry.

as you can see the scope is not specified by registry in the auth-header
>>Download failed: Download of https://quay.io/v2/auth?service=quay.io failed with code 401: UNAUTHORIZED

$ git checkout grc-sat 
Switched to branch 'grc-sat'
[ipanova@ina pulp_docker]$ for s in {qpidd,pulp_celerybeat,pulp_resource_manager,pulp_workers,httpd}; do sudo systemctl restart $s; done
[ipanova@ina pulp_docker]$ pulp-admin -vv docker repo sync run --repo-id myquay

Task Succeeded

Task Succeeded


$ pulp-admin docker repo list


+----------------------------------------------------------------------+
                          Docker Repositories
+----------------------------------------------------------------------+

Id:                  myquay
Display Name:        None
Description:         None
Content Unit Counts: 
  Docker Blob:     3
  Docker Manifest: 1
  Docker Tag:      1


with this patch we in a defensive way are setting the scope even if the registry did not provide it:

>>> Download succeeded: https://quay.io/v2/auth?scope=repository%3Aasmacdo%2Fbusybox%3Apull&service=quay.io.