Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1731229 - podman search against Red Hat Satellite 6 fails.
Summary: podman search against Red Hat Satellite 6 fails.
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Container Management - Content
Version: 6.5.0
Hardware: Unspecified
OS: Linux
medium
medium
Target Milestone: 6.8.0
Assignee: Partha Aji
QA Contact: Mirek Długosz
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-18 17:18 UTC by Bharati Kailas Shahu
Modified: 2023-12-15 16:37 UTC (History)
13 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 12:58:49 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 29742 0 Normal Closed podman search against katello 2021-02-18 13:40:54 UTC
Red Hat Knowledge Base (Solution) 4293701 0 None None None 2019-07-25 13:34:30 UTC
Red Hat Product Errata RHSA-2020:4366 0 None None None 2020-10-27 12:59:06 UTC

Description Bharati Kailas Shahu 2019-07-18 17:18:13 UTC 
Description of problem:
Tried to do "podman search" on the registry as provided by Satellite.
but getting error " ERRO[0000] error getting search results from v2 endpoint "satellite.example.com:5000", status code 404 (Not Found) "

Version-Release number of selected component (if applicable):


How reproducible:



Steps to Reproduce:
1. On Satellite server, kept redhat-ubi-images-ubi7 on container  
2. Tried to search it with following command but facing issue.
   # podman search redhat-ubi-images-ubi7

Actual results:   [root@vmexample ~]# podman search redhat-ubi-images-ubi7
  ERRO[0000] error getting search results from v2 endpoint "syssat65.gsslab.pnq2.redhat.com:5000", status code 404 (Not Found)


Expected results: It should search for image


Additional info:
Comment 6 Bryan Kearney 2019-09-05 15:34:33 UTC 
clearing the needinfo
Comment 7 Christian Marineau 2019-10-23 23:33:42 UTC 
Hi,

Working with Satellite 6.6 + podman on RHEL8 client: I am facing the same behaviour. Following the instructions from the Documentation [1] and trying to search from the client always fails with 404.

Can we validate what needs to be done from the client side to consume container images from Satellite and provide the complete solution?

[1] https://access.redhat.com/documentation/en-us/red_hat_satellite/6.6/html-single/content_management_guide/index#Managing_Container_Images
Comment 8 Gunther Mayer 2020-05-12 15:14:26 UTC 
Hi,

I opened Case 02429234 in July 2019.
Now I found the problem is not yet resolved (despite we updatet to Satellite-6.7.0) in the meantime.

As Satellite also mangles the image names, we see "non-searchability" as a showstopper on our path to empower
our users to use UBI images.

Regards, Gunther
Comment 9 Justin Sherrill 2020-05-12 16:16:26 UTC 
This seems to be working for me against the authenticated registry on Satellite 6.7:

# podman  search   sat.example.com/
INDEX         NAME                                                                    DESCRIPTION   STARS   OFFICIAL   AUTOMATED
example.com   sat.example.com/default_organization-foo-node                   0                  
example.com   sat.example.com/default_organization-foo-docker                 0               

However, i did notice that when i tried to search a term, it gave no results:

# podman  search   sat.example.com/node


I did some investigation and it appears that Satellite is returning the wrong content type (html, instead of json).  When i fixed this, it seemed to return when specifying a search term:

# podman  -vvv search   sat.example.com/node
INDEX         NAME                                                                  DESCRIPTION   STARS   OFFICIAL   AUTOMATED
example.com   sat.example.com/default_organization-foo-node                 0                  


I'll attach a patch, hopefully we can get this resolved in a z-stream

Also Gunther, keep in mind that there are ways to 'unmangle' the docker image names: https://access.redhat.com/documentation/en-us/red_hat_satellite/6.7/html/content_management_guide/managing_container_images#Managing_Container_Names
Comment 10 Justin Sherrill 2020-05-12 16:17:52 UTC 
Created redmine issue https://projects.theforeman.org/issues/29742 from this bug
Comment 11 Justin Sherrill 2020-05-12 16:37:36 UTC 
actually i take that back as for the cause, i'm still investigating it.

One thing to note is that if you are using an authenticated registry, you have to login first:

# podman login sat.example.com

# podman search sat.example.com/
Comment 12 Gunther Mayer 2020-05-12 17:27:22 UTC 
Hi Justin,
thanks for your hint.

I confirm the search-problem is resolved after I "login" (also see my case).
I had assumed a login is not needed as I can easily pull (but have to use ":5000").

1)
[root@evesp05 ~]# podman logout  -a
Removed login credentials for all registries

2) Now it complains about "auth token", I had assumed somehow this is the wrong port for containers (and it wants to authenticate for Satellite GUI)
[root@evesp05 ~]# podman search mysatellite/
ERRO[0000] error searching registry "mysatellite": couldn't search registry "mysatellite": 
unable to retrieve auth token: invalid username/password: unauthorized: authentication required 

3) Now pull _without_ "5000" does not work
[root@evesp05 ~]# podman pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi
Trying to pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi...
  unable to retrieve auth token: invalid username/password: unauthorized: authentication required
Error: error pulling image "mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi": 
unable to pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi: unable to pull image: 
Error initializing source docker://mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi:latest: u
nable to retrieve auth token: invalid username/password: unauthorized: authentication required


4) But I can pull from port 5000 (with mangled names): THIS is the reason I assumed I do not need "login". 

[root@evesp05 ~]# podman pull mysatellite:5000/vwag-red_hat_ubi_container_images-rhel7ubi
Trying to pull mysatellite:5000/vwag-red_hat_ubi_container_images-rhel7ubi...
Getting image source signatures
Copying blob cf5693de4d3c skipped: already exists
Copying blob 23302e52b49d skipped: already exists
Copying config 3c6e3294c1 done
Writing manifest to image destination
Storing signatures
3c6e3294c1ad4a2a9cbcfc0e152c303d839305dd1ff69a0e8d0a2f5af1cd51b8

5) Now I login:
[root@evesp05 ~]# podman login mysatellite
Username: user
Password: 
Login Succeeded!

6) After login, I can even pull without "5000"
[root@evesp05 ~]# podman pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi
Trying to pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi...
Getting image source signatures
Copying blob cf5693de4d3c skipped: already exists
Copying blob 23302e52b49d skipped: already exists
Copying config 3c6e3294c1 done
Writing manifest to image destination
Storing signatures
3c6e3294c1ad4a2a9cbcfc0e152c303d839305dd1ff69a0e8d0a2f5af1cd51b8

So this leads to another topic: is it correct to provide images via ":5000" without requiring authentication?

Regards, Gunther
Comment 13 Gunther Mayer 2020-05-12 17:51:53 UTC 
Hi,

i want to enable "search" for my users without (satellite-) login, so they can easily access UBIs.

1)
I changed 
Container Image Registry
Unauthenticated Pull       Yes    (this was "No", now I changed to "Yes".

2)
But the search still does not work:
[root@evesp05 ~]# podman search mysatellite/
ERRO[0000] error searching registry "mysatellite": ...: unable to retrieve auth token: invalid username/password: unauthorized: authentication required 

  

3) However now pull without ":5000" works:
[root@evesp05 ~]# podman pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi
Trying to pull mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi...
Getting image source signatures
Copying blob cf5693de4d3c skipped: already exists
Copying blob 23302e52b49d skipped: already exists
Copying config 3c6e3294c1 done
Writing manifest to image destination
Storing signatures
3c6e3294c1ad4a2a9cbcfc0e152c303d839305dd1ff69a0e8d0a2f5af1cd51b8

4) How can I allow "unauthenticated search" ?

Regards, Gunther
Comment 14 Justin Sherrill 2020-05-12 17:58:03 UTC 
Gunther:

"So this leads to another topic: is it correct to provide images via ":5000" without requiring authentication?"

The authenticated registry is 'newer' and supports more features.  The :5000 registry is will go away in a future release of satellite (Likely 7.0).  If you want you can block access to it via a firewall to only use the authenticated registry.


"4) How can I allow "unauthenticated search" ?"

You can allow this per lifecycle environment.  Simply go to Content > lifecycle environments, click on the lifecycle env you are interested in and change "Unauthenticated Pull" to yes.  


I'm still looking at the authenticated search issue.  For some reason podman isn't sending up the credentials.  Its very similar to https://github.com/containers/libpod/issues/5405
Comment 15 Justin Sherrill 2020-05-12 17:59:14 UTC 
Oh i missed your comment saying that switching Unauthenticated Pull to yes didn't work. will look into that.
Comment 16 Justin Sherrill 2020-05-12 18:17:41 UTC 
Gunther,

Strangely enough, it actually works with a term when using unauthenticated:

     
# podman search mysatellite/foo


Podman tries to be smart and uses the '/v2/catalog' api when doing a search without a term, which currently we require authentication for. So the following works currently with satellite 6.7 (and probably 6.6):

* podman search with term, without logging in and 'unauthenticated pull' set to yes on the lifecycle environment
* podman search without term when logged in 



So current issues needing resolving:

* podman search with search term when logged in and  Lifecycle Environment has 'unauthenticated pull' set to yes
* podman search without search term when not logged in (regardless of Lifecycle Environment 'unauthenticated pull' setting)
Comment 17 Gunther Mayer 2020-05-12 19:38:31 UTC 
Hi,

I confirm your findings. This really is confusing and needs make over.

0) Prerequisite: podman logout -a

1) As expected

[root@evesp05 ~]# podman search mysatellite/
ERRO[0000] error searching registry "mysatellite": couldn't search registry "mysatellite": unable to retrieve auth token: invalid username/password: unauthorized: authentication required 

2) I confirm your finding: I can search for a term, e.g. "ubi" (this is with Unauthenticated Pull =Yes")
    This comes unexpected…
[root@evesp05 ~]# podman search mysatellite/ubi |cat -n
     1	INDEX    NAME                                                                      DESCRIPTION   STARS   OFFICIAL   AUTOMATED
     2	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi                         0                  
     3	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-rhel8ubi                         0                  
     4	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7                             0                  
     5	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7-init                        0                  
     6	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7-minimal                     0                  
     7	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7_ubi-init                    0                  
     8	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7_ubi-minimal                 0                  
     9	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8                             0                  
    10	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8-init                        0                  
    11	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8-minimal                     0                  
    12	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8_ubi-init                    0                  
    13	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8_ubi-minimal                 0                  

3) Now I switched back to "Unauthenticated Pull = No"

4) This is the same as before (=expected)
[root@evesp05 ~]# podman search mysatellite/
ERRO[0000] error searching registry "mysatellite": couldn't search registry "mysatellite": unable to retrieve auth token: invalid username/password: unauthorized: authentication required

5) Searching for ubi now fails silently. Shouldn`t this give an error message?
[root@evesp05 ~]# podman search mysatellite/ubi
[root@evesp05 ~]#

6) Now I login:
root@evesp05 ~]# podman login mysatellite
Username: user
Password: 
Login Succeeded!

7) As you said, even when logged in, I cannot search for "ubi" (it fails silently)
[root@evesp05 ~]# podman search mysatellite/ubi
[root@evesp05 ~]#

8) The "/" search is fine now after login:
[root@evesp05 ~]# podman search mysatellite/ |cat -b
     1	INDEX    NAME                                                                      DESCRIPTION   STARS   OFFICIAL   AUTOMATED
     2	vw.vwg   mysatellite/vwag-rhel_images-rhel                                              0                  
     3	vw.vwg   mysatellite/vwag-rhel_images-rhel-init                                         0                  
     4	vw.vwg   mysatellite/vwag-rhel_images-rhel-minimal                                      0                  
     5	vw.vwg   mysatellite/vwag-rhel_images-rhel6                                             0                  
     6	vw.vwg   mysatellite/vwag-rhel_images-rhel6-init                                        0                  
     7	vw.vwg   mysatellite/vwag-rhel_images-rhel6_rhel                                        0                  
     8	vw.vwg   mysatellite/vwag-rhel_images-rhel7                                             0                  
     9	vw.vwg   mysatellite/vwag-rhel_images-rhel7-init                                        0                  
    10	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-rhel8ubi                         0                  
    11	vw.vwg   mysatellite/vwag-rhel_images-rhel7-minimal                                     0                  
    12	vw.vwg   mysatellite/vwag-rhel_images-rhel7_rhel                                        0                  
    13	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7                             0                  
    14	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7-init                        0                  
    15	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7-minimal                     0                  
    16	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7_ubi-init                    0                  
    17	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7_ubi-minimal                 0                  
    18	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8                             0                  
    19	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8-init                        0                  
    20	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8-minimal                     0                  
    21	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8_ubi-init                    0                  
    22	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8_ubi-minimal                 0                  
    23	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi                         0                  

9)
Now setting "Unauthenticated Pull : Yes" 
(side note: Step 8 still works fine).

10) The "ubi" search now works again:
- So this works with "Yes"
- it works with login and with logout (i.e. does not require login)
[root@evesp05 ~]# podman search mysatellite/ubi |cat -n
     1	INDEX    NAME                                                                      DESCRIPTION   STARS   OFFICIAL   AUTOMATED
     2	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-rhel7ubi                         0                  
     3	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-rhel8ubi                         0                  
     4	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7                             0                  
     5	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7-init                        0                  
     6	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7-minimal                     0                  
     7	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7_ubi-init                    0                  
     8	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi7_ubi-minimal                 0                  
     9	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8                             0                  
    10	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8-init                        0                  
    11	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8-minimal                     0                  
    12	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8_ubi-init                    0                  
    13	vw.vwg   mysatellite/vwag-red_hat_ubi_container_images-ubi8_ubi-minimal                 0                  

Regards, Gunther
Comment 18 Bryan Kearney 2020-07-13 20:05:49 UTC 
Upstream bug assigned to paji
Comment 20 Mirek Długosz 2020-09-23 15:48:00 UTC 
Tested couple of scenarios and I think they all work as expected:

1. When not logged in, searching without term returns all containers from lifecycle environments that have "Unauthenticated pull" set to yes. In other words, only unauthenticated containers are returned, and authenticated are hidden.
2. When not logged in, searching for term basically filters list from 1. for presence of term. This may return empty set if term is never found.
3. After logging in as admin, searching without term returns everything, including many duplicates.
4. After logging in, searching for term filters list from 3. You are likely to receive some duplicates.

I have not found any significant differences between docker and podman - they both return same number of results for each query. They might order results differently.


Tested on:
Satellite 6.8 snap 16
satellite-6.8.0-1.el7sat.noarch
katello-3.16.0-1.el7sat.noarch
foreman-2.1.2.12-1.el7sat.noarch
pulp-server-2.21.3-1.el7sat.noarch
foreman-proxy-2.1.2-2.el7sat.noarch

clients:
Docker version 19.03.11, build 42e35e6
podman version 2.0.6
Comment 24 errata-xmlrpc 2020-10-27 12:58:49 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.8 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:4366
Note You need to log in before you can comment on or make changes to this bug.