Bug 1962189 - several issues with ansible collections Requirements.yml
Summary: several issues with ansible collections Requirements.yml
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Repositories
Version: 6.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: 6.10.0
Assignee: Justin Sherrill
QA Contact: Peter Ondrejka
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-05-19 13:29 UTC by Peter Ondrejka
Modified: 2021-11-16 14:11 UTC (History)
3 users (show)

Fixed In Version: tfm-rubygem-katello-4.1.1,tfm-rubygem-katello-4.1.1.6-1,tfm-rubygem-katello-4.1.1.13-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-11-16 14:11:01 UTC
Target Upstream Version:


Attachments (Terms of Use)
prodlog (14.73 KB, text/plain)
2021-05-19 13:29 UTC, Peter Ondrejka
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Foreman Issue Tracker 32867 0 Normal Closed Creating ansible collection repo fails with: "Invalid URL Ensure the URL ends '/'" but repo gets created 2021-07-06 20:19:41 UTC
Foreman Issue Tracker 33251 0 None None None 2021-08-12 19:39:27 UTC
Red Hat Product Errata RHSA-2021:4702 0 None None None 2021-11-16 14:11:08 UTC

Description Peter Ondrejka 2021-05-19 13:29:16 UTC
Created attachment 1784828 [details]
prodlog

Description of problem:
There are several issues with using Requirements.yml for ansible collection repo type in the repo creation dialog

Version-Release number of selected component (if applicable):
Satellite 6.10 snap 1

How reproducible:
always

Issues:

1. Unable to sync repository defined in Requirements.yml (used several sources based on [1]). Repository is created but syncing fails with:

"Unable to synchronize any repository. You either do not have the permission to synchronize or the selected repositories do not have a feed url."

(prodlog excrept in attachment)

2. Error when editing the requirements field in the repository details page. Basically with any change in the field, upon saving the pop-up error message appears:

An error occurred saving the Repository: Task b018f191-5f1c-4f80-93d7-83b15c5a4e3e: NoMethodError: super: no superclass method `remote_options' for #<Katello::Pulp3::Repository::AnsibleCollection:0x0000000017a65e20> Did you mean? ssl_remote_options

However, the changes are saved

3. No validation whatsoever on the Requirements.yml (is it valid yml? are the expected parameters included?)

4. Link to requirements.yml documentation in the pop-up help in the repo creation dialog is dead


Additional info:
[1] https://docs.ansible.com/ansible/latest/user_guide/collections_using.html#install-multiple-collections-with-a-requirements-file

Comment 1 Justin Sherrill 2021-07-06 20:19:41 UTC
Connecting redmine issue https://projects.theforeman.org/issues/32867 from this bug

Comment 2 Bryan Kearney 2021-07-07 00:01:43 UTC
Moving this bug to POST for triage into Satellite since the upstream issue https://projects.theforeman.org/issues/32867 has been resolved.

Comment 3 Peter Ondrejka 2021-07-08 10:57:09 UTC
Revisited this in sat 6.10 snap 8, documentation link has been fixed, the remaining issues persist.

For issue 1, the error is now differrent:
- create ansible repo:
upstream url: https://galaxy.ansible.com/
requirements.yml:

---
collections:
- theforeman.foreman


- submit dialog, the following pop-up error appears:

Task 7c2cf1a1-634a-4f38-87dc-0fdca4c012d6: ArgumentError: `proxy_username` is not a valid attribute in `PulpAnsibleClient::AnsibleCollectionRemote`. Please check the name to make sure it's valid. List of attributes: [:name, :url, :ca_cert, :client_cert, :client_key, :tls_validation, :proxy_url, :username, :password, :pulp_labels, :download_concurrency, :policy, :total_timeout, :connect_timeout, :sock_connect_timeout, :sock_read_timeout, :rate_limit, :requirements_file, :auth_url, :token]

- the repo is actually created, but it cannot be synced as the above pending task holds the lock

Comment 4 Peter Ondrejka 2021-07-22 12:57:53 UTC
Checked on satellite 6.10 sn 10, points 1, 2 and 4 from the problem description have been fixed.

For problem 3, there is a validation of requirements.yml in place, saving with wrong yml syntax you get a red popup with:

Task c477d0f3-eadf-4c1c-b017-bc4f6253f099: PulpAnsibleClient::ApiError: Error message: the server returns an error HTTP status code: 400 Response headers: {"Date"=>"Thu, 22 Jul 2021 12:52:42 GMT", "Server"=>"gunicorn", "Content-Type"=>"application/json", "Vary"=>"Accept,Cookie", "Allow"=>"GET, POST, HEAD, OPTIONS", "X-Frame-Options"=>"SAMEORIGIN", "Content-Length"=>"150", "Correlation-ID"=>"5549ccf2-21dc-486e-b8e5-231be472a246", "Access-Control-Expose-Headers"=>"Correlation-ID", "Via"=>"1.1 dhcp-2-106.vms.sat.rdu2.redhat.com", "Connection"=>"close"} Response body: {"non_field_errors":["Expecting collections requirements file to be a dict with the key collections that contains a list of collections to install."]}

Which is ok, but even with this error the repo is actually created in the background so if you try to re-save with fixed yml, you will get "name already taken"

Comment 5 Brad Buckingham 2021-08-03 19:23:43 UTC
Moving to POST.  Please be aware there are multiple PRs on the redmine that need to be pulled in.

Comment 7 Peter Ondrejka 2021-08-06 09:07:59 UTC
Checked on Satellite 6.10 snap 12, the problem with saving repository in spite of error has been fixed, but unfortunately there are some remaining problems with requirements.yml validation:

1.) Putting malformed yml triggers a well formed warning, but if you put just a random string the warning message exposes the response headers (see screenshot-1)

2.) More importantly, validation on editing the requirements field of an existing repo catches malformed yml but permits to save a random string (see screenshot-2)

3.) Similarly, if you create a requirements.yml file with just a random string content, and upload it in the repository details page, it populates the requiremnets field and can be saved. (The same scenario is caught when creating a repo).

4.) I cannot reproduce this consistently but it happened to me several times that when uploading a requirements.yml file on the repository details, the requirements field got directly populated from it without asking for confirmation. This change wasn't then saved and after page refresh the original value remained in requirements field. 

5.) more of an RFE: currently the repository created by uploading a requirements file bears no memory of the input file, I wonder if users would be interested to have the information on the source file name (path?) in the repo details

Comment 11 Bryan Kearney 2021-08-12 20:04:41 UTC
Upstream bug assigned to jsherril@redhat.com

Comment 12 Bryan Kearney 2021-08-12 20:04:43 UTC
Upstream bug assigned to jsherril@redhat.com

Comment 13 Brad Buckingham 2021-08-19 13:36:27 UTC
Moving to POST as https://github.com/Katello/katello/pull/9555 was merged upstream.

Comment 15 Peter Ondrejka 2021-08-30 13:35:46 UTC
Verified on Satellite 6.10 snap 15

Comment 18 errata-xmlrpc 2021-11-16 14:11:01 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.10 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2021:4702


Note You need to log in before you can comment on or make changes to this bug.