Bug 2043704

Summary: Syncing sha-checksummed KS repository fails with: " Artifact() got an unexpected keyword argument 'sha' "
Product: Red Hat Satellite Reporter: Brad Buckingham <bbuckingham>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Lai <ltran>
Severity: urgent Docs Contact:
Priority: high    
Version: 6.10.0CC: ahumbe, ggainey, ltran, osousa, rchan, saydas, smajumda, ttereshc
Target Milestone: 6.10.3Keywords: PrioBumpGSS, Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python-pulp-rpm-3.14.12-1.el7pc Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 2024889 Environment:
Last Closed: 2022-03-08 21:26:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Comment 2 Lai 2022-02-04 20:05:40 UTC
Failing QA

Steps to reproduce:
1. Enable 'Red Hat Enterprise Linux 5 Server Kickstart x86_64 5.10' repository (I did 5.11 as well)
2. Start the sync

Expected:
Sync should pass successfully.

Actual:
Sync failed with "ModelBase object argument after ** must be a mapping, not str"

Failed on 6.10.3 snap 1 with python-pulp-rpm-3.14.12-1.el7pc

Comment 4 Grant Gainey 2022-02-04 22:11:59 UTC
Reproduced against pulp_rpm/main using the following script:

=====
#!/bin/bash
REPO_NAME="rhel5-ks"
REMOTE_URL=https://cdn.redhat.com/content/dist/rhel/server/5/5.11/x86_64/kickstart
pulp rpm remote create --name ${REPO_NAME} --url ${REMOTE_URL} --policy 'immediate' \
  --ca-cert @/home/vagrant/devel/pulp_startup/CDN_cert/redhat-uep.pem \
  --client-key @/home/vagrant/devel/pulp_startup/CDN_cert/cdn.key \
  --client-cert @/home/vagrant/devel/pulp_startup/CDN_cert/cdn.pem \
  --tls-validation false
pulp rpm repository create --name ${REPO_NAME} --remote ${REPO_NAME} | jq -r .pulp_href
pulp rpm repository sync --name ${REPO_NAME}
=====

Output:

=====
(pulp) []$ ./rhel5_ks_sync.sh 
{
  "pulp_href": "/pulp/api/v3/remotes/rpm/rpm/63dd2429-f64e-4acd-87d1-0161171ba481/",
  "pulp_created": "2022-02-04T21:48:48.359792Z",
  "name": "rhel5-ks",
  "url": "https://cdn.redhat.com/content/dist/rhel/server/5/5.11/x86_64/kickstart",
  "ca_cert": "...",
  "client_cert": "...",
  "tls_validation": false,
  "proxy_url": null,
  "pulp_labels": {},
  "pulp_last_updated": "2022-02-04T21:48:48.359809Z",
  "download_concurrency": null,
  "max_retries": null,
  "policy": "immediate",
  "total_timeout": null,
  "connect_timeout": null,
  "sock_connect_timeout": null,
  "sock_read_timeout": null,
  "headers": null,
  "rate_limit": null,
  "sles_auth_token": null
}

/pulp/api/v3/repositories/rpm/rpm/b913b8de-6f9e-4ce0-9f83-396f497d1fe5/
Started background task /pulp/api/v3/tasks/4422a9c5-9d9f-4818-ae4a-097a979f78dc/
......Error: Task /pulp/api/v3/tasks/4422a9c5-9d9f-4818-ae4a-097a979f78dc/ failed: 'pulpcore.app.models.content.Artifact() argument after ** must be a mapping, not str'
(pulp) []$
=====

This fails not because sha==sha1 - it fails because the RHEL5-KS treeinfo does not include a "checksums" stanza. See https://cdn.redhat.com/content/dist/rhel/server/5/5.11/x86_64/kickstart/treeinfo

Comment 5 Grant Gainey 2022-02-04 22:29:40 UTC
Correcting the problem being encountered:

At #839 in this code-block:

https://github.com/pulp/pulp_rpm/blob/main/pulp_rpm/app/tasks/synchronizing.py#L838-L845

(path, checksum) == ('images/pxeboot/initrd.img', 'x86_64')

We need to figure out where that happens - expected behavior would be ('images/pxeboot/initrd.img', {})

Comment 6 Grant Gainey 2022-02-07 20:11:43 UTC
The problem here is that rhel5.11-ks has no [checksums] stanza, and the treeinfo-parsing-codepath "assumes" that it must.

@ltran - congratulations, you've discovered a new bug! Please open a new BZ with the content from #c2-#c5. Add that RHEL6.1 and later do include that stanza, and work fine.

NewBug will block using rhel5-ks to test. From the associated issue and PR, and investigating CDN, other repos that would invoke the "unexpected keyword argument 'sha'" message include:

* https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os
  * see "sha" here https://cdn.redhat.com/content/dist/rhel/server/6/6.1/x86_64/os/repodata/repomd.xml
* https://cdn.redhat.com/content/dist/rhel/server/6/6.6/x86_64/os
  * see "sha" here https://cdn.redhat.com/content/dist/rhel/server/6/6.6/x86_64/os/repodata/repomd.xml

You can therefore use them to verify **this** BZ, once we have a new one opened. Does that sound reasonable?

Comment 7 Lai 2022-02-07 20:30:32 UTC
This is reasonable.  I'll start opening a new bz for this.

Comment 8 Lai 2022-02-07 20:47:20 UTC
And here's the new bz for the issue encountered: https://bugzilla.redhat.com/show_bug.cgi?id=2051713

Comment 9 Lai 2022-02-07 20:50:45 UTC
Putting this back on_qa for retesting.

Comment 10 Lai 2022-02-07 21:45:08 UTC
Steps to retest:

1. Enable "Red Hat Enterprise Linux 6 Server RPMs x86_64 6.6"
2. Sync the repo in step 1.

Expected:
syncing should be successful

Actual
syncing is successful.

Note:
I checked the Red Hat Enterprise Linux 6 Server RPMs x86_64 6.6 repo and it does have "sha" in the repomd.xml.  And since the main issue from this bz is the presence of "sha", using Red Hat Enterprise Linux 6 Server RPMs x86_64 6.6 fits the need of testing this bz.

Verified on 6.10.3 snap 1 with python3-pulp-rpm-3.14.12-1.el7pc.noarch

Comment 11 Grant Gainey 2022-02-08 12:27:19 UTC
I have changed the Summary for this bug to better reflect what's being fixed here - we fix the SHA-problem, but RHEL5-KS will still fail due to 

https://bugzilla.redhat.com/show_bug.cgi?id=2051713

Comment 16 errata-xmlrpc 2022-03-08 21:26:08 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Low: Satellite 6.10.3 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:0790