Bug 2038739

Summary: Extremely difficult to tell what repositories to Verify Checksum on when there are hundreds or thousands of packages listed as corrupted
Product: Red Hat Satellite Reporter: Taft Sanders <tasander>
Component: RepositoriesAssignee: Justin Sherrill <jsherril>
Status: CLOSED ERRATA QA Contact: Gaurav Talreja <gtalreja>
Severity: high Docs Contact:
Priority: high    
Version: 6.9.8CC: ahumbe, jsherril, osousa, pdwyer, pmendezh, saydas, vijsingh
Target Milestone: 6.9.9Keywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-3.18.1.51-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-04-20 20:34:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Taft Sanders 2022-01-09 23:41:27 UTC
Description of problem:
During pulp2to3 migration, any reported corrupted content is reported in a file of the content type. Since one of the options is to verify checksum for the corresponding repository for the content that is identified as correupted, this request is to add the corresponding repositories name to the corrupted content bundle supplied by the migration process.

Version-Release number of selected component (if applicable):
Satellite 6.10

How reproducible:
n/a

Steps to Reproduce:
1.
2.
3.

Actual results:
/tmp/unmigratable_content-20220109-11817-1oil4a6/{Rpm,File} contains a list of packages/files that are identified as corrupted.

Expected results:
Either in the same files or put the information in an additional file. A list of repositories that contain the contents in the Rpm and File lists should be included in the output to allow the user to know which repository to complete the verify checksum option against.

Additional info:
The following python script is currently being supplied to users to produce this feature. This request is to build this into the migration procedure.

corrupted_migration.py
----------------------------
import pymongo
import sys

RPM_FILE = sys.argv[1]

rpm=[]
with open(RPM_FILE, 'r') as tmp:
    for package in tmp:
        rpm.append(package.strip())
myclient=pymongo.MongoClient('mongodb://localhost:27017/')
mydb=myclient['pulp_database']
CORRUPTED_REPOS=[]
for package in rpm:
    mydoc = mydb['units_rpm'].find({"filename": package}, {"downloaded": 1})
    for x in mydoc:
        query2 = mydb['repo_content_units'].find({'unit_id': x['_id']},{'repo_id': 1})
        for y in query2:
            query3 = mydb['repos'].find({'repo_id': y['repo_id']},{'display_name': 1})
            for z in query3:
                if z['display_name'] not in CORRUPTED_REPOS:
                    CORRUPTED_REPOS.append(z['display_name'])

for name in CORRUPTED_REPOS:
        print(name)
----------------------------

Comment 1 Justin Sherrill 2022-02-15 16:05:54 UTC
Created redmine issue https://projects.theforeman.org/issues/34453 from this bug

Comment 3 Brad Buckingham 2022-02-28 11:59:31 UTC
Moving to POST as upstream PR is merged.

Comment 5 Gaurav Talreja 2022-03-28 17:17:47 UTC
Verified.

Tested on Satellite 6.9.9 Snap 1.0
Version: tfm-rubygem-katello-3.18.1.53-1.el7sat.noarch

Steps:
1. Sync some repositories with an immediate download policy, and publish them in CV.
2. 'corrupt' some files in /var/lib/pulp/content/ by either deleting them or echo'ing extra data to them like "echo  foobar >> /var/lib/pulp/content/path/to/file.rpm"
3. #  foreman-maintain content prepare
4. #  foreman-maintain content migration-stats

Observation:
Tested with older 6.9.z where identified corrupt content is added /tmp/unmigratable_content/{Rpm,File} as a list of packages/files, with no other info. 
And after-fix more info is included to that {Rpm,File} in csv format of "filename,id,Repository Name,Content View Name,Content View Version"

Comment 10 errata-xmlrpc 2022-04-20 20:34:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: Satellite 6.9.9 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:1478