Bug 1658277

Summary: Provide more verbose error message for pulp failure PulpExecutionException: Importer indicated a failed response
Product: Red Hat Satellite Reporter: sthirugn <sthirugn>
Component: PulpAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Kersom <koliveir>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.3.4CC: bmbouter, daviddavis, dkliban, egolov, ggainey, ipanova, rchan, ttereshc
Target Milestone: 6.6.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pulp-rpm-2.19.0 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-10-22 19:52:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description sthirugn@redhat.com 2018-12-11 16:44:12 UTC 
Description of problem:
Provide more verbose error message for pulp failure PulpExecutionException: Importer indicated a failed response

Version-Release number of selected component (if applicable):
satellite 6.3.5

How reproducible:
Always when pulp not able to find the content upstream

Steps to Reproduce:
1.If upstream doesn't have all relevant content and pulp tries to sync them.  In my case, capsule was trying to get content from satellite some of which doesn't exist in Satellite at all.

Actual results:
Nov 27 17:13:30 chlp2549993 pulp: nectar.downloaders.threaded:INFO: Download failed: Download of https://satellite.example.com/pulp/repos/OrgA/STAGING/CV_CSL3_1_1/content/dist/rhel/server/6/6Server/x86_64/rhscl/1/os/repodata/productid failed with code 404: Not Found
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040) Not Found
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040) Traceback (most recent call last):
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 263, in run
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)     metadata_files = self.get_metadata(metadata_files)
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 450, in get_metadata
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)     metadata_files.download_metadata_files()
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/repomd/metadata.py", line 217, in download_metadata_files
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040)     raise IOError(error_report.error_msg)
Nov 27 17:13:31 chlp2549993 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: [acf7b76c] (23560-27040) IOError: Not Found
Nov 27 17:13:31 chlp2549993 pulp: pulp.server.async.tasks:INFO: [acf7b76c] Task failed : [acf7b76c-61ca-4950-acd4-a6da29500c90]
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040) Task pulp.server.managers.repo.sync.sync[acf7b76c-61ca-4950-acd4-a6da29500c90] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040) Traceback (most recent call last):
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)     R = retval = fun(*args, **kwargs)
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 527, in __call__
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)     return super(Task, self).__call__(*args, **kwargs)
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 107, in __call__
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)     return super(PulpTask, self).__call__(*args, **kwargs)
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 438, in __protected_call__
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)     return self.run(*args, **kwargs)
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 827, in sync
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Nov 27 17:13:31 chlp2549993 pulp: celery.worker.job:ERROR: (23088-27040) PulpExecutionException: Importer indicated a failed response

Expected results:
The Error message in logs is not clear on what exactly pulp is trying to find here. Need more verbose error message on what content is actually missing.

Additional info:
Comment 4 pulp-infra@redhat.com 2018-12-11 17:02:28 UTC 
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.
Comment 5 pulp-infra@redhat.com 2018-12-11 17:02:29 UTC 
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.
Comment 6 pulp-infra@redhat.com 2018-12-11 22:02:48 UTC 
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
Comment 7 pulp-infra@redhat.com 2018-12-12 17:31:40 UTC 
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Comment 8 pulp-infra@redhat.com 2018-12-12 18:01:22 UTC 
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.
Comment 9 pulp-infra@redhat.com 2019-03-20 09:31:56 UTC 
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.
Comment 10 pulp-infra@redhat.com 2019-04-02 21:01:40 UTC 
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Comment 11 jcallaha 2019-05-22 13:57:28 UTC 
Verified in Satellite 6.6 Snap 3

Used the pulp-provided reproducer repo and got a more appropriate error message:
Yum Metadata: Error retrieving metadata: Error 'Not Found' for https://repos.fedorapeople.org/pulp/pulp/fixtures/rpm-missing-filelists/repodata/f015f92b021a6fe869a3bb1d8f1319f5239ada2656e8a6a5e5f56e7c3c93bb70-filelists.xml.gz.
Comment 13 Bryan Kearney 2019-10-22 19:52:22 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3172