Hide Forgot
Description of problem: A subtask for CFME ISOs failed while syncing content in RHEV+CFME deployment with error message: "PLP0000: Importer indicated a failed response" /var/log/pulp is empty. From messages: " Mar 23 05:27:57 localhost pulp: pulp.server.event.http:INFO: (18631-89024) {'call_report': {u'exception': None, u'task_type': u'pulp.server.managers.repo.sync.sync', u'task_id': u'53aaee8e-0607-48af-857f-e3ac52eb488a', u'tags': [u'pulp:repository:Default_Organization-Red_Hat_CloudForms-Red_Hat_CloudForms_Management_Engine_5_5_Files_x86_64', u'pulp:action:sync'], u'finish_time': None, u'_ns': u'task_status', u'start_time': u'2016-03-23T09:24:23Z', u'traceback': None, u'spawned_tasks': [], u'progress_report': {u'iso_importer': {u'error_message': None, u'traceback': None, u'finished_bytes': 7339439532L, u'num_isos': 8, u'state': u'isos_failed', u'iso_error_messages': [{u'name': u'cfme-vsphere-5.5.0.13-2.x86_64.vsphere.ova', u'error': {}}], u'state_times': {u'not_started': u'2016-03-23T09:24:23', u'manifest_in_progress': u'2016-03-23T09:24:23', u'isos_failed': u'2016-03-23T09:27:57', u'isos_in_progress': u'2016-03-23T09:24:24'}, u'num_isos_finished': 7, u'total_bytes': 7980279212L}}, u'state': u'running', u'worker_name': u'reserved_resource_worker-3.lab.eng.rdu2.redhat.com', u'result': None, u'error': None, u'_id': ObjectId('56f25f1c633fddd4fa8e29ba'), u'id': u'56f260c71191b048988befda'}, 'event_type': 'repo.sync.finish', 'payload': {'importer_id': u'iso_importer', 'exception': None, 'repo_id': u'Default_Organization-Red_Hat_CloudForms-Red_Hat_CloudForms_Management_Engine_5_5_Files_x86_64', 'traceback': None, 'started': '2016-03-23T09:24:23Z', '_ns': u'repo_sync_results', 'completed': '2016-03-23T09:27:57Z', 'importer_type_id': u'iso_importer', 'error_message': None, 'summary': {'traceback': None, 'error_message': None, 'finished_bytes': 7339439532, 'num_isos': 8, 'state': 'isos_failed', 'iso_error_messages': [{'name': 'cfme-vsphere-5.5.0.13-2.x86_64.vsphere.ova', 'error': {}}], 'total_bytes': 7980279212, 'num_isos_finished': 7, 'state_times': {'not_started': '2016-03-23T09:24:23', 'manifest_in_progress': '2016-03-23T09:24:23', 'isos_failed': '2016-03-23T09:27:57', 'isos_in_progress': '2016-03-23T09:24:24'}}, 'added_count': Mar 23 05:27:57 localhost pulp: pulp.server.event.http:INFO: (18631-89024) 7, 'result': 'failed', 'updated_count': 0, 'details': None, 'id': '56f2619d1191b048c700e4e6', 'removed_count': 0}} Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) Task pulp.server.managers.repo.sync.sync[53aaee8e-0607-48af-857f-e3ac52eb488a] raised unexpected: PulpExecutionException('Importer indicated a failed response',) Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) Traceback (most recent call last): Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) R = retval = fun(*args, **kwargs) Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 328, in __call__ Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) return super(Task, self).__call__(*args, **kwargs) Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__ Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) return self.run(*args, **kwargs) Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) raise PulpExecutionException(_('Importer indicated a failed response')) Mar 23 05:27:57 localhost pulp: celery.worker.job:ERROR: (18502-89024) PulpExecutionException: Importer indicated a failed response " Attaching excerpts from production.log and the error tab in the UI. Version-Release number of selected component (if applicable): TP3 RC2 How reproducible: Happened to me once Steps to Reproduce: 1. Start a RHEV+CFME deployment 2. Some time into the content sync the subtask for CFME ISOs fails 3. Actual results: Sync subtask for CFME ISOs fails Expected results: Sync completes successfully Additional info:
Created attachment 1139448 [details] relevant part of production.log
Created attachment 1139450 [details] errors tab from the ui
The new Redeploy button worked and I'm now seeing completed content sync for CFME.
I've seen this problem in the past when there was an issue with metadata on the CDN for file repos. We hit a similar issue with OSE syncing when there was an issue with Akamai cache, rel-eng flushed the cache and it fixed the issue.
The Content Sync resume was added to workaround these issues.