Created attachment 898707 [details] Test Repository reflecting suse repomd usage Description of problem: During sync of the repomd repositories of suse the following error "Error: 'NoneType' object has no attribute 'text'" appreas during errata sync. The delta rpms seem to be skipped aswell. Version-Release number of selected component (if applicable): rpm -qa |grep pulp python-pulp-bindings-2.4.0-0.18.beta.el6.noarch pulp-katello-plugins-0.3-1.el6.noarch python-isodate-0.5.0-1.pulp.el6.noarch python-pulp-common-2.4.0-0.18.beta.el6.noarch pulp-nodes-parent-2.4.0-0.18.beta.el6.noarch pulp-admin-client-2.4.0-0.18.beta.el6.noarch pulp-selinux-2.4.0-0.18.beta.el6.noarch createrepo-0.9.9-21.2.pulp.el6.noarch pulp-server-2.4.0-0.18.beta.el6.noarch python-pulp-rpm-common-2.4.0-0.18.beta.el6.noarch python-pulp-puppet-common-2.4.0-0.18.beta.el6.noarch python-rhsm-1.8.0-1.pulp.el6.x86_64 pulp-rpm-plugins-2.4.0-0.18.beta.el6.noarch pulp-puppet-plugins-2.4.0-0.18.beta.el6.noarch mod_wsgi-3.4-1.pulp.el6.x86_64 pulp-rpm-admin-extensions-2.4.0-0.18.beta.el6.noarch pulp-nodes-common-2.4.0-0.18.beta.el6.noarch python-pulp-client-lib-2.4.0-0.18.beta.el6.noarch pulp-puppet-tools-2.4.0-0.18.beta.el6.noarch m2crypto-0.21.1.pulp-8.el6.x86_64 python-kombu-3.0.15-5.pulp.el6.noarch How reproducible: Serve the attached test repository using http and try to sync it into pulp Steps to Reproduce: 1. serve attached test repository using http 2. sync into pulp 3. pulp-admin rpm repo create --repo-id=test6 --display-name="test6" --feed=http://localhost/suse-test-repo/ --relative-url=test6 --serve-http=true --serve-https=true 4. run pulp-admin rpm repo sync run --repo-id=test6 5. output or task details for error Actual results: Sync with rpms & errors Expected results: Sync with drpms/rpms/erratum & no errors - I have verified that its the erratum producing the errors by skipping it in a test. - I have checked that the test repository works, its not properly signed though :) Additional info: [root@axkatello01-1 ~]# pulp-admin rpm repo create --repo-id=test6 --display-name="test6" --feed=http://localhost/suse-test-repo/ --relative-url=test6 --serve-http=true --serve-https=true --skip rpm,drpm,distribution && pulp-admin rpm repo sync run --repo-id=test6 Successfully created repository [test6] +----------------------------------------------------------------------+ Synchronizing Repository [test6] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [|] ... completed Downloading repository content... [==================================================] 100% RPMs: 0/0 items Delta RPMs: 0/0 items ... completed Importing errata... [-] ... failed Task Failed Importer indicated a failed response [root@axkatello01-1 ~]# pulp-admin tasks details --task-id=bf6b2d1b-b156-4083-8931-dd265af3df63 +----------------------------------------------------------------------+ Task Details +----------------------------------------------------------------------+ Operations: sync Resources: test6 (repository) State: Failed Start Time: 2014-05-23T14:38:22Z Finish Time: 2014-05-23T14:38:25Z Result: N/A Task Id: bf6b2d1b-b156-4083-8931-dd265af3df63 Progress Report: Yum Importer: Comps: State: NOT_STARTED Content: Details: Drpm Done: 0 Drpm Total: 0 Rpm Done: 0 Rpm Total: 0 Error Details: Items Left: 0 Items Total: 0 Size Left: 0 Size Total: 0 State: FINISHED Distribution: Error Details: Items Left: 0 Items Total: 0 State: SKIPPED Errata: Error: 'NoneType' object has no attribute 'text' State: FAILED Metadata: State: FINISHED Traceback: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task R = retval = fun(*args, **kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 321, in __call__ return super(Task, self).__call__(*args, **kwargs) File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ return self.run(*args, **kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/tasks/repository .py", line 210, in sync_with_auto_publish sync_result = managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides) File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sy nc.py", line 113, in sync raise PulpExecutionException(_('Importer indicated a failed response')) PulpExecutionException: Importer indicated a failed response [root@axkatello01-1 ~]# Without erratum: [root@axkatello01-1 ~]# pulp-admin rpm repo create --repo-id=test6 --display-name="test6" --feed=http://localhost/suse-test-repo/ --relative-url=test6 --serve-http=true --serve-https=true --skip erratum && pulp-admin rpm repo sync run --repo-id=test6 Successfully created repository [test6] +----------------------------------------------------------------------+ Synchronizing Repository [test6] +----------------------------------------------------------------------+ This command may be exited via ctrl+c without affecting the request. Downloading metadata... [|] ... completed Downloading repository content... [================================ ] 64% RPMs: 1/2 items Delta RPMs: 0/0 items ... completed Downloading distribution files... [==================================================] 100% Distributions: 0/0 items ... completed Importing package groups/categories... [-] ... completed Task Succeeded Initializing repo metadata [-] ... completed Publishing Distribution files [-] ... completed Publishing RPMs [==================================================] 100% 2 of 2 items ... completed Publishing Delta RPMs [-] ... completed Publishing Errata ... skipped Publishing Comps file [-] ... completed Publishing Metadata. [==================================================] 100% 2 of 2 items ... completed Closing repo metadata [-] ... completed Publishing files to web [-] ... completed Task Succeeded
More debug output: Jul 7 13:08:00 katello01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._reserve_resource[a25f745e-d882-41aa-93fb-4f562c442942] Jul 7 13:08:01 katello01 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._reserve_resource[a25f745e-d882-41aa-93fb-4f562c442942] succeeded in 0.027217833s: u'res erved_resource_worker-0.dq' Jul 7 13:08:01 katello01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.tasks.repository.sync_with_auto_publish[ed248a84-19a2-40e4-9044-39b2816ec114] Jul 7 13:08:01 katello01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._queue_release_resource[1b66ef9a-15ed-4122-814f-8fed46c56878] Jul 7 13:08:01 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata from http://localhost/suse-test-repo/. Jul 7 13:08:01 katello01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): localhost Jul 7 13:08:02 katello01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): localhost Jul 7 13:08:02 katello01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): localhost Jul 7 13:08:02 katello01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): localhost Jul 7 13:08:02 katello01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): localhost Jul 7 13:08:02 katello01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTP connection (1): localhost Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Generating metadata databases. Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded. Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading 0 RPMs. Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units. Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: Traceback (most recent call last): Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 133, in run Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: self.get_errata(metadata_files) Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 462, in get _errata Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: self.save_fileless_units(errata_file_handle, updateinfo.PACKAGE_TAG, updateinfo.process_packag e_element) Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 522, in save_fileless_units Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: to_save = existing.check_repo(wanted, self.sync_conduit.get_units) Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/existing.py", line 37, in check_repo Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: sorted_units = _sort_by_type(wanted) Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/existing.py", line 162, in _sort_by_type Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: for unit in wanted: Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 520, in <genexpr> Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: wanted = (model.as_named_tuple for model in package_info_generator) Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 63, in package_list_generator Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: package_info = process_func(element) Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/updateinfo.py", line 39, in process_package_element Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: 'pkglist': map(_parse_collection, element.find('pkglist') or []), Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/updateinfo.py", line 82, in _parse_collection Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: 'name': element.find('name').text, Jul 7 13:08:03 katello01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: AttributeError: 'NoneType' object has no attribute 'text' Jul 7 13:08:03 katello01 pulp: pulp.server.event.http:INFO: {'call_report': {u'exception': None, u'task_type': u'pulp.server.tasks.repository.sync_with_auto_publish', u'task_id': u'ed248a84-19a2-40e4-9044-39b2816ec114', u'tags': [u'pulp:repository:test6', u'pulp:action:sync'], u'finish_time': None, u'_ns': u'task_status', u'start_time': u'2014-07-07T11:08:01Z', u'traceback': None, u'spawned_tasks': [], u'progress_report': {u'yum_importer': {u'content': {u'size_total': 0, u'items_left': 0, u'items_total': 0, u'state': u'FINISHED', u'size_left': 0, u'details': {u'rpm_total': 0, u'rpm_done': 0, u'drpm_total': 0, u'drpm_done': 0}, u'error_details': []}, u'comps': {u'state': u'NOT_STARTED'}, u'distribution': {u'items_total': 0, u'state': u'SKIPPED', u'error_details': [], u'items_left': 0}, u'errata': {u'state': u'FAILED', u'error': u"'NoneType' object has no attribute 'text'"}, u'metadata': {u'state': u'FINISHED'}}}, u'queue': u'reserved_resource_worker-0.dq', u'state': u'running', u'result': None, u'error': None, u'_id': ObjectId('53ba7f911fd87d1390414324'), u'id': u'53ba7f910ab73b04e99340b0'}, 'event_type': 'repo.sync.finish', 'payload': {'importer_id': u'yum_importer', 'exception': None, 'repo_id': u'test6', 'traceback': None, 'started': '2014-07-07T13:08:01+02:00', '_ns': u'repo_sync_results', 'completed': '2014-07-07T13:08:03+02:00', 'importer_type_id': u'yum_importer', 'error_message': None, 'summary': {'content': {'state': 'FINISHED'}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'state': 'SKIPPED'}, 'errata': {'state': 'FAILED'}, 'metadata': {'state': 'FINISHED'}}, 'added_count': 2, 'result': 'failed', 'updated_count': 0, 'details': {'content': {'size_total': 0, 'items_left': 0, 'items_total': 0, 'state': 'FINISHED', 'size_left': 0, 'details': {'rpm_total': 0, 'rpm_done': 0, 'drpm_total': 0, 'drpm_done': 0}, 'error_details': []}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'items_total': 0, 'state': 'SKIPPED', 'error_details': [], 'items_left': 0}, 'errata': {'state': 'FAILED', 'error': "'NoneType Jul 7 13:08:03 katello01 pulp: pulp.server.event.http:INFO: ' object has no attribute 'text'"}, 'metadata': {'state': 'FINISHED'}}, 'id': '53ba7f930ab73b06037945ab', 'removed_count': 0}} Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: Task pulp.server.tasks.repository.sync_with_auto_publish[ed248a84-19a2-40e4-9044-39b2816ec114] raised unexpected: PulpExecutionException('Importer indicated a failed response',) Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: Traceback (most recent call last): Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: R = retval = fun(*args, **kwargs) Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 306, in __call__ Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: return super(Task, self).__call__(*args, **kwargs) Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__ Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: return self.run(*args, **kwargs) Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/tasks/repository.py", line 210, in sync_with_auto_publish Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: sync_result = managers.repo_sync_manager().sync(repo_id, sync_config_override=overrides) Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 113, in sync Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: raise PulpExecutionException(_('Importer indicated a failed response')) Jul 7 13:08:03 katello01 pulp: celery.worker.job:ERROR: PulpExecutionException: Importer indicated a failed response Jul 7 13:08:03 katello01 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[346de107-6731-41c7-953c-aaa872bdee79] Jul 7 13:08:03 katello01 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_release_resource[1b66ef9a-15ed-4122-814f-8fed46c56878] succeeded in 0.0755167460001s: None Jul 7 13:08:03 katello01 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[346de107-6731-41c7-953c-aaa872bdee79] succeeded in 0.0414652109999s: None Jul 7 13:08:05 katello01 pulp: pulp.server.event.http:WARNING: Error response from HTTP notifier: {"displayMessage":"Couldn't find repository 'test6'","errors":["Couldn't find repository 'test6'"]}
collection in pkglist has no name attribute in that SUSE repo, that's what the parser doesn't like.
corresponding SUSE bug at https://bugzilla.novell.com/show_bug.cgi?id=886444
Possible fix for errata import: https://github.com/pulp/pulp_rpm/pull/525
Seperate BZ #1120215 opened for the delta RPM problem.
The errata sync appears to have already been fixed in the 2.6
Ah, yes... https://github.com/pulp/pulp_rpm/pull/568 was merged.
https://github.com/pulp/pulp_rpm/pull/629
Moved to https://pulp.plan.io/issues/435