Hide Forgot
Created attachment 805257 [details] Tooltip screenshot Description of problem: The attached screenshot shows that the tooltip for my failed sync process contains one single line of what could possibly be many lines of a stacktrace. Also, there's nothing shown on Notifications page. Version-Release number of selected component (if applicable): * apr-util-ldap-1.3.9-3.el6_0.1.x86_64 * candlepin-0.8.25-1.el6sam.noarch * candlepin-scl-1-5.el6_4.noarch * candlepin-scl-quartz-2.1.5-5.el6_4.noarch * candlepin-scl-rhino-1.7R3-1.el6_4.noarch * candlepin-scl-runtime-1-5.el6_4.noarch * candlepin-selinux-0.8.25-1.el6sam.noarch * candlepin-tomcat6-0.8.25-1.el6sam.noarch * elasticsearch-0.19.9-8.el6sat.noarch * foreman-1.3.0-6.el6sat.noarch * foreman-compute-1.3.0-6.el6sat.noarch * foreman-libvirt-1.3.0-6.el6sat.noarch * foreman-postgresql-1.3.0-6.el6sat.noarch * foreman-proxy-1.3.0-1.el6sat.noarch * katello-1.4.6-14.el6sat.noarch * katello-all-1.4.6-14.el6sat.noarch * katello-candlepin-cert-key-pair-1.0-1.noarch * katello-certs-tools-1.4.4-1.el6sat.noarch * katello-cli-1.4.3-15.el6sat.noarch * katello-cli-common-1.4.3-15.el6sat.noarch * katello-common-1.4.6-14.el6sat.noarch * katello-configure-1.4.5-8.el6sat.noarch * katello-configure-foreman-1.4.5-8.el6sat.noarch * katello-configure-foreman-proxy-1.4.5-8.el6sat.noarch * katello-foreman-all-1.4.6-14.el6sat.noarch * katello-glue-candlepin-1.4.6-14.el6sat.noarch * katello-glue-elasticsearch-1.4.6-14.el6sat.noarch * katello-glue-pulp-1.4.6-14.el6sat.noarch * katello-qpid-broker-key-pair-1.0-1.noarch * katello-qpid-client-key-pair-1.0-1.noarch * katello-selinux-1.4.4-4.el6sat.noarch * openldap-2.4.23-31.el6.x86_64 * pulp-katello-plugins-0.2-1.el6sat.noarch * pulp-nodes-common-2.3.0-0.14.alpha.el6sat.noarch * pulp-nodes-parent-2.3.0-0.14.alpha.el6sat.noarch * pulp-puppet-plugins-2.3.0-0.14.alpha.el6sat.noarch * pulp-rpm-plugins-2.3.0-0.14.alpha.el6sat.noarch * pulp-selinux-2.3.0-0.14.alpha.el6sat.noarch * pulp-server-2.3.0-0.14.alpha.el6sat.noarch * python-ldap-2.3.10-1.el6.x86_64 * ruby193-rubygem-ldap_fluff-0.2.2-2.el6sat.noarch * ruby193-rubygem-net-ldap-0.3.1-3.el6sat.noarch * ruby193-rubygem-runcible-1.0.7-1.el6sat.noarch * signo-0.0.20-3.el6sat.noarch * signo-katello-0.0.20-3.el6sat.noarch How reproducible: Steps to Reproduce: 1. Try to make a repo sync fail (perhaps create a repo with an invalid url?). I had all of RHEL 5 and 6 for i386, ppc, s390 and x86_64 enabled and syncing fwiw 2. 3. Actual results: Tooltip shows only one line Expected results: Additional info:
Here's what the entire stacktrace shows: 2013-09-30 00:31:09,528 urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com 2013-09-30 00:31:09,898 urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com 2013-09-30 00:31:10,528 urllib3.connectionpool:WARNING: Retrying (0 attempts remain) after connection broken by 'BadStatusLine('',)': /content/dist/rhel/power/6/6Server/ppc64/os/Packages/python-decorator-3.0.1-3.1.el6.noarch.rpm 2013-09-30 00:31:10,532 nectar.downloaders.threaded:ERROR: HTTPSConnectionPool(host='cdn.redhat.com', port=443): Max retries exceeded with url: /content/dist/rhel/power/6/6Server/ppc64/os/Packages/python-decorator-3.0.1-3.1.el6.noarch.rpm Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 160, in _fetch response = session.get(request.url, headers=headers) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 310, in get return self.request('GET', url, **kwargs) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 279, in request resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies) File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 374, in send r = adapter.send(request, **kwargs) File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 209, in send raise ConnectionError(e) ConnectionError: HTTPSConnectionPool(host='cdn.redhat.com', port=443): Max retries exceeded with url: /content/dist/rhel/power/6/6Server/ppc64/os/Packages/python-decorator-3.0.1-3.1.el6.noarch.rpm 2013-09-30 00:31:10,539 urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com 2013-09-30 00:31:11,315 urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com 2013-09-30 00:31:13,353 urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com 2013-09-30 00:37:37,695 urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com 2013-09-30 00:37:38,049 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/power/6/6Server/ppc64/os/.treeinfo failed with code 404: Not Found
Tracking upstream - https://github.com/Katello/katello/pull/3069
*** Bug 1010096 has been marked as a duplicate of this bug. ***
The fix is available as of ad5d5b72eb788c0d52da234c571bd1b4ca44d3a2 This fix updates the algorithm for parsing pulp sync status objects and updates the tooltip error output. Further, the tooltip is now a clickable link to the notifications page. This fix also re-enables the after-sync notification handling from Pulp. The user will once again be presented with a notification upon completion of a sync. And, if there are errors, a detailed view of those errors will be present on the notifications page.
Verified with snap6 compose2 (MDP2) katello-qpid-client-key-pair-1.0-1.noarch katello-glue-candlepin-1.4.6-40.el6sat.noarch katello-1.4.6-40.el6sat.noarch katello-foreman-all-1.4.6-40.el6sat.noarch katello-candlepin-cert-key-pair-1.0-1.noarch katello-configure-1.4.7-6.el6sat.noarch signo-katello-0.0.23-2.el6sat.noarch katello-glue-elasticsearch-1.4.6-40.el6sat.noarch ruby193-rubygem-katello-foreman-engine-0.0.12-3.el6sat.noarch katello-cli-1.4.3-24.el6sat.noarch katello-certs-tools-1.4.4-1.el6sat.noarch katello-qpid-broker-key-pair-1.0-1.noarch ruby193-rubygem-foreman-katello-engine-0.0.17-6.el6sat.noarch katello-common-1.4.6-40.el6sat.noarch pulp-katello-plugins-0.2-1.el6sat.noarch katello-configure-foreman-1.4.7-6.el6sat.noarch katello-selinux-1.4.4-4.el6sat.noarch katello-cli-common-1.4.3-24.el6sat.noarch katello-glue-pulp-1.4.6-40.el6sat.noarch katello-all-1.4.6-40.el6sat.noarch ruby193-rubygem-katello_api-0.0.3-4.el6sat.noarch Sync failed and now tool tip just says 'Not Found' and when I moved to notices page by clicking on it, there were no error listed. no stack trace in production.log, it says: [ERROR 2013-10-21 11:48:24 app #5873] Task PulpSyncStatus (22) is in error state [ERROR 2013-10-21 11:48:24 app #5873] *** Sync error:
Created attachment 814554 [details] no logs under notifications
Created attachment 814556 [details] tool_tip says 'Not Found'
Pulp.log says the exact error: -- pulp.log says: 2013-10-21 11:48:21,470 pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 109, in run metadata_files = self.get_metadata() File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 194, in get_metadata raise FailedException(str(e)) FailedException: Not Found 2013-10-21 11:48:21,472 pulp.server.event.http:INFO: {'call_report': {'task_group_id': '656d1c8d-bd27-498a-96ac-9003f6a4499a', 'exception': None, 'traceback': None, 'task_id': 'a2665234-6067-46b4-b8a1-447e31dc2767', 'call_request_tags': [u'pulp:repository:ACME_Corporation-product1-zoo', 'pulp:action:sync'], 'reasons': [], 'start_time': '2013-10-21T06:18:20Z', 'tags': [u'pulp:repository:ACME_Corporation-product1-zoo', 'pulp:action:sync'], 'state': 'running', 'finish_time': None, 'dependency_failures': {}, 'schedule_id': None, 'progress': {u'yum_importer': {'content': {'size_total': 0, 'items_left': 0, 'items_total': 0, 'state': 'NOT_STARTED', 'size_left': 0, 'details': {'rpm_total': 0, 'rpm_done': 0, 'drpm_total': 0, 'drpm_done': 0}, 'error_details': []}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'items_total': 0, 'state': 'NOT_STARTED', 'error_details': [], 'items_left': 0}, 'errata': {'state': 'NOT_STARTED'}, 'metadata': {'state': 'FAILED', 'error': 'Not Found'}}}, 'call_request_group_id': '656d1c8d-bd27-498a-96ac-9003f6a4499a', 'call_request_id': 'a2665234-6067-46b4-b8a1-447e31dc2767', 'principal_login': u'admin', 'response': 'accepted', 'result': None}, 'event_type': 'repo.sync.finish', 'payload': {'importer_id': u'yum_importer', 'exception': None, 'repo_id': u'ACME_Corporation-product1-zoo', 'traceback': None, 'started': '2013-10-21T11:48:20+05:30', '_ns': u'repo_sync_results', 'completed': '2013-10-21T11:48:21+05:30', 'importer_type_id': u'yum_importer', 'error_message': None, 'summary': {'content': {'state': 'NOT_STARTED'}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'state': 'NOT_STARTED'}, 'errata': {'state': 'NOT_STARTED'}, 'metadata': {'state': 'FAILED'}}, 'added_count': 0, 'result': 'failed', 'updated_count': 0, 'details': {'content': {'size_total': 0, 'items_left': 0, 'items_total': 0, 'state': 'NOT_STARTED', 'size_left': 0, 'details': {'rpm_total': 0, 'rpm_done': 0, 'drpm_total': 0, 'drpm_done': 0}, 'error_details': []}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'items_total': 0, 'state': 'NOT_STARTED', 'error_details': [], 'items_left': 0}, 'errata': {'state': 'NOT_STARTED'}, 'metadata': {'state': 'FAILED', 'error': 'Not Found'}}, 'id': '5264c72dacdb986ee80004c5', 'removed_count': 0}} 2013-10-21 11:48:21,474 pulp.server.dispatch.task:ERROR: Importer indicated a failed response Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 138, in _run result = call(*args, **kwargs) File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 116, in sync raise PulpExecutionException(_('Importer indicated a failed response')) PulpExecutionException: Importer indicated a failed response 2013-10-21 11:48:21,478 pulp.server.dispatch.task:INFO: FAILURE: Task a2665234-6067-46b4-b8a1-447e31dc2767: CallRequest: RepoSyncManager.sync(u'ACME_Corporation-product1-zoo', sync_config_override={'num_threads': 4}) 2013-10-21 11:48:24,965 pulp.server.dispatch.task:INFO: SUCCESS: Task 7dc8ceaf-8e32-426e-9faf-cf7eb359d246: CallRequest: ApplicabilityRegenerationManager.regenerate_applicability_for_repos({'sort': None, 'skip': None, 'limit': None, 'filters': {'id': {'$in': ['ACME_Corporation-product1-zoo']}}, 'fields': ['id'], '_id': ObjectId('5264c730acdb986ee80004cc'), 'id': '5264c730acdb986ee80004cc'})
Hummn. so from comment11, I got to know that the repo url was incorrect. But still I think user should get the error on UI/notifications which is logged in pulp.log ?
We do not have access to what the pulp.log file contains, only that which Pulp returns to us in the task data. In this case, all we know is: 'metadata': {'state': 'FAILED', 'error': 'Not Found'}}} And the best we can do right now, is parse that out from the output above and present it to the user. In my opinion, this is doing the best we can right now, and further improvements need to be made on the Pulp side with regards to error reporting in the future.
*** Bug 1025005 has been marked as a duplicate of this bug. ***
Closing old bugs which are not relevant based on new UI and CLI