Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1013681 - Tooltip for failed sync has only one line from stacktrace
Summary: Tooltip for failed sync has only one line from stacktrace
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: WebUI
Version: 6.0.2
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: Unspecified
Assignee: Eric Helms
QA Contact: Sachin Ghai
URL:
Whiteboard:
: 1010096 1025005 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-30 15:23 UTC by Og Maciel
Modified: 2014-03-11 16:43 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-03-11 16:43:59 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Tooltip screenshot (1.55 MB, image/png)
2013-09-30 15:23 UTC, Og Maciel
no flags Details
no logs under notifications (63.30 KB, image/png)
2013-10-21 12:00 UTC, Sachin Ghai
no flags Details
tool_tip says 'Not Found' (87.48 KB, image/png)
2013-10-21 12:01 UTC, Sachin Ghai
no flags Details

Description Og Maciel 2013-09-30 15:23:56 UTC
Created attachment 805257 [details]
Tooltip screenshot

Description of problem:

The attached screenshot shows that the tooltip for my failed sync process contains one single line of what could possibly be many lines of a stacktrace. Also, there's nothing shown on Notifications page.

Version-Release number of selected component (if applicable):

* apr-util-ldap-1.3.9-3.el6_0.1.x86_64
* candlepin-0.8.25-1.el6sam.noarch
* candlepin-scl-1-5.el6_4.noarch
* candlepin-scl-quartz-2.1.5-5.el6_4.noarch
* candlepin-scl-rhino-1.7R3-1.el6_4.noarch
* candlepin-scl-runtime-1-5.el6_4.noarch
* candlepin-selinux-0.8.25-1.el6sam.noarch
* candlepin-tomcat6-0.8.25-1.el6sam.noarch
* elasticsearch-0.19.9-8.el6sat.noarch
* foreman-1.3.0-6.el6sat.noarch
* foreman-compute-1.3.0-6.el6sat.noarch
* foreman-libvirt-1.3.0-6.el6sat.noarch
* foreman-postgresql-1.3.0-6.el6sat.noarch
* foreman-proxy-1.3.0-1.el6sat.noarch
* katello-1.4.6-14.el6sat.noarch
* katello-all-1.4.6-14.el6sat.noarch
* katello-candlepin-cert-key-pair-1.0-1.noarch
* katello-certs-tools-1.4.4-1.el6sat.noarch
* katello-cli-1.4.3-15.el6sat.noarch
* katello-cli-common-1.4.3-15.el6sat.noarch
* katello-common-1.4.6-14.el6sat.noarch
* katello-configure-1.4.5-8.el6sat.noarch
* katello-configure-foreman-1.4.5-8.el6sat.noarch
* katello-configure-foreman-proxy-1.4.5-8.el6sat.noarch
* katello-foreman-all-1.4.6-14.el6sat.noarch
* katello-glue-candlepin-1.4.6-14.el6sat.noarch
* katello-glue-elasticsearch-1.4.6-14.el6sat.noarch
* katello-glue-pulp-1.4.6-14.el6sat.noarch
* katello-qpid-broker-key-pair-1.0-1.noarch
* katello-qpid-client-key-pair-1.0-1.noarch
* katello-selinux-1.4.4-4.el6sat.noarch
* openldap-2.4.23-31.el6.x86_64
* pulp-katello-plugins-0.2-1.el6sat.noarch
* pulp-nodes-common-2.3.0-0.14.alpha.el6sat.noarch
* pulp-nodes-parent-2.3.0-0.14.alpha.el6sat.noarch
* pulp-puppet-plugins-2.3.0-0.14.alpha.el6sat.noarch
* pulp-rpm-plugins-2.3.0-0.14.alpha.el6sat.noarch
* pulp-selinux-2.3.0-0.14.alpha.el6sat.noarch
* pulp-server-2.3.0-0.14.alpha.el6sat.noarch
* python-ldap-2.3.10-1.el6.x86_64
* ruby193-rubygem-ldap_fluff-0.2.2-2.el6sat.noarch
* ruby193-rubygem-net-ldap-0.3.1-3.el6sat.noarch
* ruby193-rubygem-runcible-1.0.7-1.el6sat.noarch
* signo-0.0.20-3.el6sat.noarch
* signo-katello-0.0.20-3.el6sat.noarch

How reproducible:


Steps to Reproduce:
1. Try to make a repo sync fail (perhaps create a repo with an invalid url?). I had all of RHEL 5 and 6 for i386, ppc, s390 and x86_64 enabled and syncing fwiw
2.
3.

Actual results:

Tooltip shows only one line

Expected results:


Additional info:

Comment 1 Og Maciel 2013-09-30 15:28:44 UTC
Here's what the entire stacktrace shows:

2013-09-30 00:31:09,528 urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com
2013-09-30 00:31:09,898 urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com
2013-09-30 00:31:10,528 urllib3.connectionpool:WARNING: Retrying (0 attempts remain) after connection broken by 'BadStatusLine('',)': /content/dist/rhel/power/6/6Server/ppc64/os/Packages/python-decorator-3.0.1-3.1.el6.noarch.rpm
2013-09-30 00:31:10,532 nectar.downloaders.threaded:ERROR: HTTPSConnectionPool(host='cdn.redhat.com', port=443): Max retries exceeded with url: /content/dist/rhel/power/6/6Server/ppc64/os/Packages/python-decorator-3.0.1-3.1.el6.noarch.rpm
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/nectar/downloaders/threaded.py", line 160, in _fetch
    response = session.get(request.url, headers=headers)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 310, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 279, in request
    resp = self.send(prep, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies)
  File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 374, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 209, in send
    raise ConnectionError(e)
ConnectionError: HTTPSConnectionPool(host='cdn.redhat.com', port=443): Max retries exceeded with url: /content/dist/rhel/power/6/6Server/ppc64/os/Packages/python-decorator-3.0.1-3.1.el6.noarch.rpm
2013-09-30 00:31:10,539 urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com
2013-09-30 00:31:11,315 urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com
2013-09-30 00:31:13,353 urllib3.connectionpool:INFO: Resetting dropped connection: cdn.redhat.com
2013-09-30 00:37:37,695 urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
2013-09-30 00:37:38,049 nectar.downloaders.threaded:ERROR: Download of https://cdn.redhat.com/content/dist/rhel/power/6/6Server/ppc64/os/.treeinfo failed with code 404: Not Found

Comment 3 Eric Helms 2013-10-02 20:11:09 UTC
Tracking upstream - https://github.com/Katello/katello/pull/3069

Comment 4 Eric Helms 2013-10-02 20:11:26 UTC
*** Bug 1010096 has been marked as a duplicate of this bug. ***

Comment 5 Eric Helms 2013-10-03 12:30:34 UTC
The fix is available as of ad5d5b72eb788c0d52da234c571bd1b4ca44d3a2

This fix updates the algorithm for parsing pulp sync status objects and updates the tooltip error output. Further, the tooltip is now a clickable link to the notifications page. 

This fix also re-enables the after-sync notification handling from Pulp. The user will once again be presented with a notification upon completion of a sync. And, if there are errors, a detailed view of those errors will be present on the notifications page.

Comment 8 Sachin Ghai 2013-10-21 12:00:04 UTC
Verified with snap6 compose2 (MDP2)

katello-qpid-client-key-pair-1.0-1.noarch
katello-glue-candlepin-1.4.6-40.el6sat.noarch
katello-1.4.6-40.el6sat.noarch
katello-foreman-all-1.4.6-40.el6sat.noarch
katello-candlepin-cert-key-pair-1.0-1.noarch
katello-configure-1.4.7-6.el6sat.noarch
signo-katello-0.0.23-2.el6sat.noarch
katello-glue-elasticsearch-1.4.6-40.el6sat.noarch
ruby193-rubygem-katello-foreman-engine-0.0.12-3.el6sat.noarch
katello-cli-1.4.3-24.el6sat.noarch
katello-certs-tools-1.4.4-1.el6sat.noarch
katello-qpid-broker-key-pair-1.0-1.noarch
ruby193-rubygem-foreman-katello-engine-0.0.17-6.el6sat.noarch
katello-common-1.4.6-40.el6sat.noarch
pulp-katello-plugins-0.2-1.el6sat.noarch
katello-configure-foreman-1.4.7-6.el6sat.noarch
katello-selinux-1.4.4-4.el6sat.noarch
katello-cli-common-1.4.3-24.el6sat.noarch
katello-glue-pulp-1.4.6-40.el6sat.noarch
katello-all-1.4.6-40.el6sat.noarch
ruby193-rubygem-katello_api-0.0.3-4.el6sat.noarch


Sync failed and now tool tip just says 'Not Found' and when I moved to notices page by clicking on it, there were no error listed. 

no stack trace in production.log, it says:
[ERROR 2013-10-21 11:48:24 app  #5873] Task PulpSyncStatus (22) is in error state
[ERROR 2013-10-21 11:48:24 app  #5873] *** Sync error:

Comment 9 Sachin Ghai 2013-10-21 12:00:57 UTC
Created attachment 814554 [details]
no logs under notifications

Comment 10 Sachin Ghai 2013-10-21 12:01:52 UTC
Created attachment 814556 [details]
tool_tip says 'Not Found'

Comment 11 Sachin Ghai 2013-10-21 12:02:51 UTC
Pulp.log says the exact error:

--
pulp.log says:

2013-10-21 11:48:21,470 pulp_rpm.plugins.importers.yum.sync:ERROR: sync failed
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 109, in run
    metadata_files = self.get_metadata()
  File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 194, in get_metadata
    raise FailedException(str(e))
FailedException: Not Found
2013-10-21 11:48:21,472 pulp.server.event.http:INFO: {'call_report': {'task_group_id': '656d1c8d-bd27-498a-96ac-9003f6a4499a', 'exception': None, 'traceback': None, 'task_id': 'a2665234-6067-46b4-b8a1-447e31dc2767', 'call_request_tags': [u'pulp:repository:ACME_Corporation-product1-zoo', 'pulp:action:sync'], 'reasons': [], 'start_time': '2013-10-21T06:18:20Z', 'tags': [u'pulp:repository:ACME_Corporation-product1-zoo', 'pulp:action:sync'], 'state': 'running', 'finish_time': None, 'dependency_failures': {}, 'schedule_id': None, 'progress': {u'yum_importer': {'content': {'size_total': 0, 'items_left': 0, 'items_total': 0, 'state': 'NOT_STARTED', 'size_left': 0, 'details': {'rpm_total': 0, 'rpm_done': 0, 'drpm_total': 0, 'drpm_done': 0}, 'error_details': []}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'items_total': 0, 'state': 'NOT_STARTED', 'error_details': [], 'items_left': 0}, 'errata': {'state': 'NOT_STARTED'}, 'metadata': {'state': 'FAILED', 'error': 'Not Found'}}}, 'call_request_group_id': '656d1c8d-bd27-498a-96ac-9003f6a4499a', 'call_request_id': 'a2665234-6067-46b4-b8a1-447e31dc2767', 'principal_login': u'admin', 'response': 'accepted', 'result': None}, 'event_type': 'repo.sync.finish', 'payload': {'importer_id': u'yum_importer', 'exception': None, 'repo_id': u'ACME_Corporation-product1-zoo', 'traceback': None, 'started': '2013-10-21T11:48:20+05:30', '_ns': u'repo_sync_results', 'completed': '2013-10-21T11:48:21+05:30', 'importer_type_id': u'yum_importer', 'error_message': None, 'summary': {'content': {'state': 'NOT_STARTED'}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'state': 'NOT_STARTED'}, 'errata': {'state': 'NOT_STARTED'}, 'metadata': {'state': 'FAILED'}}, 'added_count': 0, 'result': 'failed', 'updated_count': 0, 'details': {'content': {'size_total': 0, 'items_left': 0, 'items_total': 0, 'state': 'NOT_STARTED', 'size_left': 0, 'details': {'rpm_total': 0, 'rpm_done': 0, 'drpm_total': 0, 'drpm_done': 0}, 'error_details': []}, 'comps': {'state': 'NOT_STARTED'}, 'distribution': {'items_total': 0, 'state': 'NOT_STARTED', 'error_details': [], 'items_left': 0}, 'errata': {'state': 'NOT_STARTED'}, 'metadata': {'state': 'FAILED', 'error': 'Not Found'}}, 'id': '5264c72dacdb986ee80004c5', 'removed_count': 0}}
2013-10-21 11:48:21,474 pulp.server.dispatch.task:ERROR: Importer indicated a failed response
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 116, in sync
    raise PulpExecutionException(_('Importer indicated a failed response'))
PulpExecutionException: Importer indicated a failed response
2013-10-21 11:48:21,478 pulp.server.dispatch.task:INFO: FAILURE: Task a2665234-6067-46b4-b8a1-447e31dc2767: CallRequest: RepoSyncManager.sync(u'ACME_Corporation-product1-zoo', sync_config_override={'num_threads': 4})
2013-10-21 11:48:24,965 pulp.server.dispatch.task:INFO: SUCCESS: Task 7dc8ceaf-8e32-426e-9faf-cf7eb359d246: CallRequest: ApplicabilityRegenerationManager.regenerate_applicability_for_repos({'sort': None, 'skip': None, 'limit': None, 'filters': {'id': {'$in': ['ACME_Corporation-product1-zoo']}}, 'fields': ['id'], '_id': ObjectId('5264c730acdb986ee80004cc'), 'id': '5264c730acdb986ee80004cc'})

Comment 12 Sachin Ghai 2013-10-21 12:06:58 UTC
Hummn. so from comment11, I got to know that the repo url was incorrect. But still I think user should get the error on UI/notifications which is logged in pulp.log ?

Comment 13 Eric Helms 2013-10-22 03:23:39 UTC
We do not have access to what the pulp.log file contains, only that which Pulp returns to us in the task data. In this case, all we know is:

'metadata': {'state': 'FAILED', 'error': 'Not Found'}}}

And the best we can do right now, is parse that out from the output above and present it to the user. In my opinion, this is doing the best we can right now, and further improvements need to be made on the Pulp side with regards to error reporting in the future.

Comment 14 Mike McCune 2013-11-01 20:09:15 UTC
*** Bug 1025005 has been marked as a duplicate of this bug. ***

Comment 15 Bryan Kearney 2014-03-11 16:43:59 UTC
Closing old bugs which are not relevant based on new UI and CLI


Note You need to log in before you can comment on or make changes to this bug.