Bug 1229322 - database error: too much data for sort() with no index. add an index or specify a smaller limit
Summary: database error: too much data for sort() with no index. add an index or spec...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Pulp
Version: 6.0.8
Hardware: Unspecified
OS: Unspecified
unspecified
high vote
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On:
Blocks: sat61-release-notes
TreeView+ depends on / blocked
 
Reported: 2015-06-08 13:07 UTC by Kenny Tordeurs
Modified: 2019-09-26 13:54 UTC (History)
15 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-05 13:06:41 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Pulp Redmine 1043 High CLOSED - CURRENTRELEASE Pulp's repo sync and publish history is sorted by start_time, which is not indexed 2016-10-05 13:02:46 UTC

Comment 1 RHEL Product and Program Management 2015-06-08 13:12:48 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 3 Michael Hrivnak 2015-06-09 13:57:44 UTC
It appears that there are too many sync result entries for mongo to sort without an index. I suspect that if we add an index to the "started" field on the "repo_sync_results" collection, this problem will go away.

A simpler work-around would be to manually remove entries from that collection. That collection periodically has old entries removed anyway, so no harm should come from removing them manually.

I'm CCing Randy Barlow, who can give more specific advice on how to proceed.

Code for reference: https://github.com/pulp/pulp/blob/2.4-release/server/pulp/server/managers/repo/sync.py#L291

Comment 5 Kim Borup 2015-06-10 07:42:48 UTC
0) /etc/pulp/server.conf

I tried this option however i still get the sort problem

Comment 8 Randy Barlow 2015-06-11 14:18:22 UTC
There is not a way to directly force the reaper to run that doesn't involve opening a python shell. However, the reaper's period can be adjusted. It defaults to a period of 0.25 days (i.e., four times per day). One way to get it to run very soon is to reduce the period. For example, 0.0006944 would be about once per minute. A user could set the value very low, restart Pulp, wait for the reaper to run, and then return it to a more sane value (the default is good). If this technique were done in combination with reducing the number of days of history to keep, I believe this will solve the immediate issue.

Comment 9 Bryan Kearney 2015-06-30 17:23:49 UTC
Can we please try the fix on comment 8, and if not describe roughly the content size which is causing the issue? Thanks!

Comment 14 Taylor Biggs 2015-12-10 21:02:22 UTC
I can confirm that comment 8 didn't work for me.  I used 0.0006944 and also bumped the archived days for each section down from "60" to "2".  Still get the below:


Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024) Exception caught from plugin during publish for repo [rhel-server-rhscl-6-rpms-6Server-x86_64]
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024) Traceback (most recent call last):
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/publish.py", line 127, in _do_publish
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     publish_report = publish_repo(transfer_repo, conduit, call_config)
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 495, in wrap_f
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     return f(*args, **kwargs)
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/distributors/export_distributor/distributor.py", line 136, in publish_repo
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     return self._publisher.publish()
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 558, in publish
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     return self.process_lifecycle()
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 503, in process_lifecycle
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     super(PluginStep, self).process_lifecycle()
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 127, in process_lifecycle
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     step.process()
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 204, in process
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     self._process_block()
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/plugins/util/publish_step.py", line 669, in _process_block
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     for package_unit in package_unit_generator:
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/plugins/conduits/mixins.py", line 704, in _transfer_object_generator
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     for u in units:
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association_query.py", line 572, in _merged_units_unique_units
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     for unit in associated_units:
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib/python2.7/site-packages/pulp/server/managers/repo/unit_association_query.py", line 499, in _units_from_chained_cursors
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     for element in cursor:
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 814, in next
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     if len(self.__data) or self._refresh():
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 763, in _refresh
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     self.__uuid_subtype))
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib64/python2.7/site-packages/pymongo/cursor.py", line 720, in __send_message
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     self.__uuid_subtype)
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)   File "/usr/lib64/python2.7/site-packages/pymongo/helpers.py", line 100, in _unpack_response
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024)     error_object["$err"])
Dec 10 16:00:15 kat pulp: pulp.server.managers.repo.publish:ERROR: (1527-57024) OperationFailure: database error: too much data for sort() with no index.  add an index or specify a smaller limit

Comment 15 Taylor Biggs 2015-12-10 21:32:06 UTC
... but adding the index as in comment 2 seems to be working so far.  I'll continue to test and update this bug if anything errors out in the future.

Thanks,
Taylor

Comment 16 Taylor Biggs 2015-12-12 00:59:23 UTC
Unfortunately I'm still getting the errors, but on different tables now:

query pulp_database.units_rpm query

Is there an index I can add to this table too?

Comment 17 Taylor Biggs 2015-12-12 00:59:47 UTC
Unfortunately I'm still getting the errors, but on different tables now:

query pulp_database.units_rpm query

Is there an index I can add to this table too?

Comment 18 Taylor Biggs 2015-12-12 01:41:34 UTC
Found fix, Added Bug 1290952 since this issue is related but not identical.

Comment 19 David O'Brien 2016-04-18 00:47:50 UTC
Reset docs contact <> daobrien

Comment 23 pulp-infra@redhat.com 2016-10-05 13:02:47 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 24 pulp-infra@redhat.com 2016-10-05 13:02:53 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 25 Bryan Kearney 2016-10-05 13:06:41 UTC
This was fixed in pulp 2.5, so you should see this fix in Satellite 6.2. If you are still seeing this issue, please feel free to re-open this bug and provide additional data.


Note You need to log in before you can comment on or make changes to this bug.