Bug 1359780

Summary: Applying migration pulp.server.db.migrations.0016_remove_repo_content_unit_owner_type_and_id failed.
Product: Red Hat Satellite Reporter: Andy MacMurray <amacmurr>
Component: UpgradesAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED NOTABUG QA Contact: Katello QA List <katello-qa-list>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.2.0CC: amacmurr, inecas, ohadlevy
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-28 11:18:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Console output
none
Debug log none

Description Andy MacMurray 2016-07-25 12:41:57 UTC
Created attachment 1183784 [details]
Console output

Description of problem: While performing an upgrade of satellite6-ops.rhev-ci-vms.eng.rdu2.redhat.com using snap http://ohsnap-satellite6.itos.redhat.com/releases/6.2.0/snaps/21.1 we encountered the following error:

Applying migration pulp.server.db.migrations.0016_remove_repo_content_unit_owner_type_and_id failed.

Halting migrations due to a migration failure.
reading doc for external sort failed:errno:0 Success
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 193, in main
    return _auto_manage_db(options)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 256, in _auto_manage_db
    migrate_database(options)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/manage.py", line 124, in migrate_database
    update_current_version=not options.test)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/migrate/models.py", line 186, in apply_migration
    migration.migrate()
  File "/usr/lib/python2.6/site-packages/pulp/server/db/migrations/0016_remove_repo_content_unit_owner_type_and_id.py", line 31, in migrate
    remove_duplicates(collection)
  File "/usr/lib/python2.6/site-packages/pulp/server/db/migrations/0016_remove_repo_content_unit_owner_type_and_id.py", line 54, in remove_duplicates
    collection.ensure_index([(i, -1) for i in index])
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 1407, in ensure_index
    self.__create_index(keys, kwargs)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 1297, in __create_index
    sock_info, index, True, False, False, wcn)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 530, in _insert
    check_keys, manipulate, write_concern, op_id, bypass_doc_val)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 519, in _insert_one
    check_keys, acknowledged, concern, False, self.codec_options)
  File "/usr/lib64/python2.6/site-packages/pymongo/collection.py", line 458, in _legacy_write
    rqst_id, msg, max_size, acknowledged)
  File "/usr/lib64/python2.6/site-packages/pymongo/pool.py", line 264, in legacy_write
    return helpers._check_gle_response(response)
  File "/usr/lib64/python2.6/site-packages/pymongo/helpers.py", line 236, in _check_gle_response
    raise OperationFailure(details["err"], code, result)
OperationFailure: reading doc for external sort failed:errno:0 Success




Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:

Comment 1 Andy MacMurray 2016-07-25 12:42:33 UTC
Created attachment 1183785 [details]
Debug log

Comment 2 Ohad Levy 2016-07-25 14:04:24 UTC
please note the db is nearly full, can you try again with more disk space?

Comment 3 Ohad Levy 2016-07-25 14:04:42 UTC
sorry, i mean disk space (not db).

Comment 4 Andy MacMurray 2016-07-26 11:47:04 UTC
Adding additional storage appears to have resolved this issue.

Comment 5 Bryan Kearney 2016-07-26 18:54:07 UTC
Moving 6.2 bugs out to sat-backlog.

Comment 6 Ivan Necas 2016-07-28 11:18:53 UTC
As the problem seems to be resolved, I'm closing the BZ, feel free to re-open, if you think there is something on our side to improve. Low disk space is in general pretty complicated thing to handle nicely.