Bug 1321137

Summary:	Syncing to a capsule without pulp-katello package does not work
Product:	Red Hat Satellite	Reporter:	John Mitsch <jomitsch>
Component:	Foreman Proxy	Assignee:	John Mitsch <jomitsch>
Status:	CLOSED ERRATA	QA Contact:	Kedar Bidarkar <kbidarka>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	6.2.0	CC:	bbuckingham, cwelton, ehelms, kbidarka, mmccune, omaciel
Target Milestone:	Unspecified	Keywords:	PrioBumpQA, Triaged
Target Release:	Unused
Hardware:	Unspecified
OS:	Unspecified
URL:	http://projects.theforeman.org/issues/14363
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-07-27 09:29:00 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:
Bug Blocks:	1316944, 1318423, 1321084

Description John Mitsch 2016-03-24 18:34:11 UTC

Description of problem:
Capsule errors on the sync

How reproducible:
Easy

Steps to Reproduce:
1.Set up a capsule and make sure pulp-katello is not installed
2.Try to sync the capsule

Actual results:
capsule errors on sync

Expected results:
capsule sync works correctly

Additional info:
This is because yum clone distributor relies on pulp-katello package to be installed on the capsule. We don't need this distributor anymore so it can probably be removed from our code

Workaround:
1. on capsule, yum install pulp-katello
2. sudo -u apache pulp-manage-db
3. systemctl restart httpd

Comment 2 John Mitsch 2016-03-28 15:27:32 UTC

Created redmine issue http://projects.theforeman.org/issues/14363 from this bug

Comment 3 John Mitsch 2016-03-30 16:08:19 UTC

PR open upstream https://github.com/Katello/katello/pull/5920

Comment 4 John Mitsch 2016-04-01 12:29:57 UTC

PR merged  https://github.com/Katello/katello/pull/5920

Comment 5 John Mitsch 2016-04-01 19:50:38 UTC

Moved back to assigned as there is an issue with the commit that went in upstream

Comment 6 Bryan Kearney 2016-04-01 20:09:54 UTC

Moving to POST since upstream bug http://projects.theforeman.org/issues/14363 has been closed
-------------
John Mitsch
Applied in changeset commit:katello|f5aa72dce68bc6fe8bbcf04cb78f3e8e421ae5f5.

Comment 7 John Mitsch 2016-04-01 20:29:29 UTC

The additional PR was merged here https://github.com/Katello/katello/pull/5933

Comment 8 Og Maciel 2016-04-04 13:29:57 UTC

Adding some comments here for John. Basically, I am getting the following error when syncing RHEL 5 RPMs (other versions of RHEL work fine):

    uninitialized constant Actions::Pulp::AbstractAsyncTask::PulpError

The repositories are:

* Red Hat Enterprise Linux 5 Server Kickstart i386 5.11
* Red Hat Enterprise Linux 5 Server Kickstart x86_64 5.11

This caught my attention:

    Field is required: ['variant']


Logs (/v/l/messages) show:

Apr  4 09:23:58 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Apr  4 09:23:59 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Parsing metadata.
Apr  4 09:23:59 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading metadata files.
Apr  4 09:23:59 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com
Apr  4 09:23:59 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (3): cdn.redhat.com
Apr  4 09:24:00 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Generating metadata databases.
Apr  4 09:24:13 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Determining which units need to be downloaded.
Apr  4 09:24:42 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.purge:INFO: Removing missing units.
Apr  4 09:24:51 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:INFO: Downloading additional units.
Apr  4 09:24:51 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Apr  4 09:24:52 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Apr  4 09:24:53 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): cdn.redhat.com
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832) ValidationError (Distribution:48ae9e55-ce23-4188-9002-edcd402e1ec4) (Field is required: ['variant'])
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832) Traceback (most recent call last):
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 228, in run
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)     dist_sync.run()
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 125, in run
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)     self._run(tmp_dir)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)   File "/usr/lib/python2.7/site-packages/pulp_rpm/plugins/importers/yum/parse/treeinfo.py", line 179, in _run
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)     unit.save()
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)   File "/usr/lib/python2.7/site-packages/mongoengine/document.py", line 304, in save
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)     self.validate(clean=clean)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)   File "/usr/lib/python2.7/site-packages/mongoengine/base/document.py", line 413, in validate
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832)     raise ValidationError(message, errors=errors)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp_rpm.plugins.importers.yum.sync:ERROR: (11630-68832) ValidationError: ValidationError (Distribution:48ae9e55-ce23-4188-9002-edcd402e1ec4) (Field is required: ['variant'])
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp.server.event.http:INFO: (11630-68832) {"call_report": {"exception": null, "task_type": "pulp.server.managers.repo.sync.sync", "_href": "/pulp/api/v2/tasks/ceb760ea-82bd-4bf1-8797-7f407e3feeaa/", "task_id": "ceb760ea-82bd-4bf1-8797-7f407e3feeaa", "tags": ["pulp:repository:Default_Organization-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_5_Server_Kickstart_i386_5_11", "pulp:action:sync"], "finish_time": null, "_ns": "task_status", "start_time": "2016-04-04T13:23:56Z", "traceback": null, "spawned_tasks": [], "progress_report": {"yum_importer": {"content": {"items_total": 0, "state": "FINISHED", "error_details": [], "details": {"rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0}, "size_total": 0, "size_left": 0, "items_left": 0}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "FAILED", "error_details": [], "items_left": 0, "error": "ValidationError (Distribution:48ae9e55-ce23-4188-9002-edcd402e1ec4) (Field is required: ['variant'])"}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}}, "state": "running", "worker_name": "reserved_resource_worker-2.eng.rdu2.redhat.com", "result": null, "error": null, "_id": {"$oid": "57026aebe7056a4316df9e3e"}, "id": "57026aebe7056a4316df9e3e"}, "event_type": "repo.sync.finish", "payload": {"importer_id": "yum_importer", "exception": null, "repo_id": "Default_Organization-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_5_Server_Kickstart_i386_5_11", "traceback": null, "started": "2016-04-04T13:23:56Z", "_ns": "repo_sync_results", "completed": "2016-04-04T13:24:54Z", "importer_type_id": "yum_importer", "error_message": null, "summary": {"content": {"state": "FINISHED"}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"state": "FAILED"}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "added_count": 0, "result": "failed", "updated_count": 2549, "de
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp.server.event.http:INFO: (11630-68832) tails": {"content": {"size_total": 0, "items_left": 0, "items_total": 0, "state": "FINISHED", "size_left": 0, "details": {"rpm_total": 0, "rpm_done": 0, "drpm_total": 0, "drpm_done": 0}, "error_details": []}, "comps": {"state": "NOT_STARTED"}, "purge_duplicates": {"state": "NOT_STARTED"}, "distribution": {"items_total": 0, "state": "FAILED", "error_details": [], "items_left": 0, "error": "ValidationError (Distribution:48ae9e55-ce23-4188-9002-edcd402e1ec4) (Field is required: ['variant'])"}, "errata": {"state": "NOT_STARTED"}, "metadata": {"state": "FINISHED"}}, "id": "57026b264a45522d6ed98bd1", "removed_count": 0}}
Apr  4 09:24:54 ibm-x3250m4-01 pulp: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (1): ibm-x3250m4-01.lab.eng.rdu2.redhat.com
Apr  4 09:24:54 ibm-x3250m4-01 pulp: pulp.server.async.tasks:INFO: Task failed : [ceb760ea-82bd-4bf1-8797-7f407e3feeaa]
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832) Task pulp.server.managers.repo.sync.sync[ceb760ea-82bd-4bf1-8797-7f407e3feeaa] raised unexpected: PulpExecutionException('Importer indicated a failed response',)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832) Traceback (most recent call last):
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 240, in trace_task
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)     R = retval = fun(*args, **kwargs)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 473, in __call__
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)     return super(Task, self).__call__(*args, **kwargs)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)   File "/usr/lib/python2.7/site-packages/pulp/server/async/tasks.py", line 103, in __call__
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)     return super(PulpTask, self).__call__(*args, **kwargs)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)   File "/usr/lib/python2.7/site-packages/celery/app/trace.py", line 437, in __protected_call__
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)     return self.run(*args, **kwargs)
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)   File "/usr/lib/python2.7/site-packages/pulp/server/controllers/repository.py", line 810, in sync
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832)     raise pulp_exceptions.PulpExecutionException(_('Importer indicated a failed response'))
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:ERROR: (11392-68832) PulpExecutionException: Importer indicated a failed response
Apr  4 09:24:54 ibm-x3250m4-01 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._release_resource[b63e6fd9-934f-47dd-bbe0-011a5e9b2156] succeeded in 0.00471513101365s: None

Comment 9 Og Maciel 2016-04-04 13:41:05 UTC

Ok, filed https://bugzilla.redhat.com/show_bug.cgi?id=1323715 to track the error mentioned on comment #8 as it is a different issue.

Comment 11 Kedar Bidarkar 2016-04-14 14:34:53 UTC

VERIFIED with sat62-SNAP7.1

Had not moved it to VERIFIED State earlier, bacause of this bug https://bugzilla.redhat.com/show_bug.cgi?id=1326363

But feel we can track the above bug specifically over there and move this to VERIFIED State.

Comment 13 errata-xmlrpc 2016-07-27 09:29:00 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501