Hide Forgot
Description of problem: Capsule Sync breaks on rhel6 OS, with the message "500 Internal Server Error". 2016-04-25 12:34:59 [app] [E] RestClient::InternalServerError: 500 Internal Server Error | Body: {"http_request_method": "GET", "exception": ["ServerSelectionTimeoutError: localhost:27017: [Errno 111] Connection refused\n"], "error_message": "localhost:27017: [Errno 111] Connection refused", "_href": "/pulp/api/v2/repositories/Default_Organization-Dev-rhel6_capsule_cv-rhel6_capsule-rhel6_capsule_repo/?details=true", "http_status": 500, "traceback": [" File \"/usr/lib/python2.6/site-packages/django/core/handlers/base.py\", line 112, in get_response\n response = wrapped_callback(request, *callback_args, **callback_kwargs)\n", " File \"/usr/lib/python2.6/site-packages/django/views/generic/base.py\", line 69, in view\n return self.dispatch(request, *args, **kwargs)\n", " File \"/usr/lib/python2.6/site-packages/django/views/generic/base.py\", line 87, in dispatch\n return handler(request, *args, **kwargs)\n", " File \"/usr/lib/python2.6/site-packages/pulp/server/webservices/views/decorators.py\", line 241, in _auth_decorator\n return _verify_auth(self, operation, super_user_only, method, *args, **kwargs)\n", " File \"/usr/lib/python2.6/site-packages/pulp/server/webservices/views/decorators.py\", line 154, in _verify_auth\n login = authenticate_user()\n", " File \"/usr/lib/python2.6/site-packages/pulp/server/webservices/views/decorators.py\", line 30, in check_preauthenticated\n userid = factory.authentication_manager().check_username_password(username)\n", " File \"/usr/lib/python2.6/site-packages/pulp/server/managers/auth/authentication.py\", line 95, in check_username_password\n user = self._check_username_password_local(username, password)\n", " File \"/usr/lib/python2.6/site-packages/pulp/server/managers/auth/authentication.py\", line 35, in _check_username_password_local\n user = model.User.objects(login=username).first()\n", " File \"/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py\", line 264, in first\n result = queryset[0]\n", " File \"/usr/lib/python2.6/site-packages/mongoengine/queryset/base.py\", line 161, in __getitem__\n return queryset._document._from_son(queryset._cursor[key],\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/cursor.py\", line 565, in __getitem__\n for doc in clone:\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/cursor.py\", line 1097, in next\n if len(self.__data) or self._refresh():\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/cursor.py\", line 1019, in _refresh\n self.__read_concern))\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/cursor.py\", line 850, in __send_message\n **kwargs)\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/mongo_client.py\", line 777, in _send_message_with_response\n server = topology.select_server(selector)\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/topology.py\", line 141, in select_server\n address))\n", " File \"/usr/lib64/python2.6/site-packages/pymongo/topology.py\", line 117, in select_servers\n self._error_message(selector))\n"]} | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!' | /opt/theforeman/tfm/root/usr/share/gems/gems/runcible-1.7.0/lib/runcible/base.rb:79:in `block in get_response' | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:228:in `call' | /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:228:in `process_result' | /opt/theforeman/tfm/root/usr/share/gems/gems/rbovirt-0.0.36/lib/restclient_ext/request.rb:50:in `block in transmit' Version-Release number of selected component (if applicable): sat62-snap9 How reproducible: Steps to Reproduce: 1. Add lifecycle-env to the capsule and sync 2. 3. Actual results: Capsule Sync fails Expected results: Capsule Sunc should be successful Additional info:
Created attachment 1150558 [details] capsule sync breaks
Please Note: A workaround was added for https://bugzilla.redhat.com/show_bug.cgi?id=1329640 on the satellite6 for the sync to work.
After updating the packages on the capsule, migrating pulp, and restarting pulp, I was able to sync the repositories onto said capsule from a rhel6 satellite. I ran into one bug with the rhel6 kick start repo, but that has been filed here https://bugzilla.redhat.com/show_bug.cgi?id=1329640 Can we move this to VERIFIED?
Was the workaround for rhel6-ks applied and capsule synced successfully. If that is so then we can close this bug. Otherwise currently the issue seen on my setup after the rhel6-ks cdn repo sync thing is this. Apr 28 06:04:32 qe-capsule-feature-rhel6 pulp: celery.worker.strategy:INFO: Received task: pulp.server.async.tasks._release_resource[683f96ff-01d1-4780-ae4a-45a1db8bce15] Apr 28 06:04:32 qe-capsule-feature-rhel6 pulp: celery.worker.job:INFO: Task pulp.server.async.tasks._queue_reserved_task[1c12caa5-bce7-4886-b566-49ba941a07b7] succeeded in 2.134045634s: None Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) Exception caught from plugin during publish for repo [Default_Organization-Dev-rhel6_cv-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_Kickstart_x86_64_6_7] Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) Traceback (most recent call last): Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp/server/controllers/repository.py", line 1023, in _do_publish Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) publish_report = publish_repo(transfer_repo, conduit, call_config) Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 658, in wrap_f Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) return f(*args, **kwargs) Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/distributor.py", line 174, in publish_repo Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) return self._publisher.process_lifecycle() Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 562, in process_lifecycle Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) super(PluginStep, self).process_lifecycle() Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 159, in process_lifecycle Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) step.process() Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 232, in process Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) self._process_block(item=item) Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp/plugins/util/publish_step.py", line 291, in _process_block Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) self.process_main(item=item) Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 748, in process_main Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) self._publish_distribution_files(unit) Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 816, in _publish_distribution_files Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) self._write_pulp_distribution_file(distro_files, xml_file_path) Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/distributors/yum/publish.py", line 835, in _write_pulp_distribution_file Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) old_files = [old_element.text for old_element in old_xml_root.iter('file')] Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.controllers.repository:ERROR: (4272-37728) AttributeError: iter Apr 28 06:04:33 qe-capsule-feature-rhel6 pulp: pulp.server.async.tasks:INFO: Task failed : [33bc9dbd-7a2b-4a2e-900f-5e25b898c40a]
Created attachment 1151802 [details] capsule foreman-debug
Created attachment 1151806 [details] satellite6 foreman-debug
(In reply to Kedar Bidarkar from comment #5) > pulp.server.controllers.repository:ERROR: (4272-37728) Exception caught from > plugin during publish for repo > [Default_Organization-Dev-rhel6_cv-Red_Hat_Enterprise_Linux_Server- > Red_Hat_Enterprise_Linux_6_Server_Kickstart_x86_64_6_7] Kedar, It looks like this error has to do with the rhel6 kickstart repo again, I think that is again related to 1329640 bug. -John
Looks like Capsule sync fails for rhel6-ks repo. Also a workaround was applied for rhel6-ks sync failure in satellite6 as per mentioned in comment 2. Discussed the same with john Mitsch and it looks like this issue is seen due to the bug https://bugzilla.redhat.com/show_bug.cgi?id=1329640. So, Will close this bug and will wait for the bug https://bugzilla.redhat.com/show_bug.cgi?id=1329640 to get fixed for testing the capsule syncs with rhel6-ks. In short: Capsule sync fails for rhel6 if rhel6-ks repo is to be synced to capsule and we need this to be tested again after https://bugzilla.redhat.com/show_bug.cgi?id=1329640 is fixed. If this issue is seen again after https://bugzilla.redhat.com/show_bug.cgi?id=1329640 is fixed, this bug will be re-opened.