Hide Forgot
We are using the Puppet Install Distributor with Katello to publish and manage the puppet module repositories and make them available to the Puppet Master. We are hitting the issue that we can't publish any of our repositories without first creating the directory path that Pulp expects to publish to. The issue is that the katello user on the system doesn't have the ability to create and set access permission on the large numbers of directories we are going to need to maintain for all the repos managed by katello, eg: /etc/puppet/ACME_Corporation-puppet-test-puppet1 /etc/puppet/ACME_Corporation-puppet-test-puppet2 /etc/puppet/ACME_Corporation-puppet-test-puppet3 /etc/puppet/ACME_Corporation-Library-pview1-puppet-test-puppet6 /etc/puppet/ACME_Corporation-Dev-pview1-puppet-test-puppet6 /etc/puppet/ACME_Corporation-Test-pview1-puppet-test-puppet6 /etc/puppet/ACME_Corporation-Production-pview2-puppet-test-puppet6 since these directories are calculated and generated based on the repoid this isn't something we can pre-create during install time. We can possibly publish to a different directory outside of /etc/puppet but to do this we would want to pick a location that apache had write access to. Likewise since Pulp is running in apache it would be highly advantageous for Pulp to actually create the directories during the publish routine. So this is a request to see if we can get Pulp to create these directories during publish time and if so, we can adapt Katello to specify the correct location to do so.
NOTE: I do realize the docs do state that it is the callers responsibility to create and ensure access: http://pulp-puppet-dev-guide.readthedocs.org/en/latest/plugin_conf.html#install-distributor "install_path Full path to the directory where modules should be installed. It is the user’s responsibility to ensure that Pulp can write to this directory. " but after working with the distributor a bit we are still requesting that Pulp at least attempt to create the directory in the path specified. Katello/Foreman can ensure that the parent directory exists and that apache has write access, eg: -> /etc/puppet : can be written to by apache -> /etc/puppet/ACME_Corporation-puppet-test-puppet1 : created by Pulp
https://github.com/pulp/pulp_puppet/pull/74
build: 2.3.0-0.14.alpha
Looks like this is fails-qa u'summary': u"failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", u'traceback': None}]) root@pulp-v2-server ~]# ipython Python 2.6.6 (r266:84292, Oct 12 2012, 14:23:48) Type "copyright", "credits" or "license" for more information. IPython 0.10 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [1]: from pulp.common import pic In [2]: pic.connect() In [3]: pic.POST('/v2/repositories/puppet-copy/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/puppet-copy/'}}) Request Body { "distributor_id": "puppet_install_distributor", "distributor_type_id": "puppet_install_distributor", "distributor_config": { "install_path": "/etc/puppet/puppet-copy/" }, "auto_publish": false } Response Body { "repo_id": "puppet-copy", "_ns": "repo_distributors", "last_publish": null, "auto_publish": false, "scheduled_publishes": [], "distributor_type_id": "puppet_install_distributor", "scratchpad": null, "_id": { "$oid": "5241c7ffd669f040ff0003c6" }, "config": { "install_path": "/etc/puppet/puppet-copy/" }, "id": "puppet_install_distributor" } Out[3]: (201, {u'_id': {u'$oid': u'5241c7ffd669f040ff0003c6'}, u'_ns': u'repo_distributors', u'auto_publish': False, u'config': {u'install_path': u'/etc/puppet/puppet-copy/'}, u'distributor_type_id': u'puppet_install_distributor', u'id': u'puppet_install_distributor', u'last_publish': None, u'repo_id': u'puppet-copy', u'scheduled_publishes': [], u'scratchpad': None}) In [4]: pic.POST('/v2/repositories/puppet-copy/actions/publish/', {'id':'puppet_install_distributor'}) Request Body { "id": "puppet_install_distributor" } Response Body { "task_group_id": null, "call_request_id": "6ebe2022-591a-4eb5-aee4-e673f06206c6", "exception": null, "_href": "/pulp/api/v2/tasks/6ebe2022-591a-4eb5-aee4-e673f06206c6/", "task_id": "6ebe2022-591a-4eb5-aee4-e673f06206c6", "call_request_tags": [ "pulp:repository:puppet-copy", "pulp:action:publish" ], "reasons": [], "start_time": null, "traceback": null, "schedule_id": null, "finish_time": null, "state": "waiting", "result": null, "dependency_failures": {}, "call_request_group_id": null, "progress": {}, "principal_login": "admin", "response": "accepted", "tags": [ "pulp:repository:puppet-copy", "pulp:action:publish" ] } Out[4]: (202, {u'_href': u'/pulp/api/v2/tasks/6ebe2022-591a-4eb5-aee4-e673f06206c6/', u'call_request_group_id': None, u'call_request_id': u'6ebe2022-591a-4eb5-aee4-e673f06206c6', u'call_request_tags': [u'pulp:repository:puppet-copy', u'pulp:action:publish'], u'dependency_failures': {}, u'exception': None, u'finish_time': None, u'principal_login': u'admin', u'progress': {}, u'reasons': [], u'response': u'accepted', u'result': None, u'schedule_id': None, u'start_time': None, u'state': u'waiting', u'tags': [u'pulp:repository:puppet-copy', u'pulp:action:publish'], u'task_group_id': None, u'task_id': u'6ebe2022-591a-4eb5-aee4-e673f06206c6', u'traceback': None}) In [5]: pic.GET('/v2/repositories/puppet-copy/history/publish/puppet_install_distributor/') Response Body [ { "result": "failed", "exception": null, "repo_id": "puppet-copy", "started": "2013-09-24T13:12:59-04:00", "_ns": "repo_publish_results", "completed": "2013-09-24T13:12:59-04:00", "traceback": null, "distributor_type_id": "puppet_install_distributor", "_id": { "$oid": "5241c81bd669f040ff0003d0" }, "summary": "failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", "error_message": null, "distributor_id": "puppet_install_distributor", "id": "5241c81bd669f040ff0003d0", "details": { "errors": [], "success_unit_keys": [] } } ] Out[5]: (200, [{u'_id': {u'$oid': u'5241c81bd669f040ff0003d0'}, u'_ns': u'repo_publish_results', u'completed': u'2013-09-24T13:12:59-04:00', u'details': {u'errors': [], u'success_unit_keys': []}, u'distributor_id': u'puppet_install_distributor', u'distributor_type_id': u'puppet_install_distributor', u'error_message': None, u'exception': None, u'id': u'5241c81bd669f040ff0003d0', u'repo_id': u'puppet-copy', u'result': u'failed', u'started': u'2013-09-24T13:12:59-04:00', u'summary': u"failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", u'traceback': None}]) In [6]:
This looks like normal behavior to me, unless you have reason to believe that apache does in fact have permission to create that directory. By default, apache cannot write in /etc/puppet/. The user is responsible for making sure that apache has permission to write in the destination directory, and that's stated in the documentation for the distributor. See the very last sentence here: http://pulp-puppet-dev-guide.readthedocs.org/en/latest/plugin_conf.html#install-distributor
Agreed. Moving back to ON_QA for re-test.
verified [root@pulp-v2-server ~]# rpm -qa pulp-server pulp-server-2.3.0-0.17.beta.el6.noarch [root@pulp-v2-server ~]# In [1]: from pulp.common import pic In [2]: pic.connect() In [3]: pic.POST('/v2/repositories/puppet-copy/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/puppet-copy/'}}) Request Body { "distributor_id": "puppet_install_distributor", "distributor_type_id": "puppet_install_distributor", "distributor_config": { "install_path": "/etc/puppet/puppet-copy/" }, "auto_publish": false } Response Body { "repo_id": "puppet-copy", "_ns": "repo_distributors", "last_publish": null, "auto_publish": false, "scheduled_publishes": [], "distributor_type_id": "puppet_install_distributor", "scratchpad": null, "_id": { "$oid": "524ed515d669f0087a001df1" }, "config": { "install_path": "/etc/puppet/puppet-copy/" }, "id": "puppet_install_distributor" } Out[3]: (201, {u'_id': {u'$oid': u'524ed515d669f0087a001df1'}, u'_ns': u'repo_distributors', u'auto_publish': False, u'config': {u'install_path': u'/etc/puppet/puppet-copy/'}, u'distributor_type_id': u'puppet_install_distributor', u'id': u'puppet_install_distributor', u'last_publish': None, u'repo_id': u'puppet-copy', u'scheduled_publishes': [], u'scratchpad': None}) In [4]: pic.POST('/v2/repositories/puppet-copy/actions/publish/', {'id':'puppet_install_distributor'}) Request Body { "id": "puppet_install_distributor" } Response Body { "task_group_id": null, "call_request_id": "5cc6288e-153f-4053-b9ae-f1e259ae141a", "exception": null, "_href": "/pulp/api/v2/tasks/5cc6288e-153f-4053-b9ae-f1e259ae141a/", "task_id": "5cc6288e-153f-4053-b9ae-f1e259ae141a", "call_request_tags": [ "pulp:repository:puppet-copy", "pulp:action:publish" ], "reasons": [], "start_time": null, "traceback": null, "schedule_id": null, "finish_time": null, "state": "waiting", "result": null, "dependency_failures": {}, "call_request_group_id": null, "progress": {}, "principal_login": "admin", "response": "accepted", "tags": [ "pulp:repository:puppet-copy", "pulp:action:publish" ] } Out[4]: (202, {u'_href': u'/pulp/api/v2/tasks/5cc6288e-153f-4053-b9ae-f1e259ae141a/', u'call_request_group_id': None, u'call_request_id': u'5cc6288e-153f-4053-b9ae-f1e259ae141a', u'call_request_tags': [u'pulp:repository:puppet-copy', u'pulp:action:publish'], u'dependency_failures': {}, u'exception': None, u'finish_time': None, u'principal_login': u'admin', u'progress': {}, u'reasons': [], u'response': u'accepted', u'result': None, u'schedule_id': None, u'start_time': None, u'state': u'waiting', u'tags': [u'pulp:repository:puppet-copy', u'pulp:action:publish'], u'task_group_id': None, u'task_id': u'5cc6288e-153f-4053-b9ae-f1e259ae141a', u'traceback': None}) In [5]: pic.GET('/v2/repositories/puppet-copy/history/publish/puppet_install_distributor/') Response Body [ { "result": "failed", "exception": null, "repo_id": "puppet-copy", "started": "2013-10-04T10:48:19-04:00", "_ns": "repo_publish_results", "completed": "2013-10-04T10:48:20-04:00", "traceback": null, "distributor_type_id": "puppet_install_distributor", "_id": { "$oid": "524ed534d669f0087a001dfb" }, "summary": "failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", "error_message": null, "distributor_id": "puppet_install_distributor", "id": "524ed534d669f0087a001dfb", "details": { "errors": [], "success_unit_keys": [] } } ] Out[5]: (200, [{u'_id': {u'$oid': u'524ed534d669f0087a001dfb'}, u'_ns': u'repo_publish_results', u'completed': u'2013-10-04T10:48:20-04:00', u'details': {u'errors': [], u'success_unit_keys': []}, u'distributor_id': u'puppet_install_distributor', u'distributor_type_id': u'puppet_install_distributor', u'error_message': None, u'exception': None, u'id': u'524ed534d669f0087a001dfb', u'repo_id': u'puppet-copy', u'result': u'failed', u'started': u'2013-10-04T10:48:19-04:00', u'summary': u"failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", u'traceback': None}]) In [6]: In [7]: In [8]: pic.POST('/v2/repositories/puppet-copy/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/tmp/preethi-puppet'}}) Request Body { "distributor_id": "puppet_install_distributor", "distributor_type_id": "puppet_install_distributor", "distributor_config": { "install_path": "/tmp/preethi-puppet" }, "auto_publish": false } Response Body { "repo_id": "puppet-copy", "_ns": "repo_distributors", "last_publish": null, "auto_publish": false, "scheduled_publishes": [], "distributor_type_id": "puppet_install_distributor", "scratchpad": null, "_id": { "$oid": "524ed597d669f0087a001e06" }, "config": { "install_path": "/tmp/preethi-puppet" }, "id": "puppet_install_distributor" } Out[8]: (201, {u'_id': {u'$oid': u'524ed597d669f0087a001e06'}, u'_ns': u'repo_distributors', u'auto_publish': False, u'config': {u'install_path': u'/tmp/preethi-puppet'}, u'distributor_type_id': u'puppet_install_distributor', u'id': u'puppet_install_distributor', u'last_publish': None, u'repo_id': u'puppet-copy', u'scheduled_publishes': [], u'scratchpad': None}) In [9]: pic.POST('/v2/repositories/puppet-copy/actions/publish/', {'id':'puppet_install_distributor'}) Request Body { "id": "puppet_install_distributor" } Response Body { "task_group_id": null, "call_request_id": "1884fdf1-6ce3-4d2b-8c2d-2ab93b924b25", "exception": null, "_href": "/pulp/api/v2/tasks/1884fdf1-6ce3-4d2b-8c2d-2ab93b924b25/", "task_id": "1884fdf1-6ce3-4d2b-8c2d-2ab93b924b25", "call_request_tags": [ "pulp:repository:puppet-copy", "pulp:action:publish" ], "reasons": [], "start_time": null, "traceback": null, "schedule_id": null, "finish_time": null, "state": "waiting", "result": null, "dependency_failures": {}, "call_request_group_id": null, "progress": {}, "principal_login": "admin", "response": "accepted", "tags": [ "pulp:repository:puppet-copy", "pulp:action:publish" ] } Out[9]: (202, {u'_href': u'/pulp/api/v2/tasks/1884fdf1-6ce3-4d2b-8c2d-2ab93b924b25/', u'call_request_group_id': None, u'call_request_id': u'1884fdf1-6ce3-4d2b-8c2d-2ab93b924b25', u'call_request_tags': [u'pulp:repository:puppet-copy', u'pulp:action:publish'], u'dependency_failures': {}, u'exception': None, u'finish_time': None, u'principal_login': u'admin', u'progress': {}, u'reasons': [], u'response': u'accepted', u'result': None, u'schedule_id': None, u'start_time': None, u'state': u'waiting', u'tags': [u'pulp:repository:puppet-copy', u'pulp:action:publish'], u'task_group_id': None, u'task_id': u'1884fdf1-6ce3-4d2b-8c2d-2ab93b924b25', u'traceback': None}) In [10]: pic.GET('/v2/repositories/puppet-copy/history/publish/puppet_install_distributor/') Response Body [ { "result": "success", "exception": null, "repo_id": "puppet-copy", "started": "2013-10-04T10:50:07-04:00", "_ns": "repo_publish_results", "completed": "2013-10-04T10:50:07-04:00", "traceback": null, "distributor_type_id": "puppet_install_distributor", "_id": { "$oid": "524ed59fd669f0087a001e10" }, "summary": "success", "error_message": null, "distributor_id": "puppet_install_distributor", "id": "524ed59fd669f0087a001e10", "details": { "errors": [], "success_unit_keys": [ { "version": "4.1.0", "name": "stdlib", "author": "puppetlabs" } ] } }, { "result": "failed", "exception": null, "repo_id": "puppet-copy", "started": "2013-10-04T10:48:19-04:00", "_ns": "repo_publish_results", "completed": "2013-10-04T10:48:20-04:00", "traceback": null, "distributor_type_id": "puppet_install_distributor", "_id": { "$oid": "524ed534d669f0087a001dfb" }, "summary": "failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", "error_message": null, "distributor_id": "puppet_install_distributor", "id": "524ed534d669f0087a001dfb", "details": { "errors": [], "success_unit_keys": [] } } ] Out[10]: (200, [{u'_id': {u'$oid': u'524ed59fd669f0087a001e10'}, u'_ns': u'repo_publish_results', u'completed': u'2013-10-04T10:50:07-04:00', u'details': {u'errors': [], u'success_unit_keys': [{u'author': u'puppetlabs', u'name': u'stdlib', u'version': u'4.1.0'}]}, u'distributor_id': u'puppet_install_distributor', u'distributor_type_id': u'puppet_install_distributor', u'error_message': None, u'exception': None, u'id': u'524ed59fd669f0087a001e10', u'repo_id': u'puppet-copy', u'result': u'success', u'started': u'2013-10-04T10:50:07-04:00', u'summary': u'success', u'traceback': None}, {u'_id': {u'$oid': u'524ed534d669f0087a001dfb'}, u'_ns': u'repo_publish_results', u'completed': u'2013-10-04T10:48:20-04:00', u'details': {u'errors': [], u'success_unit_keys': []}, u'distributor_id': u'puppet_install_distributor', u'distributor_type_id': u'puppet_install_distributor', u'error_message': None, u'exception': None, u'id': u'524ed534d669f0087a001dfb', u'repo_id': u'puppet-copy', u'result': u'failed', u'started': u'2013-10-04T10:48:19-04:00', u'summary': u"failed to create destination directory: [Errno 13] Permission denied: '/etc/puppet/puppet-copy/'", u'traceback': None}]) [root@pulp-v2-server ~]# ls -l /tmp/preethi-puppet/ total 4 drwxr-xr-x. 6 apache apache 4096 May 13 10:53 stdlib [root@pulp-v2-server ~]#
Pulp 2.3 released.