Bug 1009429 - seems like puppet install distributor fails when selinux is enforcing
seems like puppet install distributor fails when selinux is enforcing
Product: Pulp
Classification: Community
Component: puppet-support (Show other bugs)
2.2 Beta
Unspecified Unspecified
urgent Severity high
: ---
: 2.5.0
Assigned To: Randy Barlow
Preethi Thomas
: Triaged
Depends On:
  Show dependency treegraph
Reported: 2013-09-18 08:29 EDT by Preethi Thomas
Modified: 2014-11-24 16:33 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2014-11-24 16:33:58 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Preethi Thomas 2013-09-18 08:29:09 EDT
Description of problem:
With Selinux enforcing running the following seems to fail with permission error

pic.POST('/v2/repositories/forge/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge/'}})

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1.Create and sync a puppet repo (eg forge)
2.create the directory that you want puppet to be installed on and give apache the correct permission (/etc/puupet/forge/)
3.with selinux enforcing run 
pic.POST('/v2/repositories/forge/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge/'}})

Actual results:

Expected results:

Additional info:
Comment 1 Sayli Karmarkar 2013-09-18 13:29:09 EDT
This is a documentation bug. We need to add selinux command to the documentation to set appropriate context for repo install path directory.
Comment 2 Randy Barlow 2014-10-02 17:07:09 EDT
I believe we can fix this by moving the pulp_manage_puppet tunable from httpd_t to celery_t.
Comment 3 Randy Barlow 2014-10-02 17:16:32 EDT
In 2.5.0 our workers now run confined. I believe this means that we will not be able to publish to /etc/puppet/. In 2.4.{0,1} we were able to publish there, only because our workers were running unconfined. This means that this is a regression in our 2.5.0 beta.
Comment 4 Randy Barlow 2014-10-03 18:03:52 EDT
Comment 5 Randy Barlow 2014-10-06 10:41:02 EDT
This will add the install distributor onto the "forge" repo:

>>> pic.POST('/v2/repositories/forge/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge/'}})

This will tell that distributor to publish:

>>> pic.POST('/pulp/api/v2/repositories/forge/actions/publish/', {'id': "puppet_install_distributor"})
Comment 6 Randy Barlow 2014-10-06 11:29:42 EDT
Comment 7 Brian Bouterse 2014-10-07 10:14:55 EDT
Don't include a trailing slash when specifying the install_path, or you'll run into this bug [0].

[0]:  https://bugzilla.redhat.com/show_bug.cgi?id=1149894
Comment 8 Chris Duryee 2014-10-12 16:26:17 EDT
build: 2.5.0 0.8.beta
Comment 9 Preethi Thomas 2014-10-16 16:43:29 EDT

So I did 

 1. setfacl -m g:apache:rwx /etc/puppet/forge
 2. setfacl -m u:apache:rwx /etc/puppet/forge
 3.  sudo semanage boolean --on pulp_manage_puppet

Then tried

pic.POST('/v2/repositories/forge1/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge'}})

And it failed with

> pic.POST('/v2/repositories/forge1/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge'}})
Request Body
  "distributor_id": "puppet_install_distributor",
  "distributor_type_id": "puppet_install_distributor",
  "distributor_config": {
    "install_path": "/etc/puppet/forge"
  "auto_publish": false
Response Body
  "exception": null,
  "args": [
    "the current user does not have permission to read and write files in the destination directory"
  "_href": "/pulp/api/v2/repositories/forge1/distributors/",
  "traceback": null,
  "error_message": "the current user does not have permission to read and write files in the destination directory",
  "http_request_method": "POST",
  "http_status": 400,
  "error": {
    "code": "PLP0000",
    "data": {},
    "description": "the current user does not have permission to read and write files in the destination directory",
    "sub_errors": []
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.6/site-packages/pulp/common/pic.py", line 133, in POST
    return _request('POST', path, body)
  File "/usr/lib/python2.6/site-packages/pulp/common/pic.py", line 114, in _request
    (response.status, response_body))
pulp.common.pic.RequestError: Server response: 400
{u'exception': None, u'args': [u'the current user does not have permission to read and write files in the destination directory'], u'_href': u'/pulp/api/v2/repositories/forge1/distributors/', u'traceback': None, u'error_message': u'the current user does not have permission to read and write files in the destination directory', u'http_request_method': u'POST', u'http_status': 400, u'error': {u'code': u'PLP0000', u'data': {}, u'description': u'the current user does not have permission to read and write files in the destination directory', u'sub_errors': []}}

Then did

sudo chown -R apache:apache /etc/puppet/forge
sudo chown apache:apache /etc/puppet

And ran the pic from above. Still got the same error.

Now I set selinux disabled.

Ran the pic above. And it worked.

>>> pic.POST('/v2/repositories/forge1/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge'}})
Request Body
  "distributor_id": "puppet_install_distributor",
  "distributor_type_id": "puppet_install_distributor",
  "distributor_config": {
    "install_path": "/etc/puppet/forge"
  "auto_publish": false
Response Body
  "repo_id": "forge1",
  "_href": "/pulp/api/v2/repositories/forge1/distributors/puppet_install_distributor/",
  "_ns": "repo_distributors",
  "last_publish": null,
  "auto_publish": false,
  "scheduled_publishes": [],
  "distributor_type_id": "puppet_install_distributor",
  "scratchpad": null,
  "_id": {
    "$oid": "544024b812a08e620e0e4cad"
  "config": {
    "install_path": "/etc/puppet/forge"
  "id": "puppet_install_distributor"
(201, {u'repo_id': u'forge1', u'_href': u'/pulp/api/v2/repositories/forge1/distributors/puppet_install_distributor/', u'_ns': u'repo_distributors', u'last_publish': None, u'auto_publish': False, u'scheduled_publishes': [], u'distributor_type_id': u'puppet_install_distributor', u'scratchpad': None, u'_id': {u'$oid': u'544024b812a08e620e0e4cad'}, u'config': {u'install_path': u'/etc/puppet/forge'}, u'id': u'puppet_install_distributor'})
Now I set selinux to enforcing and run the publish and it succeeds.

>>> pic.POST('/pulp/api/v2/repositories/forge1/actions/publish/', {'id': "puppet_install_distributor"})
Request Body
  "id": "puppet_install_distributor"
Response Body
  "spawned_tasks": [
      "_href": "/pulp/api/v2/tasks/75d298fd-597c-4f4b-a83d-b3ba9a731849/",
      "task_id": "75d298fd-597c-4f4b-a83d-b3ba9a731849"
  "result": null,
  "error": null
(202, {u'spawned_tasks': [{u'_href': u'/pulp/api/v2/tasks/75d298fd-597c-4f4b-a83d-b3ba9a731849/', u'task_id': u'75d298fd-597c-4f4b-a83d-b3ba9a731849'}], u'result': None, u'error': None})

[root@qe-blade-12 ~]# ls -ladZ of /etc/puppet/forge/*
ls: cannot access of: No such file or directory
drwxr-xr-x. apache apache unconfined_u:object_r:puppet_etc_t:s0 /etc/puppet/forge/x11
[root@qe-blade-12 ~]# 
[root@qe-blade-12 ~]# 
[root@qe-blade-12 ~]# ls -ladZ of /etc/puppet/*
ls: cannot access of: No such file or directory
-rw-r--r--. root   root   system_u:object_r:puppet_etc_t:s0 /etc/puppet/auth.conf
drwxrwxr-x+ apache apache unconfined_u:object_r:puppet_etc_t:s0 /etc/puppet/forge
drwxr-xr-x. root   root   system_u:object_r:puppet_etc_t:s0 /etc/puppet/modules
-rw-r--r--. root   root   system_u:object_r:puppet_etc_t:s0 /etc/puppet/puppet.conf
[root@qe-blade-12 ~]# 

This is from the audit.log when I try the first pic

type=AVC msg=audit(1413491976.104:2577): avc:  denied  { write } for  pid=25186 comm="httpd" name="forge" dev=dm-0 ino=2230907 scontext=unconfined_u:system_r:httpd_t:s0 tcontext=unconfined_u:object_r:puppet_etc_t:s0 tclass=dir
type=SYSCALL msg=audit(1413491976.104:2577): arch=c000003e syscall=21 success=no exit=-13 a0=7f38dc017b20 a1=7 a2=7f391555ba08 a3=7f38f51ef238 items=0 ppid=25098 pid=25186 auid=0 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=310 comm="httpd" exe="/usr/sbin/httpd" subj=unconfined_u:system_r:httpd_t:s0 key=(null)
Comment 10 Preethi Thomas 2014-10-16 16:44:04 EDT
[root@qe-blade-12 ~]# sudo audit2allow -al

#============= httpd_t ==============
#!!!! The source type 'httpd_t' can write to a 'dir' of the following types:
# httpd_squirrelmail_t, var_lib_t, var_run_t, var_log_t, httpd_mediawiki_tmp_t, dirsrv_var_log_t, zarafa_var_lib_t, dirsrv_var_run_t, httpd_var_lib_t, httpd_var_run_t, squirrelmail_spool_t, dirsrvadmin_config_t, mnt_t, var_lock_t, tmp_t, var_t, tmpfs_t, httpd_log_t, dirsrv_config_t, httpd_tmp_t, pulp_cert_t, dirsrvadmin_tmp_t, httpd_cache_t, httpd_tmpfs_t, httpd_bugzilla_ra_content_t, httpd_bugzilla_rw_content_t, httpd_cvs_ra_content_t, httpd_cvs_rw_content_t, httpd_git_ra_content_t, httpd_git_rw_content_t, httpd_sys_ra_content_t, httpd_sys_rw_content_t, httpd_sys_rw_content_t, httpd_nagios_ra_content_t, httpd_nagios_rw_content_t, passenger_tmp_t, httpd_nutups_cgi_ra_content_t, httpd_nutups_cgi_rw_content_t, httpd_apcupsd_cgi_ra_content_t, httpd_apcupsd_cgi_rw_content_t, httpd_sys_content_t, httpd_dspam_ra_content_t, httpd_dspam_rw_content_t, httpd_mediawiki_ra_content_t, httpd_mediawiki_rw_content_t, httpd_squid_ra_content_t, httpd_squid_rw_content_t, httpd_prewikka_ra_content_t, httpd_prewikka_rw_content_t, httpd_smokeping_cgi_ra_content_t, httpd_smokeping_cgi_rw_content_t, passenger_var_run_t, httpd_openshift_ra_content_t, httpd_openshift_rw_content_t, httpd_dirsrvadmin_ra_content_t, httpd_dirsrvadmin_rw_content_t, httpd_w3c_validator_ra_content_t, httpd_w3c_validator_rw_content_t, cluster_var_lib_t, cluster_var_run_t, httpd_user_ra_content_t, httpd_user_rw_content_t, httpd_awstats_ra_content_t, httpd_awstats_rw_content_t, root_t, httpdcontent, httpd_cobbler_ra_content_t, httpd_cobbler_rw_content_t, httpd_munin_ra_content_t, httpd_munin_rw_content_t, cluster_conf_t

allow httpd_t puppet_etc_t:dir write;
[root@qe-blade-12 ~]#
Comment 11 Preethi Thomas 2014-10-16 16:44:56 EDT
[root@qe-blade-12 ~]# rpm -qa pulp-server
[root@qe-blade-12 ~]#
Comment 12 Randy Barlow 2014-10-21 12:05:08 EDT
Hmmm, I am unable to reproduce these errors and am able to successfully publish with selinux enabled.

Can you ls -laZ /etc/puppet for me?

Also, it's curious that you are seeing that httpd_t needs access to puppet_etc_t. It's should be celery_t, not httpd_t.
Comment 13 Preethi Thomas 2014-10-21 13:28:53 EDT
The error happens when you create the distributor with selinux enabled. 

Publish works with selinux enabled.

Steps to reproduce.

1. sudo setenforce 1 

2 Create a single puppet repo with a single unit in it. I do this by running the following:
  - pulp-admin puppet repo create --repo-id=forge --feed=http://forge.puppetlabs.com
  - sync the first 200 units and then cancel the task manually. The sync command is:  pulp-admin puppet repo sync run --repo-id=forge
  - pulp-admin puppet repo create --repo-id forge_copy
  - pulp-admin puppet repo copy --filters {} --from-repo-id forge --to-repo-id forge_copy
  - pulp-admin puppet repo remove --filters {} --repo-id forge
  - pulp-admin puppet repo copy --match author=lvillani --from-repo-id forge_copy --top-repo-id forge

3. Create a directory to publish into,and give it the right permissions
  - mkdir /etc/puppet/forge
  - sudo chown -R apache:apache /etc/puppet/forge

4. Create the install distributor named 'puppet_install_distributor' by running:

pic.POST('/v2/repositories/forge/distributors/', {'distributor_type_id':'puppet_install_distributor', 'distributor_id':'puppet_install_distributor', 'auto_publish': False, 'distributor_config': {'install_path':'/etc/puppet/forge'}})
Comment 14 Randy Barlow 2014-10-21 16:40:09 EDT
In addition to the other pull requests referenced in this BZ, this is also required:

Comment 15 Dennis Kliban 2014-10-22 15:14:39 EDT
included in pulp-server 2.5.0-0.11
Comment 16 Preethi Thomas 2014-10-23 12:58:58 EDT

For rhel7 

you need to run
semanage boolean --modify --on pulp_manage_puppet

So the doc needs to be updated to reflect that.

Creating the install distributor and publishing succeeded.
[root@cloud-qe-1-vm-1 ~]# rpm -qa pulp-server
[root@cloud-qe-1-vm-1 ~]# 
[root@cloud-qe-1-vm-1 ~]#
Comment 17 Randy Barlow 2014-10-27 15:47:32 EDT
In addition to the previous pull requests, this one fixes the documentation:


Note You need to log in before you can comment on or make changes to this bug.