Bug 1215731 - Syncing a PULP_MANIFEST puppet repo over file:// fails with No such file or directory: u'///dir/modules.json'
Summary: Syncing a PULP_MANIFEST puppet repo over file:// fails with No such file or d...
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Docs Puppet Guide   
(Show other bugs)
Version: 6.0.8
Hardware: All
OS: Linux
high
medium vote
Target Milestone: 6.2
Assignee: Stephen Wadeley
QA Contact: Lucie Jirakova
URL:
Whiteboard:
Keywords: Triaged
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-04-27 14:58 UTC by Justin Sherrill
Modified: 2018-12-06 20:49 UTC (History)
22 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-18 08:06:08 UTC
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
complete logs with exception (28.82 KB, text/plain)
2016-04-14 12:38 UTC, Sachin Ghai
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Pulp Redmine 2167 Normal CLOSED - NOTABUG Cannot sync repositories from filesystem with selinux enabled 2016-08-12 12:30 UTC
Red Hat Bugzilla 1361417 None None None 2019-02-27 10:56 UTC

Internal Trackers: 1361417

Description Justin Sherrill 2015-04-27 14:58:48 UTC
Description of problem:

Following http://www.katello.org/docs/2.1/user_guide/puppet_integration/index.html

to generate a PULP_MANIFEST style repo and sync it works fine unless you use a file:// url to perform the sync.

If you do, the sync errors with:

"FileRetrievalException: [Errno 2] No such file or directory: u'///var/www/html/puppetsync/modules.json'",

Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 240, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/async/tasks.py", line 328, in __call__
    return super(Task, self).__call__(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/celery/app/trace.py", line 437, in __protected_call__
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/repo/sync.py", line 114, in sync
    raise PulpExecutionException(_('Importer indicated a failed response'))



Version-Release number of selected component (if applicable):

pulp-server-2.6.0.1-1.beta.1.el6_6sat.noarch

How reproducible:
Always

Steps to Reproduce:
1. Use pulp-puppet-module-builder to generate a PULP_MANIFEST style puppet repo
2. Attempt to sync the repo using a file:// url


Actual results:
Failure

Expected results:
Success

Additional info:

Comment 1 Justin Sherrill 2015-04-27 14:59:53 UTC
needinfo'ing the pulp team, as this seems to be a pulp issue

Comment 3 Michael Hrivnak 2015-04-27 22:27:43 UTC
The pulp docs recommend a whopping 3 slashes, as in file:///home/me/mystuff/

https://pulp-puppet.readthedocs.org/en/latest/user-guide/recipes.html#building-and-importing-modules

Did you try that?

Comment 4 Justin Sherrill 2015-04-27 23:59:57 UTC
Michael,

yes, file urls use 3 slashes, two for the protocol, one for the actual path, notice in the log:

No such file or directory: u'///var/www/html/puppetsync/modules.json'",

even shows the 3 slashes.  The actual url i used was:  file:///var/www/html/puppetsync/

-Justin

Comment 5 Michael Hrivnak 2015-05-18 22:14:07 UTC
Jeff, I think you have knowledge of this feature. Can you take a look and file an upstream bug if necessary?

Comment 6 Jeff Ortel 2015-06-15 14:56:18 UTC
Justin, Can you attach a tarball containing the contents of your: /var/www/html/puppetsync/ directory?

Comment 7 Justin Sherrill 2015-06-15 21:07:49 UTC
I don't have that data any more, but it is trivial to reproduce.  If you have any trouble or really need the data I used let me know

Comment 8 Erik van Pienbroek 2015-07-08 13:51:21 UTC
I'm running into this issue as well and it is still valid with the upstream Katello 2.2.

It's easy to reproduce. Just use pulp-puppet-module-builder, add a product+repo for it in Katello and perform a sync. The sync job will always fail with the message that a file named modules.json could not be found in the directory generated by pulp-puppet-module-builder.

The directory generated by pulp-puppet-module-builder contains several tarballs (one for each Puppet module) and a file named 'PULP_MANIFEST'. There's no file named 'modules.json'.

Full job details can be found at http://fpaste.org/241285/36362367/
The most interesting pieces of this job details are:
  progress_report:
    puppet_importer:
<<snip>>
      metadata:
        query_finished_count: 0
        error_message: Error downloading metadata
        execution_time: 0
        query_total_count: 1
        traceback:
        - - /usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/forge.py
          - 124
          - _parse_metadata
          - metadata_json_docs = downloader.retrieve_metadata(self.progress_report)
        - - /usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/downloaders/local.py
          - 70
          - retrieve_metadata
          - raise FileRetrievalException(report.error_msg)
        state: failed
        error: ! 'FileRetrievalException: [Errno 2] No such file or directory: u''//modules/modules.json'''
        current_query: modules/modules.json

Comment 13 Michael Hrivnak 2015-10-30 19:38:00 UTC
I verified that this functionality works correctly with upstream pulp 2.6.4.

Can you attach log output that was generated during the attempted sync? I suspect a filesystem permission or selinux issue is preventing pulp from accessing PULP_MANIFEST. In the log, there may be a statement such as:

ERROR: Fetch URL: <your url>/PULP_MANIFEST failed: [Errno 13] Permission denied: ...

The importer can sync either from a PULP_MANIFEST style repo, or a forge style. If the former fails, it tries the latter, which is why you're seeing the error about modules.json. Look a little further back in the log, and hopefully you'll see an error explaining why PULP_MANIFEST is not accessible.

Comment 15 Justin Sherrill 2016-01-05 21:57:02 UTC
I'm actually no longer able to reproduce this, it seems like it was fixed in Satellite 6.1 sometime before 6.1.5.  

Leo, I tested with your customer's modules as well and it seemed to work fine.  Is the customer still seeing this on a fully updated 6.1.5?

Comment 16 Leo Thomas 2016-01-06 21:22:21 UTC
Hello Justin, 

That case got closed after we provided him the workaround. So I am not sure whether the issue still persist or not with the latest version. However, at that time with the customer's module we were able to reproduce the issue.  If it can't be reproduced with the latest Satellite version the issue might have fixed.

Comment 19 Sachin Ghai 2016-04-14 12:37:01 UTC
Verified with sat6.2 beta snap8.1

I copied the modules on filesystem under /modules and changed the dir permissions to '755' and synced the modules by setting url file:/// and I'm able to reproduce the issue:


==> /var/log/messages <==
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.directory:ERROR: Fetch URL: file:///modules/PULP_MANIFEST failed: [Errno 13] Permission denied: u'///modules/PULP_MANIFEST'
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:INFO: Beginning sync for repository <Default_Organization-puppet-puppetgit>
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:INFO: Beginning metadata retrieval for repository <Default_Organization-puppet-puppetgit>
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960) Exception while retrieving metadata for repository <Default_Organization-puppet-puppetgit>
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960) Traceback (most recent call last):
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960)   File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/forge.py", line 113, in _parse_metadata
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960)     metadata_json_docs = downloader.retrieve_metadata(self.progress_report)
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960)   File "/usr/lib/python2.7/site-packages/pulp_puppet/plugins/importers/downloaders/local.py", line 58, in retrieve_metadata
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960)     raise FileRetrievalException(report.error_msg)
Apr 14 08:30:35 cloud-qe-3 pulp: pulp_puppet.plugins.importers.forge:ERROR: (7753-16960) FileRetrievalException: FileRetrievalException: [Errno 2] No such file or directory: u'///modules/modules.json'


Looks like selinux issue:

type=AVC msg=audit(1460637035.939:5311): avc:  denied  { read } for  pid=7753 comm="celery" name="PULP_MANIFEST" dev="dm-0" ino=162121353 scontext=system_u:system_r:celery_t:s0 tcontext=unconfined_u:object_r:admin_home_t:s0 tclass=file
type=SYSCALL msg=audit(1460637035.939:5311): arch=c000003e syscall=2 success=no exit=-13 a0=3977ee0 a1=0 a2=1b6 a3=24 items=0 ppid=7498 pid=7753 auid=4294967295 uid=48 gid=48 euid=48 suid=48 fsuid=48 egid=48 sgid=48 fsgid=48 tty=(none) ses=4294967295 comm="celery" exe="/usr/bin/python2.7" subj=system_u:system_r:celery_t:s0 key=(null)

Comment 20 Sachin Ghai 2016-04-14 12:38 UTC
Created attachment 1147231 [details]
complete logs with exception

Comment 25 Brad Buckingham 2016-07-24 16:08:19 UTC
Created redmine issue http://projects.theforeman.org/issues/15812 from this bug

Comment 26 David Davis 2016-08-11 15:15:24 UTC
Opened an issue against pulp since I was able to reproduce with just pulp-admin:

https://pulp.plan.io/issues/2167

Comment 27 David Davis 2016-08-11 20:50:34 UTC
We need to update the docs text. See:

https://pulp.plan.io/issues/1560

Comment 28 Bryan Kearney 2016-08-11 22:06:56 UTC
Upstream bug assigned to daviddavis@redhat.com

Comment 29 Bryan Kearney 2016-08-11 22:07:01 UTC
Upstream bug component is Repositories

Comment 30 pulp-infra@redhat.com 2016-08-12 12:30:24 UTC
The Pulp upstream bug status is at CLOSED - NOTABUG. Updating the external tracker on this bug.

Comment 31 pulp-infra@redhat.com 2016-08-12 12:30:29 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 32 David Davis 2016-08-12 17:44:22 UTC
We'll need some documentation around how to set SELinux labels on files. Here's an example:

"If you have SELinux enabled, in order to sync from the file system, you'll need to apply a label to the files in order for Satellite 6 to access them. Two options are httpd_sys_r_content_t or pulp_tmp_t. Note: if you choose httpd_sys_r_content_t then the webserver can also read the files so that may or may not be good. One way to apply these labels would be to use chcon. See https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Security-Enhanced_Linux/sect-Security-Enhanced_Linux-Working_with_SELinux-SELinux_Contexts_Labeling_Files.html for more info."

This applies to all content types (not just puppet).

Comment 33 Bryan Kearney 2016-08-12 18:07:05 UTC
Upstream bug assigned to daviddavis@redhat.com

Comment 34 Bryan Kearney 2016-08-12 18:07:11 UTC
Upstream bug component is Repositories

Comment 35 Bryan Kearney 2016-08-12 18:07:16 UTC
Upstream bug assigned to daviddavis@redhat.com

Comment 36 Andrew Dahms 2016-11-01 13:19:30 UTC
Moving to 'NEW' while assigned to the default assignee.

Comment 37 Stephen Wadeley 2016-11-01 14:23:37 UTC
Hello


As per 
Bug 1301367 - pulp-puppet-module-builder and SELinux 

the guide was updated to use /var/www/puppet-modules

See "Procedure 3.3. Publishing a Git Repository to a Local Directory" [1]



[1] https://access.redhat.com/documentation/en/red-hat-satellite/6.2/paged/puppet-guide/35-adding-puppet-modules-from-a-git-repository

Comment 38 Stephen Wadeley 2016-11-01 15:32:18 UTC
Hello

I grep'd al guides and the use of "pulp_tmp_t" and "httpd_sys_r_content_t" are not documented. So I will add them to the doc mentioned in comment 37.

Comment 40 Andrew Dahms 2016-11-02 01:23:27 UTC
Assigning to Stephen for review.


Note You need to log in before you can comment on or make changes to this bug.