Red Hat Bugzilla – Bug 1312640
hammer repo export throws error: unknown file type:
Last modified: 2016-07-27 05:24:05 EDT
1) Create and Sync (immediate) a repo 2) Export: # hammer repository export --id 32 [...........................................] [100%] unknown file type: /var/lib/pulp/published/yum/master/group_export_distributor/Default_Organization-Test_Product-synced-repo
This appears to be http://projects.theforeman.org/issues/13781 which is fixed in upstream. # hammer repository export --id 32 Ignoring ruby-libvirt-0.5.2 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.5.2 [.................................................................................................................................] [100%] unknown file type: /var/lib/pulp/published/yum/master/group_export_distributor/Default_Organization-Test_Product-synced-repo [root@sat-r220-06 ~]# setenforce 0 [root@sat-r220-06 ~]# hammer repository export --id 32 Ignoring ruby-libvirt-0.5.2 because its extensions are not built. Try: gem pristine ruby-libvirt --version 0.5.2 [.................................................................................................................................] [100%] ************************* # audit2allow -a #============= passenger_t ============== allow passenger_t httpd_sys_rw_content_t:dir { read search open getattr }; allow passenger_t httpd_sys_rw_content_t:file { read getattr open ioctl }; #============= streamer_t ============== allow streamer_t tmp_t:dir write; ************************ NOTE: the streamer_t denial is not related to export. I believe it's an unrelated issue.
Upstream bug component is Content Management
Moving to POST since upstream bug http://projects.theforeman.org/issues/13781 has been closed
[root@ibm-x3250m4-01 ~]# LANG=en_US.UTF-8 hammer -v -u admin -p changeme repository export --id 1 [ERROR 2016-03-18 16:13:51 Exception] ERF42-3196 [Foreman::Exception]: Unable to export, 'pulp_export_destination' setting is not set to a valid directory. Could not export the repository: ERF42-3196 [Foreman::Exception]: Unable to export, 'pulp_export_destination' setting is not set to a valid directory. [ERROR 2016-03-18 16:13:51 Exception] RestClient::InternalServerError (500 Internal Server Error): /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:178:in `block in transmit' /opt/rh/rh-ruby22/root/usr/share/ruby/net/http.rb:853:in `start' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:172:in `transmit' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/api.rb:286:in `call_client' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/api.rb:217:in `http_call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/api.rb:162:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/resource.rb:14:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/apipie/command.rb:43:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.2/lib/hammer_cli_foreman/commands.rb:189:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman_tasks-0.0.10/lib/hammer_cli_foreman_tasks/async_command.rb:14:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/abstract.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/subcommand/execution.rb:11:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/abstract.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/subcommand/execution.rb:11:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/abstract.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:133:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/bin/hammer:125:in `<top (required)>' /usr/bin/hammer:23:in `load' /usr/bin/hammer:23:in `<main>'
Created attachment 1137866 [details] Settings for exporting It is necessary to first define a path (writable by the apache user) for the 'pulp_export_destination' setting, as shown here.
To verify this issue: * Update your 'pulp_export_destination' setting and set it to "/var/www/html/pub" (web UI, Settings menu, search for pulp_export_destination) * Create a repository (I used an existing, synchronized RHEL repo) and sync it * Use the hammer repository export command as per first comment here
Created attachment 1137881 [details] Stacktrace generated when exporting a repository via hammer
Mike McCune also could not test this feature, so I am failing it. Tested against Satellite 6.2.0 SNAP 4.0 build
There are a few issues found: 1. there was a missing cherry-pick for katello-selinux which will be in the next snap. I ran "audit2allow -a" and saw "allow passenger_t httpd_sys_rw_content_t:dir search;". After updating the selinux policy, you should see a message like "#!!!! This avc is allowed in the current policy" which indicates that selinux is copacetic. Note that fresh installs that didnt have a prior denial here will not have any message, it will just work:) 2. the directory needs to be owned by foreman user and group, not apache. This is documented in the upstream docs at http://www.katello.org/docs//user_guide/disconnected/, downstream docs are being updated for this (https://bugzilla.redhat.com/show_bug.cgi?id=1285244#c50 and https://bugzilla.redhat.com/show_bug.cgi?id=1285244#c55) I think it was apache.apache in the past, probably on one of the etherpads at one point. 3. default selinux policy does not allow passenger to write to /var/www/html/*, even if the file-level permissions are correct. Typically I export repos to /mnt/exports, so I can pretend that I'm exporting to a mount that can be shared. I don't know how common it will be to export to /var/www/html/pub for the disconnected use case, since it short-circuits the "export, walk it over, then import" flow. Having said that, we can add an additional selinux rule to allow issue 3 if you think it would be helpful. It would be impossible to guess all the places someone would export and create rules in advance, so maybe a kbase article would be more helpful that shows how to do it generically. Marking bz as NEEDSINFO on omaciel for to get feedback on third issue.
Something I should have mentioned, for issue 3 in comment #11 the selinux denial only affects certain directories like /var/www/html/. Dirs like /mnt/export should be unaffected and do not require additional rules.
Hi Chris, [root@ibm-x3250m4-01 pub]# mkdir /mnt/export [root@ibm-x3250m4-01 pub]# chown foreman.foreman /mnt/export [root@ibm-x3250m4-01 pub]# ls -l /mnt/export/ total 0 [root@ibm-x3250m4-01 pub]# ls -ld /mnt/export/ drwxr-xr-x. 2 foreman foreman 6 Mar 21 14:34 /mnt/export/ [root@ibm-x3250m4-01 pub]# LANG=en_US.UTF-8 hammer -v -u admin -p changeme repository export --id 1 [ERROR 2016-03-21 14:35:06 Exception] ERF42-6337 [Foreman::Exception]: Unable to export, 'pulp_export_destination' setting is not a writable directory. Could not export the repository: ERF42-6337 [Foreman::Exception]: Unable to export, 'pulp_export_destination' setting is not a writable directory. [ERROR 2016-03-21 14:35:06 Exception] RestClient::InternalServerError (500 Internal Server Error): /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/abstract_response.rb:48:in `return!' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:230:in `process_result' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:178:in `block in transmit' /opt/rh/rh-ruby22/root/usr/share/ruby/net/http.rb:853:in `start' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:172:in `transmit' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in `post' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/api.rb:286:in `call_client' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/api.rb:217:in `http_call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/api.rb:162:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-bindings-0.0.14/lib/apipie_bindings/resource.rb:14:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/apipie/command.rb:43:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman-0.5.1.2/lib/hammer_cli_foreman/commands.rb:189:in `send_request' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_foreman_tasks-0.0.10/lib/hammer_cli_foreman_tasks/async_command.rb:14:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/abstract.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/subcommand/execution.rb:11:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/abstract.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/subcommand/execution.rb:11:in `execute' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:68:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/lib/hammer_cli/abstract.rb:22:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/clamp-1.0.0/lib/clamp/command.rb:133:in `run' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli-0.5.1.3/bin/hammer:125:in `<top (required)>' /usr/bin/hammer:23:in `load' /usr/bin/hammer:23:in `<main>' By the way, I think that a KB article would be great for those who like myself are struggling a bit :)
Og, I created https://bugzilla.redhat.com/show_bug.cgi?id=1321589 so you don't have to do the steps noted in issue 3. Jitendra hit it as well, so IMO it is a common enough use case to warrant adding a rule.
QE: I have been able to test this issue using a small YUM repo but had issues when exporting a large-ish Red Hat repository. To properly verify this issue I think we need to: * Test exporting Background download policy repos * Test exporting OnDemand download policy repos * Verify https://bugzilla.redhat.com/show_bug.cgi?id=1323730
New BZ to raise a correct error message when exporting individual non-yum repos BZ #1330166
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501