Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1192581 - hammer-cli-import broken due to API changes in Sat6.1
hammer-cli-import broken due to API changes in Sat6.1
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Transitions (Show other bugs)
6.1.0
Unspecified Unspecified
urgent Severity urgent (vote)
: Unspecified
: Unused
Assigned To: Grant Gainey
Lukas Pramuk
: Regression, Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-02-13 13:37 EST by Grant Gainey
Modified: 2017-02-23 15:30 EST (History)
8 users (show)

See Also:
Fixed In Version: rubygem-hammer_cli_import-0.10.9.1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-08-12 01:25:46 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
hammer_cli_import-0.10.6.gem (74.50 KB, application/octet-stream)
2015-02-16 12:51 EST, Grant Gainey
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1592 normal SHIPPED_LIVE Important: Red Hat Satellite 6.1.1 on RHEL 6 2015-08-12 05:04:35 EDT

  None (edit)
Description Grant Gainey 2015-02-13 13:37:34 EST
Description of problem:

Once BZ#1192556 is fixed and hammer-cli-import can talk to the Sat6 server, it breaks due to API changes.

Version-Release number of selected component (if applicable):

hammer-cli-import 0.10.6
Sat6-nightly (13-FEB-2015)

How reproducible:

- Update hammer-cli-import to 0.10.6
- Attempt to import exported organizations:
- Get following traceback and be sad:

[root@ibm-ls21-04 hammer-cli-import]# hammer -v import organization --csv-file /tmp/exports/users.csv --debug --verbose
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:68: warning: already initialized constant DEFAULT_LABEL_INDENT
Importing from /tmp/exports/users.csv
Creating new organization: RED HAT SATELLITE ENGINEERING
entity_hash: {:description=>"Imported 'RED HAT SATELLITE ENGINEERING' organization from Red Hat Satellite 5", :name=>"RED HAT SATELLITE ENGINEERING", :id=>1}
Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: name while processing CSV line:
{"organization_id"=>"1", "organization"=>"RED HAT SATELLITE ENGINEERING"}
/usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/action.rb:62:in `validate!'
/usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:158:in `call'
/usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/resource.rb:14:in `call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:127:in `api_call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:139:in `api_call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:144:in `mapped_api_call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:425:in `_create_entity'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:379:in `create_entity'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/organization.rb:84:in `import_single_row'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:520:in `call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:520:in `cvs_iterate'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/importtools.rb:302:in `call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/importtools.rb:302:in `handle_missing_and_supress'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:519:in `cvs_iterate'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/csvhelper.rb:52:in `csv_each'
/usr/lib/ruby/gems/1.8/gems/fastercsv-1.5.4/lib/faster_csv.rb:1545:in `each'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/csvhelper.rb:43:in `csv_each'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:518:in `cvs_iterate'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:526:in `import'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:562:in `execute'
/usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:23:in `run'
/usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in `execute'
/usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:23:in `run'
/usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in `execute'
/usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:23:in `run'
/usr/lib/ruby/gems/1.8/gems/clamp-0.6.2/lib/clamp/command.rb:125:in `run'
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/bin/hammer:108
/usr/bin/hammer:19:in `load'
/usr/bin/hammer:19
Summary
  No action taken.

Actual results:

Traceback

Expected results:

Entities created

Additional info:

This all Just Works in 6.0.x. Something in 6.1 has invalidated hammer-cli-import.
Comment 2 Grant Gainey 2015-02-13 14:50:22 EST
Output from /var/log/foreman/production.log while getting the above traceback:
===
Processing by Katello::Api::V2::OrganizationsController#index as application/json;version=2
  Parameters: {"per_page"=>"999999", "api_version"=>"v2", "organization"=>{}}
Authorized user admin(Admin User)
  Rendered /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/views/katello/api/v2/organizations/index.json.rabl within api/v2/layouts/index_layout (6.5ms)
Completed 200 OK in 53ms (Views: 7.9ms | ActiveRecord: 13.9ms)
===

The code to call the API is this:
===
      # Call API. Ideally accessed via +api_call+ instance method.
      # This is supposed to be the only way to access @api.
      def api_call(resource, action, params = {}, headers = {}, dbg = false)
        @api.resource(resource).call(action, params, headers)
      rescue
        puts("Error on api.resource(#{resource.inspect}).call(#{action.inspect}, #{params.inspect}):")
        raise
      end

===

which is failing like this:
===
Error on api.resource(:organizations).call(:create, {:description=>"Imported 'RED HAT SATELLITE ENGINEERING' organization from Red Hat Satellite 5", :name=>"RED HAT SATELLITE ENGINEERING", :id=>1}):
===

because of this error-return:
===
Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: name
===
Comment 3 Grant Gainey 2015-02-13 15:44:30 EST
As a test, modified the call-code for organization-create specifically to be this:
===
        p = {'organization'=>params}
        @api.resource(resource).call(action, p, headers)
===

and it failed with the same error:
===
Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: name
===

while spitting out the following debug output:
===
Error on api.resource(:organizations).call(:create, {"organization"=>{:description=>"Imported 'RED HAT SATELLITE ENGINEERING' organization from Red Hat Satellite 5", :name=>"RED HAT SATELLITE ENGINEERING", :id=>1}}):
===

and the following production.log statement:
===
Processing by Katello::Api::V2::OrganizationsController#index as application/json;version=2
  Parameters: {"organization"=>{}, "api_version"=>"v2"}
Authorized user admin(Admin User)
  Rendered /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/views/katello/api/v2/organizations/index.json.rabl within api/v2/layouts/index_layout (7.1ms)
Completed 200 OK in 50ms (Views: 9.2ms | ActiveRecord: 10.5ms)
===
Comment 4 Martin Bacovsky 2015-02-16 05:03:30 EST
The cause of this error is a validation of parameters that was added to apipie-bindings recently. It currently checks just required parameters according to apipie docs.

In this case the problem originates in somewhat inconsistent API parameter format in Katello and Foreman. Unfortunately Organization is resource from Foreman that is extended in Katello and it has both nested and not nested parameter format (See https://<your-satellite>/apidoc/v2/organizations/create.html) I believe that Organization PUT/POST is the only resource where the problem is present.

The solution for the issue you are experiencing would be to fix the parameters you are sending to match the API documentation (or fix the API/API documentation). Minimal Org params should look like:

<pre>
{
            "name" => "myOrg",
    "organization" => {
        "name" => "myOrg"
    }
}
</pre>
Comment 5 Grant Gainey 2015-02-16 12:51:33 EST
Created attachment 992298 [details]
hammer_cli_import-0.10.6.gem

jsherrill requested a reproducer, so here's one.

Install the attached gem, hammer_cli_import-0.10.6.gem, in order to get past the problem in BZ#1192581

Put the below into a file, /tmp/users.csv:

===
organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active
1,RED HAT SATELLITE ENGINEERING,1,admin,Nimda,Admin,,root@localhost,Organization Administrator;Satellite Administrator,2015-01-28 18:38:36,2015-02-11 19:13:25,enabled
===

and then run the following import command:

===
hammer import organization --csv-file /tmp/users.csv --debug --verbose
===

and watch it fail:

===
(1192581) ~/hammer-cli-import $ hammer import organization --csv-file /tmp/exports/users.csv --debug --verbose 
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:68: warning: already initialized constant DEFAULT_LABEL_INDENT
Importing from /tmp/exports/users.csv
Creating new organization: RED HAT SATELLITE ENGINEERING
entity_hash: {:description=>"Imported 'RED HAT SATELLITE ENGINEERING' organization from Red Hat Satellite 5", :name=>"RED HAT SATELLITE ENGINEERING", :id=>1}
Caught ApipieBindings::MissingArgumentsError:ApipieBindings::MissingArgumentsError: name while processing CSV line:
{"organization_id"=>"1", "organization"=>"RED HAT SATELLITE ENGINEERING"}
/usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/action.rb:62:in `validate!'
/usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/api.rb:158:in `call'
/usr/lib/ruby/gems/1.8/gems/apipie-bindings-0.0.11/lib/apipie_bindings/resource.rb:14:in `call'
/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.6/lib/hammer_cli_import/base.rb:134:in `api_call'
===

Be sad.
Comment 6 Adam Price 2015-02-16 14:09:02 EST
@grant if you actually want debug and verbose output, they need to come before the commands, because they're global flags.

so:
hammer -v -d import organization --csv-file /tmp/users.csv
Comment 7 Grant Gainey 2015-02-16 14:45:42 EST
hammer-cli-import recognizes its own --debug and --verbose.  -v/-d control output from the underlying hammer-infrastructure.
Comment 8 Grant Gainey 2015-02-18 09:59:07 EST
Fixing by special-casing org-create-call.

hammer-cli-import.github 59faf5b8cea6405c8d1d7ac07d1e94ce1ad09d49
Comment 9 Grant Gainey 2015-02-18 12:55:12 EST
Modified by hammer-cli-import.github c106fa0a884567922124cba9108c7b88148dea34
Comment 13 Lukas Pramuk 2015-03-10 06:30:57 EDT
rubygem-hammer_cli_import-0.10.9.1
no such version on any compose?! ON_QA?
Comment 16 Lukas Pramuk 2015-03-10 10:16:41 EDT
VERIFIED. rubygem-hammer_cli_import-0.10.6.3-1.el6_6sat

# hammer import organization --verbose --csv-file /tmp/exports/users.csv --upload-manifests-from '/tmp/manifests''
/usr/lib/ruby/gems/1.8/gems/hammer_cli-0.1.4/lib/hammer_cli/./apipie/../abstract.rb:68: warning: already initialized constant DEFAULT_LABEL_INDENT
Importing from /tmp/exports/users.csv
Creating new organization: RED HAT SATELLITE ENGINEERING
Uploading manifest /tmp/manifests/RED_HAT_SATELLITE_ENGINEERING.zip to org-id 3
Waiting for the task [4b8d6deb-4594-4913-affb-ef5e30b47bcb] .................
Creating new organization: Organization #2
Uploading manifest /tmp/manifests/Organization__2.zip to org-id 4
Waiting for the task [6f5e9af6-4a48-4f7c-829b-7b83f52f4714] ..........
Creating new organization: Organization #3
Uploading manifest /tmp/manifests/Organization__3.zip to org-id 5
Waiting for the task [3946507d-7ec2-48f4-8924-41d71e3d97d8] ..........
Summary
  Created 3 organizations.
  Uploaded 3 manifests.
Comment 17 Bryan Kearney 2015-08-11 09:23:09 EDT
This bug is slated to be released with Satellite 6.1.
Comment 18 errata-xmlrpc 2015-08-12 01:25:46 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:1592

Note You need to log in before you can comment on or make changes to this bug.