Bug 1165151 - hammer import content-view is failing with "Error: can't convert nil into Integer"
Summary: hammer import content-view is failing with "Error: can't convert nil into Int...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Transitions
Version: 6.0.4
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: Unspecified
Assignee: Grant Gainey
QA Contact: Roman Plevka
URL:
Whiteboard:
Depends On:
Blocks: GSS_Sat6Beta_Tracker, GSS_Sat6_Tracker
TreeView+ depends on / blocked
 
Reported: 2014-11-18 12:56 UTC by Neha
Modified: 2019-04-16 14:24 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-08-12 05:19:14 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2015:1592 0 normal SHIPPED_LIVE Important: Red Hat Satellite 6.1.1 on RHEL 6 2015-08-12 09:04:35 UTC

Description Neha 2014-11-18 12:56:55 UTC
Description of problem:
hammer import content-view is failing with "Error: can't convert nil into Integer"

hammer -d import content-view --csv-file /var/spacewalk-export/exports/CHANNELS/export.csv --dir /var/spacewalk-export/exports/CHANNELS 
[ INFO 2014-11-17 08:57:18 Init] Initialization of Hammer CLI (0.1.1) has started...
[DEBUG 2014-11-17 08:57:18 Init] Running at ruby 2.0.0-p353
[ INFO 2014-11-17 08:57:18 Init] Configuration from the file /etc/hammer/cli_config.yml has been loaded
[ INFO 2014-11-17 08:57:18 Init] Configuration from the file /etc/hammer/cli.modules.d/foreman.yml has been loaded
[ INFO 2014-11-17 08:57:18 Init] Configuration from the file /etc/hammer/cli.modules.d/import.yml has been loaded
[ INFO 2014-11-17 08:57:18 Init] Configuration from the file /etc/hammer/cli.modules.d/katello.yml has been loaded
[DEBUG 2014-11-17 08:57:18 Connection] Registered: foreman
[Foreman] username: mw225
[Foreman] password for mw225: 
[DEBUG 2014-11-17 08:57:26 API] Global headers: {
        :content_type => "application/json",
              :accept => "application/json;version=2",
    "Accept-Language" => "en"
}
[ INFO 2014-11-17 08:57:26 Modules] Extension module hammer_cli_foreman (0.1.1) loaded
[ INFO 2014-11-17 08:57:26 Modules] Extension module hammer_cli_import (0.10.4) loaded
[ INFO 2014-11-17 08:57:26 HammerCLI::MainCommand] subcommand organization (HammerCLIForeman::Organization) was removed.
[ INFO 2014-11-17 08:57:26 HammerCLI::MainCommand] subcommand organization (HammerCLIKatello::Organization) was created.
[ INFO 2014-11-17 08:57:26 Modules] Extension module hammer_cli_katello (0.0.4) loaded
[DEBUG 2014-11-17 08:57:26 Init] Using locale 'en'
[DEBUG 2014-11-17 08:57:26 Init] 'mo' files for locale domain 'hammer-cli' loaded from '/usr/share/gems/gems/hammer_cli-0.1.1/locale'
[DEBUG 2014-11-17 08:57:26 Init] 'mo' files for locale domain 'hammer-cli-foreman' loaded from '/usr/share/gems/gems/hammer_cli_foreman-0.1.1/locale'
[DEBUG 2014-11-17 08:57:26 Init] 'mo' files for locale domain 'hammer-cli-katello' loaded from '/usr/share/gems/gems/hammer_cli_katello-0.0.4/locale'
[ INFO 2014-11-17 08:57:26 HammerCLI::MainCommand] Called with options: {"option_debug"=>true}
[ INFO 2014-11-17 08:57:26 HammerCLIImport::ImportCommand] Called with options: {}
[ INFO 2014-11-17 08:57:26 HammerCLIImport::ImportCommand::LocalRepositoryImportCommand] Called with options: {"option_dir"=>"/var/spacewalk-export/exports/CHANNELS", "option_filter"=>false, "option_synchronize"=>false, "option_wait"=>false, "option_no_async"=>false, "option_csv_file"=>"/var/spacewalk-export/exports/CHANNELS/export.csv", "option_delete"=>false, "option_recover"=>:rename, "option_quiet"=>false, "option_debug"=>false, "option_verbose"=>false, "option_logfile"=>"/root/import.log"}
[ERROR 2014-11-17 08:57:27 Exception] Error: can't convert nil into Integer
Error: can't convert nil into Integer
[ERROR 2014-11-17 08:57:27 Exception] 

TypeError (can't convert nil into Integer):
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:29:in `Integer'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:29:in `from_s'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:205:in `block in pm_decode_row'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:203:in `each'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:203:in `pm_decode_row'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:118:in `block (3 levels) in load_persistent_maps'
    /usr/share/ruby/csv.rb:1717:in `each'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:117:in `block (2 levels) in load_persistent_maps'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:113:in `each'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:113:in `block in load_persistent_maps'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:111:in `each'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb:111:in `load_persistent_maps'
    /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/base.rb:548:in `execute'
    /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
    /usr/share/gems/gems/hammer_cli-0.1.1/lib/hammer_cli/abstract.rb:22:in `run'
    /usr/share/gems/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in `execute'
    /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
    /usr/share/gems/gems/hammer_cli-0.1.1/lib/hammer_cli/abstract.rb:22:in `run'
    /usr/share/gems/gems/clamp-0.6.2/lib/clamp/subcommand/execution.rb:11:in `execute'
    /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:67:in `run'
    /usr/share/gems/gems/hammer_cli-0.1.1/lib/hammer_cli/abstract.rb:22:in `run'
    /usr/share/gems/gems/clamp-0.6.2/lib/clamp/command.rb:125:in `run'
    /usr/share/gems/gems/hammer_cli-0.1.1/bin/hammer:102:in `<top (required)>'
    /usr/bin/hammer:23:in `load'
    /usr/bin/hammer:23:in `<main>'

Version-Release number of selected component (if applicable):


How reproducible:
Not Always

Steps to Reproduce:
hammer -d import content-view --csv-file /var/spacewalk-export/exports/CHANNELS/export.csv --dir /var/spacewalk-export/exports/CHANNELS

Actual results:
TypeError (can't convert nil into Integer)
Should import content view

Expected results:
Should import content view

Additional info:
Not reproducible internally

Comment 3 RHEL Program Management 2014-11-18 13:24:07 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 5 Grant Gainey 2014-12-04 18:30:47 UTC
I am having no luck in recreating this error. 

The stacktrace above comes during hammer-import initializing - it's loading up its record of what has happened on previous runs (that's what 'persistentmap.rb' manages)

Repeated runs, using the customer's data and the attached .transitions_data files, all seem to "just work". 

If one were to change this file:

/usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.2/lib/hammer_cli_import/persistentmap.rb

by adding the following:

===
*** /usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.2/lib/hammer_cli_import/persistentmap.rb.ORIG	2014-12-04 19:07:31.950036215 +0100
--- /usr/lib/ruby/gems/1.8/gems/hammer_cli_import-0.10.2/lib/hammer_cli_import/persistentmap.rb	2014-12-04 19:22:56.559390969 +0100
*************** module PersistentMap
*** 191,196 ****
--- 191,197 ----
      end
  
      def pm_decode_row(map_sym, row)
+       debug ">>> pm_decode_row, map_sym [#{map_sym}] row [#{row.inspect}]"
        key_spec, val_spec = self.class.map_description[map_sym]
        key = []
        value = []

===

and then run the command with --debug:

# hammer -d import content-view --csv-file /tmp/exports/CHANNELS/export.csv --dir /tmp/exports/CHANNELS  --debug

You will see output like this in /root/import.log:

===
D, [2014-12-04T19:23:00.774680 #19632] DEBUG -- : >>> pm_decode_row, map_sym [redhat_repositories] row [["2", "415", nil, "-"]]
===

This might give us a hint as to what's going on, should it happen again.

Comment 8 Grant Gainey 2014-12-05 13:30:31 UTC
Fun - here's the problem:

persistentmap.rb:28 

  def from_s(x)
      Integer(x)
  end

On machines running ruby-1.8.7.374-2.el6.x86_64, it returns a 0 when passed nil..

On machines running ruby-2.0.0.353-22.el7_0.x86_64, it throws an exception when passed nil..

As a result, processing .transition_data where entities have been deleted (and therefore have 'nil' for their sat6-ids), fails on RHEL7 where 2.0 is the default ruby.

While we're working on a fix - in .transition_data, find any files with '=' in the 'deleted' column, and try setting the sat6 column to -1 as a workaround.

Comment 9 Grant Gainey 2014-12-05 13:36:05 UTC
c#8 typo - s/with '=' in/with '-' in/

Comment 10 Grant Gainey 2014-12-05 13:47:05 UTC
Aaand here's the actual fix, works on both versions:

===
--- /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb.SAVE	2014-12-05 04:34:54.793460071 -0500
+++ /usr/share/gems/gems/hammer_cli_import-0.10.4/lib/hammer_cli_import/persistentmap.rb	2014-12-05 04:35:18.147460870 -0500
@@ -26,7 +26,7 @@
 
   class << Fixnum
     def from_s(x)
-      Integer(x)
+      Integer(x) rescue 0
     end
   end

Comment 12 Grant Gainey 2014-12-05 15:25:23 UTC
katello/hammer-cli-import commit d2147d41ee791e6907217353158ad3fd665e01fb

Comment 16 Roman Plevka 2015-05-19 15:29:34 UTC
VERIFIED

used data:
# head -n2 ../.transition_data/redhat_repositories-2015-05-18T14\:53\:21+0000.csv
org_id,channel_id,sat6,delete
1,101,,-


results:

Satellite-6.0.4-RHEL-7-20150227.0:
# hammer import content-view --verbose --csv-file CHANNELS/export.csv 
Error: can't convert nil into Integer


Satellite-6.1.0-RHEL-7-20150513.0:
# hammer import content-view --verbose --csv-file CHANNELS/export.csv 
Importing from CHANNELS/export.csv
No such {redhat_,}content_view: 105
Creating new content view: Clone of RHN Tools for RHEL (v. 6 for 64-bit x86_64)
Summary
  Created 1 content view.

Comment 17 Bryan Kearney 2015-08-11 13:33:27 UTC
This bug is slated to be released with Satellite 6.1.

Comment 18 errata-xmlrpc 2015-08-12 05:19:14 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2015:1592


Note You need to log in before you can comment on or make changes to this bug.