Hide Forgot
Description of problem: hammer import config-file on RHEL7 fails to generate puppet modules. It uses "puppet module generate <name>" and on RHEL7 it fails to parse command interview questions at all, while on RHEL6 command output with interview questions is parsed successfully. Version-Release number of selected component (if applicable): @Sat6.2.0-Beta-Snap tfm-rubygem-hammer_cli_import-0.10.23-1.el7sat.noarch How reproducible: always on RHEL7 Steps to Reproduce: 1. # cat users.csv organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active 1,SatelliteQE,1,admin,Admin,Admin,,root@localhost,Satellite Administrator,2014-09-10 19:39:59,2016-04-07 01:00:14,enabled 2. # hammer import organization --verbose --csv-file users.csv Importing from users.csv Creating new organization: SatelliteQE Summary Created 1 organization. 3. # cat config-files-latest.csv org_id,channel_id,channel,channel_type,path,file_type,file_id,revision,is_binary,contents,delim_start,delim_end,username,groupname,filemode,symbolic_link,selinux_ctx 1,3,config-1,normal,/etc/config1,file,3,2,N,config1 file ver.2,{|,|},apache,apache,0,, 4. # hammer import config-file --debug --csv-file config-files-latest.csv --generate-only Importing from config-files-latest.csv Caught EOFError:end of file reached while processing CSV line: {"org_id"=>"1", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/config1", "file_type"=>"file", "file_id"=>"3", "revision"=>"2", "is_binary"=>"N", "contents"=>"config1 file ver.2", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"apache", "groupname"=>"apache", "filemode"=>"0", "symbolic_link"=>nil, "selinux_ctx"=>nil} Writing converted files Found module satelliteqe-config_1 Caught Errno::ENOENT:No such file or directory @ dir_s_mkdir - /root/puppet_work_dir/satelliteqe-config_1/files while post_import Summary No action taken. 5. # less import.log: ... I, [2016-04-11T08:03:46.664346 #17137] INFO -- : Importing from config-files-latest.csv D, [2016-04-11T08:03:46.665697 #17137] DEBUG -- : Removing /root/puppet_work_dir/satelliteqe-config_1 E, [2016-04-11T08:03:47.255283 #17137] ERROR -- : Caught EOFError:end of file reached while processing CSV line: {"org_id"=>"1", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/config1", "file_type"=>"file", "file_id"=>"3", "revision"=>"2", "is_binary"=>"N", "contents"=>"config1 file ver.2", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"apache", "groupname"=>"apache", "filemode"=>"0", "symbolic_link"=>nil, "selinux_ctx"=>nil} E, [2016-04-11T08:03:47.255556 #17137] ERROR -- : /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:146:in `readline' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:146:in `block (2 levels) in generate_module_template_for' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:143:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:143:in `block in generate_module_template_for' /opt/rh/rh-ruby22/root/usr/share/ruby/open3.rb:199:in `popen_run' /opt/rh/rh-ruby22/root/usr/share/ruby/open3.rb:93:in `popen3' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:141:in `generate_module_template_for' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:193:in `generate_module' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:251:in `import_single_row' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:527:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:527:in `block (2 levels) in cvs_iterate' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/importtools.rb:324:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/importtools.rb:324:in `handle_missing_and_supress' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:526:in `block in cvs_iterate' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/csvhelper.rb:55:in `block in csv_each' /opt/rh/rh-ruby22/root/usr/share/ruby/csv.rb:1739:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/csvhelper.rb:44:in `csv_each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:525:in `cvs_iterate' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:533:in `import' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:569:in `execute' Actual results: fails to generate a puppet module (afterwards it is uploaded) Expected results: generates a puppet module successfully (to be then uploaded onto server)
Additional info: lib/hammer_cli_import/configfile.rb:146 in `readline' - it fails to parse command output with interview questions at all on RHEL7. puppet module generate command which is being run by hammer import works itself: # cd /root/puppet_work_dir/ # puppet module generate satelliteqe-config_1 We need to create a metadata.json file for this module. Please answer the following questions; if the question is not applicable to this module, feel free to leave it blank. Puppet uses Semantic Versioning (semver.org) to version modules. What version is this module? [0.1.0] --> Who wrote this module? [satelliteqe] --> What license does this module code fall under? [Apache 2.0] --> How would you describe this module in a single sentence? --> Where is this module's source code repository? --> Where can others go to learn more about this module? --> Where can others go to file issues about this module? --> ---------------------------------------- { "name": "satelliteqe-config_1", "version": "0.1.0", "author": "satelliteqe", "summary": null, "license": "Apache 2.0", "source": "", "project_page": null, "issues_url": null, "dependencies": [ {"name":"puppetlabs-stdlib","version_requirement":">= 1.0.0"} ] } ---------------------------------------- About to generate this metadata; continue? [n/Y] --> Y Notice: Generating module at /root/puppet_work_dir/satelliteqe-config_1... Notice: Populating templates... Finished; module generated in satelliteqe-config_1. satelliteqe-config_1/Gemfile satelliteqe-config_1/Rakefile satelliteqe-config_1/manifests satelliteqe-config_1/manifests/init.pp satelliteqe-config_1/spec satelliteqe-config_1/spec/classes satelliteqe-config_1/spec/classes/init_spec.rb satelliteqe-config_1/spec/spec_helper.rb satelliteqe-config_1/tests satelliteqe-config_1/tests/init.pp satelliteqe-config_1/README.md satelliteqe-config_1/metadata.json
lib/hammer_cli_import/configfile.rb: rd = '' until rd.include? '?' rd = stdout.readline <<< 146: failing to parse output, RHEL6 is ok #debug "Read #{rd}" end
Created redmine issue http://projects.theforeman.org/issues/14616 from this bug
Upstream bug component is Uncategorized
*** Bug 1350192 has been marked as a duplicate of this bug. ***
*** Bug 1315260 has been marked as a duplicate of this bug. ***
Moving to POST since upstream bug http://projects.theforeman.org/issues/14616 has been closed
FailedQA. @satellite-6.2.0-20.1.el7sat.noarch tfm-rubygem-hammer_cli_import-0.11.0-1.el7sat.noarch 1) # cat > users.csv << EOF organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active 1,SatelliteQE,1,admin,Admin,Admin,,root@localhost,Satellite Administrator,2014-09-10 19:39:59,2016-04-07 01:00:14,enabled EOF 2) # hammer import organization --verbose --csv-file users.csv Importing from users.csv Creating new organization: SatelliteQE Summary Created 1 organization. 3) # cat > config-files-latest.csv << EOF org_id,channel_id,channel,channel_type,path,file_type,file_id,revision,is_binary,contents,delim_start,delim_end,username,groupname,filemode,symbolic_link,selinux_ctx 1,3,config-1,normal,/etc/config1,file,3,2,N,config1 file ver.2,{|,|},apache,apache,0,, EOF 4) # hammer import config-file --debug --csv-file config-files-latest.csv --generate-only Importing from config-files-latest.csv Failed puppet module satelliteqe-config_1: metadata.json not created Writing converted files Summary Failed 1 puppet module. >>> failed to create puppet module (on rhel7)
...however, puppet module generate is successful: # puppet module generate satelliteqe-config_1 We need to create a metadata.json file for this module. Please answer the following questions; if the question is not applicable to this module, feel free to leave it blank. Puppet uses Semantic Versioning (semver.org) to version modules. What version is this module? [0.1.0] --> Who wrote this module? [satelliteqe] --> What license does this module code fall under? [Apache 2.0] --> How would you describe this module in a single sentence? --> Where is this module's source code repository? --> Where can others go to learn more about this module? --> Where can others go to file issues about this module? --> ---------------------------------------- { "name": "satelliteqe-config_1", "version": "0.1.0", "author": "satelliteqe", "summary": null, "license": "Apache 2.0", "source": "", "project_page": null, "issues_url": null, "dependencies": [ {"name":"puppetlabs-stdlib","version_requirement":">= 1.0.0"} ] } ---------------------------------------- About to generate this metadata; continue? [n/Y] --> Notice: Generating module at /root/puppet_work_dir/satelliteqe-config_1... Notice: Populating templates... Finished; module generated in satelliteqe-config_1. satelliteqe-config_1/Gemfile satelliteqe-config_1/Rakefile satelliteqe-config_1/manifests satelliteqe-config_1/manifests/init.pp satelliteqe-config_1/spec satelliteqe-config_1/spec/classes satelliteqe-config_1/spec/classes/init_spec.rb satelliteqe-config_1/spec/spec_helper.rb satelliteqe-config_1/tests satelliteqe-config_1/tests/init.pp satelliteqe-config_1/README.md satelliteqe-config_1/metadata.json
In previous comment I missed to paste one cmd: # cd /root/puppet_work_dir/ # puppet module generate satelliteqe-config_1 ...
forgot to paste relevant part of import.log (with debug msg) I, [2016-07-18T07:52:36.146520 #30537] INFO -- : Importing from config-files-latest.csv D, [2016-07-18T07:52:36.148544 #30537] DEBUG -- : Removing /root/puppet_work_dir/satelliteqe-config_1 D, [2016-07-18T07:52:36.852570 #30537] DEBUG -- : Done reading I, [2016-07-18T07:52:36.852805 #30537] INFO -- : Failed puppet module satelliteqe-config_1: metadata.json not created D, [2016-07-18T07:52:36.853055 #30537] DEBUG -- : name etcconfig1, path /etc/config1, type file I, [2016-07-18T07:52:36.853158 #30537] INFO -- : Writing converted files I, [2016-07-18T07:52:36.853321 #30537] INFO -- : Summary I, [2016-07-18T07:52:36.853490 #30537] INFO -- : Failed 1 puppet module.
On debug level I cannot find output of debug "Read #{rd}" This means that whole parsing cycle L:143-153 is being skipped as was before and thrown EOFException is just masked. Please see comment #2: rd = stdout.readline <<< 146: failing to parse output, RHEL6 is ok
Running 'gem install json_pure' will fix the import of configfiles. Still investigating solution.
VERIFIED. @satellite-6.2.0-20.3.el7sat.noarch tfm-rubygem-hammer_cli_import-0.11.1-1.el7sat.noarch used manual reproducer describe in comment #0 ... 4) # hammer import config-file --debug --csv-file config-files-latest.csv --generate-only Importing from config-files-latest.csv Writing converted files Found module defaultorganization-config_1 Summary Wrote 1 puppet module. Wrote 1 puppet file.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501