| Summary: | hammer import config-file on RHEL7 fails to generate puppet modules | ||
|---|---|---|---|
| Product: | Red Hat Satellite | Reporter: | Lukas Pramuk <lpramuk> |
| Component: | Uncategorized | Assignee: | Tom McKay <tomckay> |
| Status: | CLOSED ERRATA | QA Contact: | Lukas Pramuk <lpramuk> |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 6.2.0 | CC: | ahumbe, bbuckingham, bkearney, ehelms, lpramuk |
| Target Milestone: | Unspecified | Keywords: | Regression, Triaged |
| Target Release: | Unused | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| URL: | http://projects.theforeman.org/issues/14616 | ||
| Whiteboard: | |||
| Fixed In Version: | rubygem-hammer_cli_import-0.11.1-1 | Doc Type: | Bug Fix |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2016-07-27 11:02:03 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
Additional info:
lib/hammer_cli_import/configfile.rb:146 in `readline' - it fails to parse command output with interview questions at all on RHEL7.
puppet module generate command which is being run by hammer import works itself:
# cd /root/puppet_work_dir/
# puppet module generate satelliteqe-config_1
We need to create a metadata.json file for this module. Please answer the
following questions; if the question is not applicable to this module, feel free
to leave it blank.
Puppet uses Semantic Versioning (semver.org) to version modules.
What version is this module? [0.1.0]
-->
Who wrote this module? [satelliteqe]
-->
What license does this module code fall under? [Apache 2.0]
-->
How would you describe this module in a single sentence?
-->
Where is this module's source code repository?
-->
Where can others go to learn more about this module?
-->
Where can others go to file issues about this module?
-->
----------------------------------------
{
"name": "satelliteqe-config_1",
"version": "0.1.0",
"author": "satelliteqe",
"summary": null,
"license": "Apache 2.0",
"source": "",
"project_page": null,
"issues_url": null,
"dependencies": [
{"name":"puppetlabs-stdlib","version_requirement":">= 1.0.0"}
]
}
----------------------------------------
About to generate this metadata; continue? [n/Y]
--> Y
Notice: Generating module at /root/puppet_work_dir/satelliteqe-config_1...
Notice: Populating templates...
Finished; module generated in satelliteqe-config_1.
satelliteqe-config_1/Gemfile
satelliteqe-config_1/Rakefile
satelliteqe-config_1/manifests
satelliteqe-config_1/manifests/init.pp
satelliteqe-config_1/spec
satelliteqe-config_1/spec/classes
satelliteqe-config_1/spec/classes/init_spec.rb
satelliteqe-config_1/spec/spec_helper.rb
satelliteqe-config_1/tests
satelliteqe-config_1/tests/init.pp
satelliteqe-config_1/README.md
satelliteqe-config_1/metadata.json
lib/hammer_cli_import/configfile.rb:
rd = ''
until rd.include? '?'
rd = stdout.readline <<< 146: failing to parse output, RHEL6 is ok
#debug "Read #{rd}"
end
Created redmine issue http://projects.theforeman.org/issues/14616 from this bug Upstream bug component is Uncategorized Upstream bug component is Uncategorized *** Bug 1350192 has been marked as a duplicate of this bug. *** *** Bug 1315260 has been marked as a duplicate of this bug. *** Moving to POST since upstream bug http://projects.theforeman.org/issues/14616 has been closed FailedQA.
@satellite-6.2.0-20.1.el7sat.noarch
tfm-rubygem-hammer_cli_import-0.11.0-1.el7sat.noarch
1) # cat > users.csv << EOF
organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active
1,SatelliteQE,1,admin,Admin,Admin,,root@localhost,Satellite Administrator,2014-09-10 19:39:59,2016-04-07 01:00:14,enabled
EOF
2) # hammer import organization --verbose --csv-file users.csv
Importing from users.csv
Creating new organization: SatelliteQE
Summary
Created 1 organization.
3) # cat > config-files-latest.csv << EOF
org_id,channel_id,channel,channel_type,path,file_type,file_id,revision,is_binary,contents,delim_start,delim_end,username,groupname,filemode,symbolic_link,selinux_ctx
1,3,config-1,normal,/etc/config1,file,3,2,N,config1 file ver.2,{|,|},apache,apache,0,,
EOF
4) # hammer import config-file --debug --csv-file config-files-latest.csv --generate-only
Importing from config-files-latest.csv
Failed puppet module satelliteqe-config_1: metadata.json not created
Writing converted files
Summary
Failed 1 puppet module.
>>> failed to create puppet module (on rhel7)
...however, puppet module generate is successful:
# puppet module generate satelliteqe-config_1
We need to create a metadata.json file for this module. Please answer the
following questions; if the question is not applicable to this module, feel free
to leave it blank.
Puppet uses Semantic Versioning (semver.org) to version modules.
What version is this module? [0.1.0]
-->
Who wrote this module? [satelliteqe]
-->
What license does this module code fall under? [Apache 2.0]
-->
How would you describe this module in a single sentence?
-->
Where is this module's source code repository?
-->
Where can others go to learn more about this module?
-->
Where can others go to file issues about this module?
-->
----------------------------------------
{
"name": "satelliteqe-config_1",
"version": "0.1.0",
"author": "satelliteqe",
"summary": null,
"license": "Apache 2.0",
"source": "",
"project_page": null,
"issues_url": null,
"dependencies": [
{"name":"puppetlabs-stdlib","version_requirement":">= 1.0.0"}
]
}
----------------------------------------
About to generate this metadata; continue? [n/Y]
-->
Notice: Generating module at /root/puppet_work_dir/satelliteqe-config_1...
Notice: Populating templates...
Finished; module generated in satelliteqe-config_1.
satelliteqe-config_1/Gemfile
satelliteqe-config_1/Rakefile
satelliteqe-config_1/manifests
satelliteqe-config_1/manifests/init.pp
satelliteqe-config_1/spec
satelliteqe-config_1/spec/classes
satelliteqe-config_1/spec/classes/init_spec.rb
satelliteqe-config_1/spec/spec_helper.rb
satelliteqe-config_1/tests
satelliteqe-config_1/tests/init.pp
satelliteqe-config_1/README.md
satelliteqe-config_1/metadata.json
In previous comment I missed to paste one cmd: # cd /root/puppet_work_dir/ # puppet module generate satelliteqe-config_1 ... forgot to paste relevant part of import.log (with debug msg) I, [2016-07-18T07:52:36.146520 #30537] INFO -- : Importing from config-files-latest.csv D, [2016-07-18T07:52:36.148544 #30537] DEBUG -- : Removing /root/puppet_work_dir/satelliteqe-config_1 D, [2016-07-18T07:52:36.852570 #30537] DEBUG -- : Done reading I, [2016-07-18T07:52:36.852805 #30537] INFO -- : Failed puppet module satelliteqe-config_1: metadata.json not created D, [2016-07-18T07:52:36.853055 #30537] DEBUG -- : name etcconfig1, path /etc/config1, type file I, [2016-07-18T07:52:36.853158 #30537] INFO -- : Writing converted files I, [2016-07-18T07:52:36.853321 #30537] INFO -- : Summary I, [2016-07-18T07:52:36.853490 #30537] INFO -- : Failed 1 puppet module. On debug level I cannot find output of debug "Read #{rd}"
This means that whole parsing cycle L:143-153 is being skipped as was before and thrown EOFException is just masked.
Please see comment #2:
rd = stdout.readline <<< 146: failing to parse output, RHEL6 is ok
Running 'gem install json_pure' will fix the import of configfiles. Still investigating solution. VERIFIED. @satellite-6.2.0-20.3.el7sat.noarch tfm-rubygem-hammer_cli_import-0.11.1-1.el7sat.noarch used manual reproducer describe in comment #0 ... 4) # hammer import config-file --debug --csv-file config-files-latest.csv --generate-only Importing from config-files-latest.csv Writing converted files Found module defaultorganization-config_1 Summary Wrote 1 puppet module. Wrote 1 puppet file. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1501 |
Description of problem: hammer import config-file on RHEL7 fails to generate puppet modules. It uses "puppet module generate <name>" and on RHEL7 it fails to parse command interview questions at all, while on RHEL6 command output with interview questions is parsed successfully. Version-Release number of selected component (if applicable): @Sat6.2.0-Beta-Snap tfm-rubygem-hammer_cli_import-0.10.23-1.el7sat.noarch How reproducible: always on RHEL7 Steps to Reproduce: 1. # cat users.csv organization_id,organization,user_id,username,last_name,first_name,position,email,role,creation_time,last_login_time,active 1,SatelliteQE,1,admin,Admin,Admin,,root@localhost,Satellite Administrator,2014-09-10 19:39:59,2016-04-07 01:00:14,enabled 2. # hammer import organization --verbose --csv-file users.csv Importing from users.csv Creating new organization: SatelliteQE Summary Created 1 organization. 3. # cat config-files-latest.csv org_id,channel_id,channel,channel_type,path,file_type,file_id,revision,is_binary,contents,delim_start,delim_end,username,groupname,filemode,symbolic_link,selinux_ctx 1,3,config-1,normal,/etc/config1,file,3,2,N,config1 file ver.2,{|,|},apache,apache,0,, 4. # hammer import config-file --debug --csv-file config-files-latest.csv --generate-only Importing from config-files-latest.csv Caught EOFError:end of file reached while processing CSV line: {"org_id"=>"1", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/config1", "file_type"=>"file", "file_id"=>"3", "revision"=>"2", "is_binary"=>"N", "contents"=>"config1 file ver.2", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"apache", "groupname"=>"apache", "filemode"=>"0", "symbolic_link"=>nil, "selinux_ctx"=>nil} Writing converted files Found module satelliteqe-config_1 Caught Errno::ENOENT:No such file or directory @ dir_s_mkdir - /root/puppet_work_dir/satelliteqe-config_1/files while post_import Summary No action taken. 5. # less import.log: ... I, [2016-04-11T08:03:46.664346 #17137] INFO -- : Importing from config-files-latest.csv D, [2016-04-11T08:03:46.665697 #17137] DEBUG -- : Removing /root/puppet_work_dir/satelliteqe-config_1 E, [2016-04-11T08:03:47.255283 #17137] ERROR -- : Caught EOFError:end of file reached while processing CSV line: {"org_id"=>"1", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/config1", "file_type"=>"file", "file_id"=>"3", "revision"=>"2", "is_binary"=>"N", "contents"=>"config1 file ver.2", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"apache", "groupname"=>"apache", "filemode"=>"0", "symbolic_link"=>nil, "selinux_ctx"=>nil} E, [2016-04-11T08:03:47.255556 #17137] ERROR -- : /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:146:in `readline' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:146:in `block (2 levels) in generate_module_template_for' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:143:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:143:in `block in generate_module_template_for' /opt/rh/rh-ruby22/root/usr/share/ruby/open3.rb:199:in `popen_run' /opt/rh/rh-ruby22/root/usr/share/ruby/open3.rb:93:in `popen3' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:141:in `generate_module_template_for' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:193:in `generate_module' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/configfile.rb:251:in `import_single_row' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:527:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:527:in `block (2 levels) in cvs_iterate' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/importtools.rb:324:in `call' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/importtools.rb:324:in `handle_missing_and_supress' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:526:in `block in cvs_iterate' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/csvhelper.rb:55:in `block in csv_each' /opt/rh/rh-ruby22/root/usr/share/ruby/csv.rb:1739:in `each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/csvhelper.rb:44:in `csv_each' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:525:in `cvs_iterate' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:533:in `import' /opt/theforeman/tfm/root/usr/share/gems/gems/hammer_cli_import-0.10.23/lib/hammer_cli_import/base.rb:569:in `execute' Actual results: fails to generate a puppet module (afterwards it is uploaded) Expected results: generates a puppet module successfully (to be then uploaded onto server)