Red Hat Bugzilla – Bug 1226981
hammer import config-file fails with utf-8 encoded organization name
Last modified: 2017-02-23 14:58:12 EST
Description of problem: while trying to use hammer import config-file with csv file referring to an organization which name is UTF-8 encoded, the script fails to locate the puppet_work_dir for this org, as the directory name should consist of <org name>-config_1, but there's only '-config_1' directory on the filesystem. Version-Release number of selected component (if applicable): # hammer --version hammer (0.1.4.11) * hammer_cli_foreman (0.1.4.12) * hammer_cli_foreman_bootdisk (0.1.2.7) * hammer_cli_foreman_discovery (0.0.1.10) * hammer_cli_foreman_docker (unknown version) * hammer_cli_gutterball (unknown version) * hammer_cli_import (0.10.15) * hammer_cli_katello (0.0.7.13) How reproducible: export (sat5) and import (sat6) organization with name like: [U+217B1][U+22AC8][U+285C9][U+2931A][U+236F3]鯧[U+2AADE][U+28587][U+22EB6][U+2801F] and try to import config-file related to it using hammer import config-file. The script that is writing the puppet files will fail: # hammer import config-file --csv-file BM8sxwbVGZ.csv --generate-only Caught EOFError:end of file reached while processing CSV line: {"org_id"=>"4", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/sysconfig/rhn/systemid", "file_type"=>"file", "file_id"=>"8", "revision"=>"1", "is_binary"=>"N", "contents"=>"hostname={| rhn.system.hostname |}\nsys_ip_address={| rhn.system.ip_address |}\nip_address={| rhn.system.net_interface.ip_address(eth0) |}\nnetmask={| rhn.system.net_interface.netmask(eth0) |}\nmac_address={| rhn.system.net_interface.hardware_address(eth0) |}\n", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"root", "groupname"=>"root", "filemode"=>"600", "symbolic_link"=>nil, "selinux_ctx"=>nil} Writing converted files Caught Errno::ENOENT:No such file or directory - /root/puppet_work_dir/-config_1/files while post_import Summary No action taken. # hammer organization list ---|----------------------|--------------------------------------|------------ ID | NAME | LABEL | DESCRIPTION ---|----------------------|--------------------------------------|------------ 1 | Default Organization | Default_Organization | 49 | [U+217B1][U+22AC8][U+285C9][U+2931A][U+236F3]鯧[U+2AADE][U+28587][U+22EB6][U+2801F] | a52725d9-3f53-4ef2-9fe8-91181a879e8e | ---|----------------------|--------------------------------------|------------ Steps to Reproduce: 1. create utf-8-encoded organization in Satellite 5 2. spacewalk-export --org=<org id> 3. on sat6: hammer import organization --csv-file users.csv 4. hammer import config-file --csv-file config-files-latest.csv Actual results: puppet_work_dir path wrong thus the script fails Expected results: the files/dirs created by the script should not rely on non-standard characters and should successfully create the configs Additional info:
Since this issue was entered in Red Hat Bugzilla, the release flag has been set to ? to ensure that it is properly evaluated for this release.
* Puppet module-names are <username>-<classname>. * <username> is restricted to ASCII-alphanumeric ([0-9a-zA-Z]) *only* * hammer-import uses sat5-org-name as username * If org-name is entirely characters that are not ASCII-alphanumeric, we build a module-name of "-<classname>" Fix is to notice when <username> becomes empty. In this case, we create a unique name, "orgid<ID>", where <ID> is the Sat6 ID of the matching Org In testing, it was noticed that if a config-file has UTF8 characters in its path, it results in Pulp failing to accept the uploaded module. This issue is also addressed in this fix - similarly to the <username> fix, remove characters that aren't [0-9a-zA-Z_\.\-], and if the resulting string is empty, create a puppet-name of "fileid<ID>" hammer-cli-import.github: af41236fc4115a74706f460cf6fef37fdc8e827c
VERIFIED: using org name (containing only unsupported characters, so it ends up 0 length): 殣[U+212F8][U+27ECE][U+2A882][U+27F61]ᣊ[U+206B2][U+27069]㥧[U+29115] using path name (containing utf8 and alphanumeric part), so it ends up shortened (but >0 length): 烞[U+25F2E][U+261C1]픯ὃ[U+2A07D]뫄[U+28479]䎾鄉4N48ywLlha Satellite-6.1.0-RHEL-7-20150826.0 # hammer organization info --id 148 Id: 148 Name: 殣[U+212F8][U+27ECE][U+2A882][U+27F61]ᣊ[U+206B2][U+27069]㥧[U+29115] Users: # cat ~/.transition_data/organizations-mnZwfp.csv sat5,sat6,delete 1,148, # cat /tmp/tmp.fMuc1HyobG/tmpSODNmv username,symbolic_link,contents,groupname,file_type,delim_start,org_id,channel_type,channel_id,delim_end,selinux_ctx,file_id,filemode,path,is_binary,channel,revision root,,"hostname={| rhn.system.hostname |} sys_ip_address={| rhn.system.ip_address |} ip_address={| rhn.system.net_interface.ip_address(eth0) |} netmask={| rhn.system.net_interface.netmask(eth0) |} mac_address={| rhn.system.net_interface.hardware_address(eth0) |} ",root,file,{|,1,normal,3,|},,8,600,烞[U+25F2E][U+261C1]픯ὃ[U+2A07D]뫄[U+28479]䎾鄉4N48ywLlha,N,config-1,1 # hammer import config-file --csv-file /tmp/tmp.fMuc1HyobG/tmpSODNmv --generate-only --verbose Importing from /tmp/tmp.fMuc1HyobG/tmpSODNmv Writing converted files Found module orgid148-config_1 Summary Wrote 1 puppet module. Wrote 1 puppet file. # cat ~/puppet_work_dir/orgid148-config_1/templates/4N48ywLlha.erb hostname=<%= @fqdn %> sys_ip_address=<%= @ipaddress %> ip_address=<%= @ipaddress_eth0 %> netmask=<%= @netmask_eth0 %> mac_address=<%= @macaddress_eth0 %>
Sorry for bringing this up again, but is there any particular reason why we don't use the orgid<ID> format for ALL the cases? It would make the mechanism much simpler. Thanks
Because, while the customer *can* figure out what org this is, they have to really dig for it. We use org-name as the preferred entity because it's something the customer will recognize immediately. orgId<ID> is a really bad choice from a usability standpoint - it's just the best we have in the face of puppet's inability to deal w/utf8. "Simpler for us" isn't the end-goal - "simpler for the customer", is.
@ggainley: Fair enough, thanks for the answer. marking as VERIFIED
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2015:1786