Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.
Bug 1226981 - hammer import config-file fails with utf-8 encoded organization name
hammer import config-file fails with utf-8 encoded organization name
Status: CLOSED ERRATA
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Transitions (Show other bugs)
6.1.0
Unspecified Unspecified
unspecified Severity high (vote)
: 6.1.2
: Unused
Assigned To: Grant Gainey
Roman Plevka
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-06-01 11:12 EDT by Roman Plevka
Modified: 2017-02-23 14:58 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-09-15 03:22:04 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:1786 normal SHIPPED_LIVE Satellite 6.1.2 bug fix update 2015-09-15 07:20:04 EDT

  None (edit)
Description Roman Plevka 2015-06-01 11:12:13 EDT
Description of problem:
while trying to use hammer import config-file with csv file referring to an organization which name is UTF-8 encoded, the script fails to locate the puppet_work_dir for this org, as the directory name should consist of <org name>-config_1, but there's only '-config_1' directory on the filesystem.


Version-Release number of selected component (if applicable):
# hammer --version
hammer (0.1.4.11)
 * hammer_cli_foreman (0.1.4.12)
 * hammer_cli_foreman_bootdisk (0.1.2.7)
 * hammer_cli_foreman_discovery (0.0.1.10)
 * hammer_cli_foreman_docker (unknown version)
 * hammer_cli_gutterball (unknown version)
 * hammer_cli_import (0.10.15)
 * hammer_cli_katello (0.0.7.13)

How reproducible:
export (sat5) and import (sat6) organization with name like:
﷐[U+217B1]﷑﷐[U+22AC8]﷑﷐[U+285C9]﷑﷐[U+2931A]﷑﷐[U+236F3]﷑鯧﷐[U+2AADE]﷑﷐[U+28587]﷑﷐[U+22EB6]﷑﷐[U+2801F]﷑
and try to import config-file related to it using hammer import config-file.
The script that is writing the puppet files will fail:

# hammer import config-file --csv-file BM8sxwbVGZ.csv --generate-only
Caught EOFError:end of file reached while processing CSV line:
{"org_id"=>"4", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/sysconfig/rhn/systemid", "file_type"=>"file", "file_id"=>"8", "revision"=>"1", "is_binary"=>"N", "contents"=>"hostname={| rhn.system.hostname |}\nsys_ip_address={| rhn.system.ip_address |}\nip_address={| rhn.system.net_interface.ip_address(eth0) |}\nnetmask={| rhn.system.net_interface.netmask(eth0) |}\nmac_address={| rhn.system.net_interface.hardware_address(eth0) |}\n", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"root", "groupname"=>"root", "filemode"=>"600", "symbolic_link"=>nil, "selinux_ctx"=>nil}
Writing converted files
Caught Errno::ENOENT:No such file or directory - /root/puppet_work_dir/-config_1/files while post_import
Summary
  No action taken.

# hammer organization list
---|----------------------|--------------------------------------|------------
ID | NAME                 | LABEL                                | DESCRIPTION
---|----------------------|--------------------------------------|------------
1  | Default Organization | Default_Organization                 |            
49 | ﷐[U+217B1]﷑﷐[U+22AC8]﷑﷐[U+285C9]﷑﷐[U+2931A]﷑﷐[U+236F3]﷑鯧﷐[U+2AADE]﷑﷐[U+28587]﷑﷐[U+22EB6]﷑﷐[U+2801F]﷑ | a52725d9-3f53-4ef2-9fe8-91181a879e8e |            
---|----------------------|--------------------------------------|------------


Steps to Reproduce:
1. create utf-8-encoded organization in Satellite 5
2. spacewalk-export --org=<org id>
3. on sat6: hammer import organization --csv-file users.csv
4. hammer import config-file --csv-file config-files-latest.csv

Actual results:
puppet_work_dir path wrong thus the script fails

Expected results:
the files/dirs created by the script should not rely on non-standard characters and should successfully create the configs

Additional info:
Comment 1 RHEL Product and Program Management 2015-06-01 11:12:34 EDT
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.
Comment 4 Grant Gainey 2015-08-18 11:36:24 EDT
* Puppet module-names are <username>-<classname>.
* <username> is restricted to ASCII-alphanumeric ([0-9a-zA-Z]) *only*
* hammer-import uses sat5-org-name as username
* If org-name is entirely characters that are not ASCII-alphanumeric, we build a module-name of "-<classname>"

Fix is to notice when <username> becomes empty. In this case, we create a unique name, "orgid<ID>", where <ID> is the Sat6 ID of the matching Org

In testing, it was noticed that if a config-file has UTF8 characters in its path, it results in Pulp failing to accept the uploaded module. This issue is also addressed in this fix - similarly to the <username> fix, remove characters that aren't [0-9a-zA-Z_\.\-], and if the resulting string is empty, create a puppet-name of "fileid<ID>"

hammer-cli-import.github: af41236fc4115a74706f460cf6fef37fdc8e827c
Comment 6 Roman Plevka 2015-08-31 10:41:32 EDT
VERIFIED:
using org name (containing only unsupported characters, so it ends up 0 length): 殣﷐[U+212F8]﷑﷐[U+27ECE]﷑﷐[U+2A882]﷑﷐[U+27F61]﷑ᣊ﷐[U+206B2]﷑﷐[U+27069]﷑㥧﷐[U+29115]﷑
using path name (containing utf8 and alphanumeric part), so it ends up shortened (but >0 length): 烞﷐[U+25F2E]﷑﷐[U+261C1]﷑픯ὃ﷐[U+2A07D]﷑뫄﷐[U+28479]﷑䎾鄉4N48ywLlha

Satellite-6.1.0-RHEL-7-20150826.0

# hammer organization info --id 148
Id:                     148
Name:                   殣﷐[U+212F8]﷑﷐[U+27ECE]﷑﷐[U+2A882]﷑﷐[U+27F61]﷑ᣊ﷐[U+206B2]﷑﷐[U+27069]﷑㥧﷐[U+29115]﷑
Users:                  

# cat ~/.transition_data/organizations-mnZwfp.csv 
sat5,sat6,delete
1,148,


# cat /tmp/tmp.fMuc1HyobG/tmpSODNmv
username,symbolic_link,contents,groupname,file_type,delim_start,org_id,channel_type,channel_id,delim_end,selinux_ctx,file_id,filemode,path,is_binary,channel,revision
root,,"hostname={| rhn.system.hostname |}
sys_ip_address={| rhn.system.ip_address |}
ip_address={| rhn.system.net_interface.ip_address(eth0) |}
netmask={| rhn.system.net_interface.netmask(eth0) |}
mac_address={| rhn.system.net_interface.hardware_address(eth0) |}
",root,file,{|,1,normal,3,|},,8,600,烞﷐[U+25F2E]﷑﷐[U+261C1]﷑픯ὃ﷐[U+2A07D]﷑뫄﷐[U+28479]﷑䎾鄉4N48ywLlha,N,config-1,1


# hammer import config-file --csv-file /tmp/tmp.fMuc1HyobG/tmpSODNmv --generate-only --verbose
Importing from /tmp/tmp.fMuc1HyobG/tmpSODNmv
Writing converted files
Found module orgid148-config_1
Summary
  Wrote 1 puppet module.
  Wrote 1 puppet file.



# cat ~/puppet_work_dir/orgid148-config_1/templates/4N48ywLlha.erb
hostname=<%= @fqdn %>
sys_ip_address=<%= @ipaddress %>
ip_address=<%= @ipaddress_eth0 %>
netmask=<%= @netmask_eth0 %>
mac_address=<%= @macaddress_eth0 %>
Comment 7 Roman Plevka 2015-09-07 08:19:14 EDT
Sorry for bringing this up again, but is there any particular reason why we don't use the orgid<ID> format for ALL the cases? It would make the mechanism much simpler.
Thanks
Comment 8 Grant Gainey 2015-09-07 08:40:35 EDT
Because, while the customer *can* figure out what org this is, they have to really dig for it. We use org-name as the preferred entity because it's something the customer will recognize immediately.

orgId<ID> is a really bad choice from a usability standpoint - it's just the best we have in the face of puppet's inability to deal w/utf8.

"Simpler for us" isn't the end-goal - "simpler for the customer", is.
Comment 9 Roman Plevka 2015-09-07 08:57:09 EDT
@ggainley: Fair enough, thanks for the answer.

marking as VERIFIED
Comment 11 errata-xmlrpc 2015-09-15 03:22:04 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1786

Note You need to log in before you can comment on or make changes to this bug.