Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1226981

Summary: hammer import config-file fails with utf-8 encoded organization name
Product: Red Hat Satellite Reporter: Roman Plevka <rplevka>
Component: TransitionsAssignee: Grant Gainey <ggainey>
Status: CLOSED ERRATA QA Contact: Roman Plevka <rplevka>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: bkearney, cwelton, ggainey, mmccune
Target Milestone: Unspecified   
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-09-15 07:22:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Roman Plevka 2015-06-01 15:12:13 UTC
Description of problem:
while trying to use hammer import config-file with csv file referring to an organization which name is UTF-8 encoded, the script fails to locate the puppet_work_dir for this org, as the directory name should consist of <org name>-config_1, but there's only '-config_1' directory on the filesystem.


Version-Release number of selected component (if applicable):
# hammer --version
hammer (0.1.4.11)
 * hammer_cli_foreman (0.1.4.12)
 * hammer_cli_foreman_bootdisk (0.1.2.7)
 * hammer_cli_foreman_discovery (0.0.1.10)
 * hammer_cli_foreman_docker (unknown version)
 * hammer_cli_gutterball (unknown version)
 * hammer_cli_import (0.10.15)
 * hammer_cli_katello (0.0.7.13)

How reproducible:
export (sat5) and import (sat6) organization with name like:
﷐[U+217B1]﷑﷐[U+22AC8]﷑﷐[U+285C9]﷑﷐[U+2931A]﷑﷐[U+236F3]﷑鯧﷐[U+2AADE]﷑﷐[U+28587]﷑﷐[U+22EB6]﷑﷐[U+2801F]﷑
and try to import config-file related to it using hammer import config-file.
The script that is writing the puppet files will fail:

# hammer import config-file --csv-file BM8sxwbVGZ.csv --generate-only
Caught EOFError:end of file reached while processing CSV line:
{"org_id"=>"4", "channel_id"=>"3", "channel"=>"config-1", "channel_type"=>"normal", "path"=>"/etc/sysconfig/rhn/systemid", "file_type"=>"file", "file_id"=>"8", "revision"=>"1", "is_binary"=>"N", "contents"=>"hostname={| rhn.system.hostname |}\nsys_ip_address={| rhn.system.ip_address |}\nip_address={| rhn.system.net_interface.ip_address(eth0) |}\nnetmask={| rhn.system.net_interface.netmask(eth0) |}\nmac_address={| rhn.system.net_interface.hardware_address(eth0) |}\n", "delim_start"=>"{|", "delim_end"=>"|}", "username"=>"root", "groupname"=>"root", "filemode"=>"600", "symbolic_link"=>nil, "selinux_ctx"=>nil}
Writing converted files
Caught Errno::ENOENT:No such file or directory - /root/puppet_work_dir/-config_1/files while post_import
Summary
  No action taken.

# hammer organization list
---|----------------------|--------------------------------------|------------
ID | NAME                 | LABEL                                | DESCRIPTION
---|----------------------|--------------------------------------|------------
1  | Default Organization | Default_Organization                 |            
49 | ﷐[U+217B1]﷑﷐[U+22AC8]﷑﷐[U+285C9]﷑﷐[U+2931A]﷑﷐[U+236F3]﷑鯧﷐[U+2AADE]﷑﷐[U+28587]﷑﷐[U+22EB6]﷑﷐[U+2801F]﷑ | a52725d9-3f53-4ef2-9fe8-91181a879e8e |            
---|----------------------|--------------------------------------|------------


Steps to Reproduce:
1. create utf-8-encoded organization in Satellite 5
2. spacewalk-export --org=<org id>
3. on sat6: hammer import organization --csv-file users.csv
4. hammer import config-file --csv-file config-files-latest.csv

Actual results:
puppet_work_dir path wrong thus the script fails

Expected results:
the files/dirs created by the script should not rely on non-standard characters and should successfully create the configs

Additional info:

Comment 1 RHEL Program Management 2015-06-01 15:12:34 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 4 Grant Gainey 2015-08-18 15:36:24 UTC
* Puppet module-names are <username>-<classname>.
* <username> is restricted to ASCII-alphanumeric ([0-9a-zA-Z]) *only*
* hammer-import uses sat5-org-name as username
* If org-name is entirely characters that are not ASCII-alphanumeric, we build a module-name of "-<classname>"

Fix is to notice when <username> becomes empty. In this case, we create a unique name, "orgid<ID>", where <ID> is the Sat6 ID of the matching Org

In testing, it was noticed that if a config-file has UTF8 characters in its path, it results in Pulp failing to accept the uploaded module. This issue is also addressed in this fix - similarly to the <username> fix, remove characters that aren't [0-9a-zA-Z_\.\-], and if the resulting string is empty, create a puppet-name of "fileid<ID>"

hammer-cli-import.github: af41236fc4115a74706f460cf6fef37fdc8e827c

Comment 6 Roman Plevka 2015-08-31 14:41:32 UTC
VERIFIED:
using org name (containing only unsupported characters, so it ends up 0 length): 殣﷐[U+212F8]﷑﷐[U+27ECE]﷑﷐[U+2A882]﷑﷐[U+27F61]﷑ᣊ﷐[U+206B2]﷑﷐[U+27069]﷑㥧﷐[U+29115]﷑
using path name (containing utf8 and alphanumeric part), so it ends up shortened (but >0 length): 烞﷐[U+25F2E]﷑﷐[U+261C1]﷑픯ὃ﷐[U+2A07D]﷑뫄﷐[U+28479]﷑䎾鄉4N48ywLlha

Satellite-6.1.0-RHEL-7-20150826.0

# hammer organization info --id 148
Id:                     148
Name:                   殣﷐[U+212F8]﷑﷐[U+27ECE]﷑﷐[U+2A882]﷑﷐[U+27F61]﷑ᣊ﷐[U+206B2]﷑﷐[U+27069]﷑㥧﷐[U+29115]﷑
Users:                  

# cat ~/.transition_data/organizations-mnZwfp.csv 
sat5,sat6,delete
1,148,


# cat /tmp/tmp.fMuc1HyobG/tmpSODNmv
username,symbolic_link,contents,groupname,file_type,delim_start,org_id,channel_type,channel_id,delim_end,selinux_ctx,file_id,filemode,path,is_binary,channel,revision
root,,"hostname={| rhn.system.hostname |}
sys_ip_address={| rhn.system.ip_address |}
ip_address={| rhn.system.net_interface.ip_address(eth0) |}
netmask={| rhn.system.net_interface.netmask(eth0) |}
mac_address={| rhn.system.net_interface.hardware_address(eth0) |}
",root,file,{|,1,normal,3,|},,8,600,烞﷐[U+25F2E]﷑﷐[U+261C1]﷑픯ὃ﷐[U+2A07D]﷑뫄﷐[U+28479]﷑䎾鄉4N48ywLlha,N,config-1,1


# hammer import config-file --csv-file /tmp/tmp.fMuc1HyobG/tmpSODNmv --generate-only --verbose
Importing from /tmp/tmp.fMuc1HyobG/tmpSODNmv
Writing converted files
Found module orgid148-config_1
Summary
  Wrote 1 puppet module.
  Wrote 1 puppet file.



# cat ~/puppet_work_dir/orgid148-config_1/templates/4N48ywLlha.erb
hostname=<%= @fqdn %>
sys_ip_address=<%= @ipaddress %>
ip_address=<%= @ipaddress_eth0 %>
netmask=<%= @netmask_eth0 %>
mac_address=<%= @macaddress_eth0 %>

Comment 7 Roman Plevka 2015-09-07 12:19:14 UTC
Sorry for bringing this up again, but is there any particular reason why we don't use the orgid<ID> format for ALL the cases? It would make the mechanism much simpler.
Thanks

Comment 8 Grant Gainey 2015-09-07 12:40:35 UTC
Because, while the customer *can* figure out what org this is, they have to really dig for it. We use org-name as the preferred entity because it's something the customer will recognize immediately.

orgId<ID> is a really bad choice from a usability standpoint - it's just the best we have in the face of puppet's inability to deal w/utf8.

"Simpler for us" isn't the end-goal - "simpler for the customer", is.

Comment 9 Roman Plevka 2015-09-07 12:57:09 UTC
@ggainley: Fair enough, thanks for the answer.

marking as VERIFIED

Comment 11 errata-xmlrpc 2015-09-15 07:22:04 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1786