Red Hat Satellite engineering is moving the tracking of its product development work on Satellite to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "Satellite project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs will be migrated starting at the end of May. If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "Satellite project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/SAT-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 1012628 - content: uploading puppet module with multibyte in name/manifest throws python error
Summary: content: uploading puppet module with multibyte in name/manifest throws pytho...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Content Management
Version: Nightly
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: Unspecified
Assignee: David Davis
QA Contact: Katello QA List
URL:
Whiteboard:
Depends On: 1016310
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-09-26 18:16 UTC by Corey Welton
Modified: 2013-10-15 15:29 UTC (History)
1 user (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-10-15 15:29:23 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)

Description Corey Welton 2013-09-26 18:16:41 UTC 
Description of problem:


Version-Release number of selected component (if applicable):
1.4.6-13.el6sat

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:
1.  Attempt to upload a puppet file with multibyte characters in the name.  Note: if you are trying to modify existing module for testing purposes, you'll need to edit both the filename and the name in the manifest itself.

 repo content_upload --repo=zooup --product=Zooshop --filepath=./puppetfoo/你好-good-2.0.0.tar.gz --content_type=puppet --org=ACME_Corporation
2. View results

Current results:
/usr/lib64/python2.6/tarfile.py:2350: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if name == members[i].name:
Invalid puppet module './puppetfoo/你好-good-2.0.0.tar.gz'. Please check the file and try again.
[root@qe-blade-05 ~]# kk --version


Expected results:
Either disallow multibyte filenames (if applicable) or handle error gracefully or handle properly.

Additional info:
Comment 2 David Davis 2013-10-07 21:03:03 UTC 
While we need some fixes to support this, the error you hit is incorrect. I'm guessing the author in your metadata.json wasn't 你好. I've created a unicode puppet module and it looks like we're hitting an error in pulp:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 1: ordinal not in range(128)
2013-10-07 17:01:39,015 pulp.server.dispatch.task:ERROR: Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/upload.py", line 227, in import_uploaded_unit
    importer_instance.upload_unit(transfer_repo, unit_type_id, unit_key, unit_metadata, file_path, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/importer.py", line 75, in upload_unit
    upload.handle_uploaded_unit(repo, type_id, unit_key, metadata, file_path, conduit)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/upload.py", line 52, in handle_uploaded_unit
    metadata_parser.extract_metadata(module, file_path, repo.working_dir)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/metadata.py", line 81, in extract_metadata
    metadata_json = _extract_non_standard_json(module, filename, temp_dir)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/metadata.py", line 132, in _extract_non_standard_json
    extraction_dir = os.path.join(temp_dir, module.author, module.name, module.version)
  File "/usr/lib64/python2.6/posixpath.py", line 70, in join
    path += '/' + b
PulpExecutionException: Pulp exception occurred: PulpExecutionException
2013-10-07 17:01:39,016 pulp.server.dispatch.task:INFO: FAILURE: Task 8f735c0e-1551-42d7-baab-b5b3d8684cbc: CallRequest: ContentUploadManager.import_uploaded_unit(u'ACME_Corporation-PForge_1381163037581-pforge', 'puppet_module', {'version': '2.0.0', 'name': 'good', 'author': '\xe4\xbd\xa0\xe5\xa5\xbd'}, {'description': 'NTP Module for Debian, Ubuntu, CentOS, RHEL, OEL, Fedora, FreeBSD, ArchLinux and Gentoo.', 'license': 'Apache Version 2.0', 'author': 'Puppet Labs', 'project_page': 'http://github.com/puppetlabs/puppetlabs-ntp', 'summary': 'NTP Module', 'source': 'git://github.com/puppetlabs/puppetlabs-ntp', 'version': '2.0.0', 'dependencies': [{'name': 'puppetlabs/stdlib', 'version_requirement': '>= 0.1.6'}], 'checksums': {'manifests/config.pp': '8d9afb6e4327277c96c5617ad687043a', 'spec/system/basic_spec.rb': 'f0fa6706b1011be24759dc06ccbd91e2', 'Gemfile': '779e8b104b94622ac8c7d1d7bcf32c5a', 'LICENSE': 'f0b6fdc310531526f257378d7bad0044', 'tests/init.pp': 'd398e7687ec1d893ef23d1b7d2afc094', 'spec/system/restrict_spec.rb': 'eb9d8deb680830f78e0041bb3c7d2326', 'manifests/params.pp': '2fe94c599d794811d950bd48879439d7', 'manifests/init.pp': '8883844f75084d31b4e5a9a50c900d87', 'spec/system/ntp_config_spec.rb': 'd60bc7ba907c080d891245d6f13da307', '.bundle/config': '7f1c988748783d2a8d455376eed1470c', 'spec/system/ntp_install_spec.rb': '9b3796bb71472daa3a52aa01b0592656', 'spec/unit/puppet/type/README.markdown': 'de26a7643813abd6c2e7e28071b1ef94', '.fixtures.yml': '909729694bab62c1e36001512b68a8fd', 'spec/spec_helper.rb': '0db89c9a486df193c0e40095422e19dc', 'spec/system/class_spec.rb': '83dcabc0e04432b52939ae8a5df992ca', 'spec/system/ntp_service_spec.rb': 'fbeec08f5489ce8ef82f2225f7107b9e', 'templates/ntp.conf.erb': '262e0a58bcd88d0cb65369081db30b32', 'CONTRIBUTING.md': '2ef1d6f4417dde9af6c7f46f5c8a864b', 'spec/spec.opts': 'a600ded995d948e393fbe2320ba8e51c', 'spec/unit/puppet/provider/README.markdown': 'e52668944ee6af2fb5d5b9e798342645', 'CHANGELOG': '1ef0e3fa661bd1b8cb8e1233343dd81c', 'README.markdown': '74a9a19e7866998e5a594845b15b0e57', 'Gemfile.lock': '1abd1da36ca6957b6d456ddd75e2a665', 'Rakefile': '0428ea3759a4692c91604396c406a9c1', 'spec/system/preferred_servers_spec.rb': '379fe9b56c10903178f03d4d87002a14', 'spec/classes/ntp_spec.rb': 'ec9345d2bb9976c012b117b8c7f8f052', 'manifests/service.pp': '350238b50e9cb896d270a2c76a64334f', 'spec/fixtures/modules/my_ntp/templates/ntp.conf.erb': '566e373728e9b13eda516115ff0a9fb0', 'Modulefile': 'c806ff020129ab4a367a502be3f1be75', '.nodeset.yml': '8d1b7762d4125ce53379966a1daf355c', 'manifests/install.pp': 'ac33c5733f4321a9af7a4735265c1986', '.travis.yml': '193ec2b14cc9644c88f4249422580da2', 'spec/spec_helper_system.rb': '4c85f6e43892c45ce75ade44a32f45f8'}, 'types': None, 'name': '\xe4\xbd\xa0\xe5\xa5\xbd-good'}, 'ce87ab49-5126-4349-909c-d3d68d2f882a')
2013-10-07 17:01:39,057 pulp.server.webservices.middleware.exception:ERROR: Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/middleware/exception.py", line 44, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/middleware/postponed.py", line 39, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/web/application.py", line 279, in wsgi
    result = self.handle_with_processors()
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 26, in _handle_with_processors
    return process(self.processors)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.6/site-packages/web/application.py", line 566, in processor
    return handler()
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>
    return p(lambda : process(processors))
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.6/site-packages/web/application.py", line 581, in processor
    result = handler()
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>

I tried a bunch of different formats (unicode, utf, etc) but I hit that error.
Comment 3 David Davis 2013-10-15 15:29:23 UTC 
I found this thanks to #puppet:

http://docs.puppetlabs.com/puppet/3/reference/modules_fundamentals.html#allowed-module-names

Closing as NOTABUG per Mike McCune.
Note You need to log in before you can comment on or make changes to this bug.