Bug 1016310 - content: uploading puppet module with multibyte in name/manifest throws python error
Summary: content: uploading puppet module with multibyte in name/manifest throws pytho...
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Pulp
Classification: Retired
Component: API/integration
Version: Master
Hardware: Unspecified
OS: Unspecified
low
unspecified
Target Milestone: ---
: ---
Assignee: pulp-bugs
QA Contact: pulp-qe-list
URL:
Whiteboard:
Depends On:
Blocks: 1012614 1012628
TreeView+ depends on / blocked
 
Reported: 2013-10-07 21:06 UTC by David Davis
Modified: 2013-10-15 15:39 UTC (History)
2 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2013-10-15 15:39:19 UTC
Embargoed:


Attachments (Terms of Use)
Module I created to test this bug (15.28 KB, application/x-gzip)
2013-10-09 19:42 UTC, David Davis
no flags Details

Description David Davis 2013-10-07 21:06:17 UTC
Description of problem:

Get error when trying to upload multibyte puppet module for repo.

Version-Release number of selected component (if applicable):

2.3.0-0.16-alpha

Error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe4 in position 1: ordinal not in range(128)
2013-10-07 17:01:39,015 pulp.server.dispatch.task:ERROR: Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/dispatch/task.py", line 138, in _run
    result = call(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/pulp/server/managers/content/upload.py", line 227, in import_uploaded_unit
    importer_instance.upload_unit(transfer_repo, unit_type_id, unit_key, unit_metadata, file_path, conduit, call_config)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/importer.py", line 75, in upload_unit
    upload.handle_uploaded_unit(repo, type_id, unit_key, metadata, file_path, conduit)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/upload.py", line 52, in handle_uploaded_unit
    metadata_parser.extract_metadata(module, file_path, repo.working_dir)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/metadata.py", line 81, in extract_metadata
    metadata_json = _extract_non_standard_json(module, filename, temp_dir)
  File "/usr/lib/python2.6/site-packages/pulp_puppet/plugins/importers/metadata.py", line 132, in _extract_non_standard_json
    extraction_dir = os.path.join(temp_dir, module.author, module.name, module.version)
  File "/usr/lib64/python2.6/posixpath.py", line 70, in join
    path += '/' + b
PulpExecutionException: Pulp exception occurred: PulpExecutionException
2013-10-07 17:01:39,016 pulp.server.dispatch.task:INFO: FAILURE: Task 8f735c0e-1551-42d7-baab-b5b3d8684cbc: CallRequest: ContentUploadManager.import_uploaded_unit(u'ACME_Corporation-PForge_1381163037581-pforge', 'puppet_module', {'version': '2.0.0', 'name': 'good', 'author': '\xe4\xbd\xa0\xe5\xa5\xbd'}, {'description': 'NTP Module for Debian, Ubuntu, CentOS, RHEL, OEL, Fedora, FreeBSD, ArchLinux and Gentoo.', 'license': 'Apache Version 2.0', 'author': 'Puppet Labs', 'project_page': 'http://github.com/puppetlabs/puppetlabs-ntp', 'summary': 'NTP Module', 'source': 'git://github.com/puppetlabs/puppetlabs-ntp', 'version': '2.0.0', 'dependencies': [{'name': 'puppetlabs/stdlib', 'version_requirement': '>= 0.1.6'}], 'checksums': {'manifests/config.pp': '8d9afb6e4327277c96c5617ad687043a', 'spec/system/basic_spec.rb': 'f0fa6706b1011be24759dc06ccbd91e2', 'Gemfile': '779e8b104b94622ac8c7d1d7bcf32c5a', 'LICENSE': 'f0b6fdc310531526f257378d7bad0044', 'tests/init.pp': 'd398e7687ec1d893ef23d1b7d2afc094', 'spec/system/restrict_spec.rb': 'eb9d8deb680830f78e0041bb3c7d2326', 'manifests/params.pp': '2fe94c599d794811d950bd48879439d7', 'manifests/init.pp': '8883844f75084d31b4e5a9a50c900d87', 'spec/system/ntp_config_spec.rb': 'd60bc7ba907c080d891245d6f13da307', '.bundle/config': '7f1c988748783d2a8d455376eed1470c', 'spec/system/ntp_install_spec.rb': '9b3796bb71472daa3a52aa01b0592656', 'spec/unit/puppet/type/README.markdown': 'de26a7643813abd6c2e7e28071b1ef94', '.fixtures.yml': '909729694bab62c1e36001512b68a8fd', 'spec/spec_helper.rb': '0db89c9a486df193c0e40095422e19dc', 'spec/system/class_spec.rb': '83dcabc0e04432b52939ae8a5df992ca', 'spec/system/ntp_service_spec.rb': 'fbeec08f5489ce8ef82f2225f7107b9e', 'templates/ntp.conf.erb': '262e0a58bcd88d0cb65369081db30b32', 'CONTRIBUTING.md': '2ef1d6f4417dde9af6c7f46f5c8a864b', 'spec/spec.opts': 'a600ded995d948e393fbe2320ba8e51c', 'spec/unit/puppet/provider/README.markdown': 'e52668944ee6af2fb5d5b9e798342645', 'CHANGELOG': '1ef0e3fa661bd1b8cb8e1233343dd81c', 'README.markdown': '74a9a19e7866998e5a594845b15b0e57', 'Gemfile.lock': '1abd1da36ca6957b6d456ddd75e2a665', 'Rakefile': '0428ea3759a4692c91604396c406a9c1', 'spec/system/preferred_servers_spec.rb': '379fe9b56c10903178f03d4d87002a14', 'spec/classes/ntp_spec.rb': 'ec9345d2bb9976c012b117b8c7f8f052', 'manifests/service.pp': '350238b50e9cb896d270a2c76a64334f', 'spec/fixtures/modules/my_ntp/templates/ntp.conf.erb': '566e373728e9b13eda516115ff0a9fb0', 'Modulefile': 'c806ff020129ab4a367a502be3f1be75', '.nodeset.yml': '8d1b7762d4125ce53379966a1daf355c', 'manifests/install.pp': 'ac33c5733f4321a9af7a4735265c1986', '.travis.yml': '193ec2b14cc9644c88f4249422580da2', 'spec/spec_helper_system.rb': '4c85f6e43892c45ce75ade44a32f45f8'}, 'types': None, 'name': '\xe4\xbd\xa0\xe5\xa5\xbd-good'}, 'ce87ab49-5126-4349-909c-d3d68d2f882a')
2013-10-07 17:01:39,057 pulp.server.webservices.middleware.exception:ERROR: Pulp exception occurred: PulpExecutionException
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/middleware/exception.py", line 44, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/middleware/postponed.py", line 39, in __call__
    return self.app(environ, start_response)
  File "/usr/lib/python2.6/site-packages/web/application.py", line 279, in wsgi
    result = self.handle_with_processors()
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 26, in _handle_with_processors
    return process(self.processors)
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.6/site-packages/web/application.py", line 566, in processor
    return handler()
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>
    return p(lambda : process(processors))
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in process
    return p(lambda : process(processors))
  File "/usr/lib/python2.6/site-packages/web/application.py", line 581, in processor
    result = handler()
  File "/usr/lib/python2.6/site-packages/pulp/server/webservices/application.py", line 23, in <lambda>

Comment 1 David Davis 2013-10-09 19:42:13 UTC
Created attachment 810136 [details]
Module I created to test this bug

Comment 2 Michael Hrivnak 2013-10-09 20:36:47 UTC
According to the OP, this is low-priority. Initial investigation suggests this may be a can of worms trying to properly fix.

Comment 3 David Davis 2013-10-15 15:31:00 UTC
Per 1012628#c3, you can close this out.

Comment 4 David Davis 2013-10-15 15:33:20 UTC
See https://bugzilla.redhat.com/show_bug.cgi?id=1012614#c4 for the yum equivalent.

Comment 5 Michael Hrivnak 2013-10-15 15:39:19 UTC
Closing by request of OP

http://docs.puppetlabs.com/puppet/3/reference/modules_fundamentals.html#allowed-module-names


Note You need to log in before you can comment on or make changes to this bug.