Bug 1269145

Summary: Syncing some repos results in UnicodeEncodeError: 'ascii' codec can't encode character
Product: Red Hat Satellite Reporter: Justin Sherrill <jsherril>
Component: RepositoriesAssignee: Katello Bug Bin <katello-bugs>
Status: CLOSED ERRATA QA Contact: Kedar Bidarkar <kbidarka>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.1.0CC: bbuckingham, bkearney, bmbouter, cwelton, daviddavis, dkliban, ggainey, ipanova, kbidarka, mhrivnak, pcreech, rchan, ttereshc
Target Milestone: UnspecifiedKeywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-07-27 11:31:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Justin Sherrill 2015-10-06 12:58:38 UTC
Description of problem:

Cloned from https://pulp.plan.io/issues/1264

Syncing some repos (such as F21 and F22 updates repo) results in an error and traceback.


Version-Release number of selected component (if applicable):


How reproducible:
Always according to user

Steps to Reproduce:
1. create a repo pointed to http://dl.fedoraproject.org/pub/fedora/linux/updates/22/x86_64/
2. Sync the repo

Actual results:
Error with traceback below


Expected results:
Should sync successfully. 

Additional info:


Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 148, in run
        self.update_content(metadata_files)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 324, in update_content
        rpms_to_download, drpms_to_download = self._decide_what_to_download(metadata_files)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 342, in decide_what_to_download
        rpms_to_download, rpms_count, rpms_total_size = self._decide_rpms_to_download(metadata_files)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 374, in _decide_rpms_to_download
        wanted = self._identify_wanted_versions(package_info_generator)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/sync.py", line 687, in _identify_wanted_versions
        for model in package_info_generator:
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/packages.py", line 63, in package_list_generator
        package_info = process_func(element)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/repomd/primary.py", line 180, in process_package_element
        model.raw_xml = utils.element_to_raw_xml(package_element, [rpm_namespace], COMMON_SPEC_URL)
File "/usr/lib/python2.6/site-packages/pulp_rpm/plugins/importers/yum/utils.py", line 55, in element_to_raw_xml
        tree.write(io)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 663, in write
        self._write(file, self._root, encoding, {})
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 707, in _write
        self._write(file, n, encoding, namespaces)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 705, in _write
        file.write(_escape_cdata(node.text, encoding))
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 807, in _escape_cdata
        return _encode_entity(text)
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 792, in _encode_entity
        return _encode(pattern.sub(escape_entities, text), "ascii")
File "/usr/lib64/python2.6/xml/etree/ElementTree.py", line 751, in _encode 
        return s.encode(encoding)
UnicodeEncodeError: 'ascii' codec can't encode character u'\U000ff004' in position 145: ordinal not in range(128)

Comment 2 pulp-infra@redhat.com 2015-10-06 13:30:21 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2015-10-06 13:30:22 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2015-10-06 14:00:23 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 5 Michael Hrivnak 2015-10-09 14:29:40 UTC
We discovered an additional problem that is related, and I am adding the tracker for that new redmine issue.

Comment 6 pulp-infra@redhat.com 2015-10-09 15:00:16 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2015-10-09 15:00:18 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 8 pulp-infra@redhat.com 2015-10-20 15:00:16 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 9 pulp-infra@redhat.com 2015-10-22 14:30:22 UTC
The Pulp upstream bug status is at CLOSED - NOTABUG. Updating the external tracker on this bug.

Comment 10 pulp-infra@redhat.com 2015-10-22 15:00:33 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 11 pulp-infra@redhat.com 2015-11-09 21:00:31 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 13 Bryan Kearney 2016-02-15 18:30:23 UTC
Pulp 2.7 appears to fix this. Moving this to POST.

Comment 15 Kedar Bidarkar 2016-04-13 11:59:01 UTC
Syncing of the fedora 22 url was successful without any error as mentioned in the bug description.

2016-04-13 07:04:58 [app] [I] Processing by Katello::Api::V2::RepositoriesController#sync_complete as */*
2016-04-13 07:04:58 [app] [I]   Parameters: {"call_report"=>"[FILTERED]", "event_type"=>"repo.sync.finish", "payload"=>{"importer_id"=>"yum_importer", "exception"=>nil, "repo_id"=>"Default_Organization-fedora_prd-fedora", "traceback"=>nil, "started"=>"2016-04-13T09:45:25Z", "_ns"=>"repo_sync_results", "completed"=>"2016-04-13T11:04:58Z", "importer_type_id"=>"yum_importer", "error_message"=>nil, "summary"=>{"content"=>{"state"=>"FINISHED"}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"state"=>"FINISHED"}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "added_count"=>40075, "result"=>"success", "updated_count"=>0, "details"=>{"content"=>{"size_total"=>31885621008, "items_left"=>0, "items_total"=>36325, "state"=>"FINISHED", "size_left"=>0, "details"=>{"rpm_total"=>18919, "rpm_done"=>18919, "drpm_total"=>17406, "drpm_done"=>17406}, "error_details"=>nil}, "comps"=>{"state"=>"FINISHED"}, "purge_duplicates"=>{"state"=>"FINISHED"}, "distribution"=>{"items_total"=>0, "state"=>"FINISHED", "error_details"=>nil, "items_left"=>0}, "errata"=>{"state"=>"FINISHED"}, "metadata"=>{"state"=>"FINISHED"}}, "id"=>"570e27da952da6d3130ec07b", "removed_count"=>0}, "token"=>"ZhGSbRpxPy7Q6QvHbmbFY6iBysgySKGm", "api_version"=>"v2", "repository"=>{}}
2016-04-13 07:04:59 [app] [I] Sync_complete called for fedora, running after_sync.
2016-04-13 07:04:59 [app] [I] Completed 200 OK in 97ms (Views: 1.0ms | ActiveRecord: 4.5ms)
2016-04-13 07:04:59 [app] [I] Started GET "/katello/sync_management/sync_status?repoids%5B%5D=36&_=1460540702900" for 10.10.50.140 at 2016-04-13 07:04:59 -0400
2016-04-13 07:04:59 [app] [I] Processing by Katello::SyncManagementController#sync_status as JSON
2016-04-13 07:04:59 [app] [I]   Parameters: {"repoids"=>["36"], "_"=>"1460540702900"}
2016-04-13 07:04:59 [app] [I] Completed 200 OK in 148ms (Views: 0.7ms | ActiveRecord: 14.8ms)


VERIFIED With sat62-snap7.1

Comment 16 Bryan Kearney 2016-07-27 11:31:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:1501