Bug 700119 - satellite-sync: UnicodeDecodeError('ascii', 'P\xc3\xa1\xc2...', 1, 2, 'ordinal not in range(128)')
Summary: satellite-sync: UnicodeDecodeError('ascii', 'P\xc3\xa1\xc2...', 1, 2, 'ordina...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 5
Classification: Red Hat
Component: Satellite Synchronization
Version: 541
Hardware: Unspecified
OS: Unspecified
medium
high
Target Milestone: ---
Assignee: Miroslav Suchý
QA Contact: Jan Pazdziora (Red Hat)
URL:
Whiteboard:
Depends On: 696970
Blocks: sat541-blockers
TreeView+ depends on / blocked
 
Reported: 2011-04-27 14:03 UTC by Michael Mráka
Modified: 2014-05-19 06:57 UTC (History)
5 users (show)

Fixed In Version: spacewalk-backend-1.2.13-47
Doc Type: Bug Fix
Doc Text:
Clone Of: 696970
Environment:
Last Closed: 2011-06-17 02:42:51 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Michael Mráka 2011-04-27 14:03:58 UTC
+++ This bug was initially created as a clone of Bug #696970 +++

Description of problem:
Running satellite-sync from disk export with non-default locale fails with thraceback.

Version-Release number of selected component (if applicable):
spacewalk-backend-1.5.3-1.el6

How reproducible:
always

Steps to Reproduce:
1. export LC_CTYPE=cs_CZ.UTF-8
2. export LC_TIME=cs_CZ.UTF-8
3. rhn-satellite-exporter -d exp -c rhel-i386-server-6 --no-rpms --no-packages --no-kickstarts --print-report
4. satellite-sync -l -m rhel6/
  
Actual results:
08:41:09 Red Hat Network Satellite - file-system synchronization
08:41:09    mp:  /tmp/rhel6
08:41:09    db:  spaceuser/<password>@dbname=spaceschema;host=localhost;port=5432
08:41:09 
08:41:09 Retrieving / parsing channel-families data
08:41:09 channel-families data complete
08:41:09 
08:41:09 Retrieving / parsing product names data
08:41:09 
08:41:09 Retrieving / parsing channel data
08:41:09    p = previously imported/synced channel
08:41:09    . = channel not yet imported/synced
08:41:09    base-channels:
+++ sending log as an email +++

SYNC ERROR: unhandled exception occurred:

(Check logs/email for potentially more detail)

UnicodeDecodeError('ascii', 'P\xc3\xa1\xc2\xa015.\xc2\xa0duben\xc2\xa02011,\xc2\xa005:12:40\xc2\xa0', 1, 2, 'ordinal not in range(128)')
'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)


Expected results:
list of channels as usual

Additional info:
/var/log/rhn/rhn_server_satellite.log:
Traceback (most recent call last):
  File "/usr/bin/satellite-sync", line 139, in main
    return satsync.Runner().main()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 227, in main
    ret = method()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 302, in _step_channels
    self.syncer.process_channels()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 667, in process_channels
    self._printChannelTree(doEOSYN=doEOSYN)
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 755, in _printChannelTree
    self._printChannel(plabel, channel_object, p_format, (plabel in ch_requested_imported))
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 726, in _printChannel
    log(1, format % (status, label, pkgs_count, self._formatChannelExportType(channel_object)))
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 712, in _formatChannelExportType
    return _("%10s import from %s") % (export_type, self._formatDateTime(end_date))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

Comment 1 Michael Mráka 2011-04-27 14:07:41 UTC
Affected package spacewalk-backend-1.2.13-46.el6sat.noarch.

Steps to reproduce
1. create a custom channel
2. push a package there
3. export LC_CTYPE=cs_CZ.UTF-8
4. export LC_TIME=cs_CZ.UTF-8
5. rhn-satellite-exporter -c _the_channel_ -d /tmp/exp
6. satellite-sync -m /tmp/exp -l

Comment 2 Michael Mráka 2011-04-29 10:11:43 UTC
Backported changes to SATELLITE-5.4 as
commit 3522b7a313c233fe7bba2bdd6bff18fd8cac0bb6
    696970 - disabled localization
    backend logging subclasses aren't prepared for full localization yet
commit 66f05d0280053af05e08fff764b2767022445172
    696970 - unified time printing via formatDateTime()
commit 560345a08e74c5d655e93c4b1d6f543b42d804da
    696970 - %%c is the preferred date/time representation for the current locale

Comment 4 Jan Pazdziora (Red Hat) 2011-05-16 15:00:06 UTC
Taking.

Comment 5 Jan Pazdziora (Red Hat) 2011-05-16 15:02:18 UTC
With spacewalk-backend-1.2.13-46.el6sat.noarch, satellite-sync fails:

10:59:29 Retrieving / parsing channel data
10:59:29    p = previously imported/synced channel
10:59:29    . = channel not yet imported/synced
10:59:29    base-channels:
+++ sending log as an email +++

SYNC ERROR: unhandled exception occurred:

(Check logs/email for potentially more detail)

UnicodeDecodeError('ascii', 'Po\xc2\xa016.\xc2\xa0kv\xc4\x9bten\xc2\xa02011,\xc2\xa010:59:13\xc2\xa0', 2, 3, 'ordinal not in range(128)')
'ascii' codec can't decode byte 0xc2 in position 2: ordinal not in range(128)

When doing rpm -Fvh Satellite/* to get spacewalk-backend-1.2.13-48.el6sat.noarch packages, satellite-sync passes again:

# satellite-sync -m . -l
10:59:40 Red Hat Network Satellite - file-system synchronization
10:59:40    mp:  /root/dump
10:59:40    db:  rhnsat/<password>@rhnsat
10:59:40 
10:59:40 Retrieving / parsing channel-families data
10:59:40 channel-families data complete
10:59:40 
10:59:40 Retrieving / parsing channel data
10:59:40    p = previously imported/synced channel
10:59:40    . = channel not yet imported/synced
10:59:40    base-channels:
10:59:40       . test-1                                      1       full import from Mon May 16 10:59:13 2011
10:59:40 
    Import complete:
        Begin time: Mon May 16 10:59:39 2011
        End time:   Mon May 16 10:59:40 2011
        Elapsed:    0 hours, 0 minutes, 0 seconds

When I downgrade back to -46 with rpm -Uvh --oldpackage, it fails again.

I was not able to reproduce the error on RHEL 5 but comment 1 indicates this is only RHEL 6 issue.

As such, marking VERIFIED.

Comment 8 Clifford Perry 2011-06-17 02:42:51 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

https://rhn.redhat.com/errata/RHEA-2011-0875.html

Comment 9 Magnus Glantz 2014-05-16 22:42:04 UTC
I hit this issue on RHN Satellite 5.5.

SYNC ERROR: unhandled exception occurred:

Exception reported from rhn-satellite.our.domain
Time: Fri May 16 10:31:22 2014
Exception type <type 'exceptions.UnicodeDecodeError'>

Exception Handler Information
Traceback (most recent call last):
 File "/usr/bin/satellite-sync", line 139, in main
   return satsync.Runner().main()
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 225, in main
   ret = method()
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 320, in _step_download_packages
   return self.syncer.download_package_metadata()
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1097, in download_package_metadata
   stream_loader.process, is_slow=True)
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1572, in _proces_batch
   prompt, nevermorethan, process_function_args)
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1552, in _processWithProgressBar
   process_function(chunk, *process_function_args)
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1996, in process
   self.handler.process(stream)
 File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/xmlSource.py", line 135, in process
   Traceback(ostream=sys.stderr, with_locals=1)
 File "/usr/lib/python2.6/site-packages/spacewalk/common/rhnTB.py", line 174, in Traceback
   outstring = exc.getvalue()
 File "/usr/lib64/python2.6/StringIO.py", line 270, in getvalue
   self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 29125: ordinal not in range(128)


# On the RHN Sat that your want to export channel data from
1. rhn-satellite-exporter -c channel-with-issues --no-rpms -d /path/to/destination
2. cd /path/to
3. tar cvzf destination.tar.gz destination
4. scp destination.tar.gz user:/import
5. ssh user

# On the RHN Sat that you want to import the data to
6. cd /import
(
7. Repeating below until no error messages 
8. satellite-sync -m . -c channel-with-issues
9. Notice that there was issue with /var/satellite/import/base/packages/xy/rhn-package-xyz.xml.gz
10. cd /var/satellite/import/base/packages/xy
11. gunzip rhn-package-xyz.xml.gz
12. vi rhn-package-xyz.xml.gz # fixing invalid unicode characters..
13. gzip rhn-package-xyz.xml
)
14. satellite-sync --iss-parent=rhnsat.that.i.exported.from.domain --ca-cert=/root/rhnsat.that.i.export.from.domain.cert -c channel-with-issues
15. Done

Comment 10 Miroslav Suchý 2014-05-19 06:54:57 UTC
I suggest you to file new BZ - or even better, to contact tech. support. Commenting on closed BZ will have likely zero impact.

Comment 11 Magnus Glantz 2014-05-19 06:57:20 UTC
Sorry, didn't see it was closed. I'll open up a new BZ.


Note You need to log in before you can comment on or make changes to this bug.