Bug 1299862

Summary: satellite-sync from base channel dump RHEL7 x86_64 2015-03-08 is failing with DataError: invalid byte sequence for encoding "UTF8": 0xc220
Product: Red Hat Satellite 5 Reporter: Jan Hutař <jhutar>
Component: Satellite SynchronizationAssignee: Tomáš Kašpárek <tkasparek>
Status: CLOSED DUPLICATE QA Contact: Red Hat Satellite QA List <satqe-list>
Severity: high Docs Contact:
Priority: high    
Version: 570CC: dyordano, jdobes, jiri.blaha, mgazdik, tlestach
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-04-13 13:27:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jan Hutař 2016-01-19 12:36:25 UTC
Description of problem:
`satellite-sync -c rhel-x86_64-server-7` from base channel dump RHEL7 x86_64 2015-03-08 is failing with DataError: invalid byte sequence for encoding "UTF8": 0xc220


Version-Release number of selected component (if applicable):
satellite-schema-5.7.0.20-1.el6sat.noarch
spacewalk-backend-tools-2.3.3-33.el6sat.noarch


How reproducible:
always


Steps to Reproduce:
1. Download and extract dump "RHEL 7 (x86_64) + EUS + RHN Tools + Optional
   (Base 2015-03-08)" from:
   https://rhn.redhat.com/rhn/software/channel/downloads/Download.do?cid=24541
2. # satellite-sync -m /where/is/dump -c rhel-x86_64-server-7


Actual results:
DataError: invalid byte sequence for encoding "UTF8": 0xc220

When I have added some debug statements into the code, I have discovered this
is the problematic package:

  Red_Hat_Enterprise_Linux-Release_Notes-7-kn-IN-1.0-9.9.el7

This is the problematic query and its parameters:

>>> select id from rhnPackageChangeLogData where name = %(name)s and time = %(time)s and text = %(text)s
>>> {'text': '- Red Hat Enterprise Linux 7.1 \xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xaa\xc3\x83\xc2\xa0\xc3\x82\xc2\xad\xc3\x82\xc2\x8d\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb0\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\x95\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xbe\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb6\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xa8 \xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\x9f\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xbf\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xaa\xc3\x83\xc2\xa0\xc3\x82\xc2\xad\xc3\x82\xc2\x8d\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xaa\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xa3\xc3\x83\xc2\xa0\xc3\x82\xc2\xad\xc3\x82\xc2\x80\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb0 \xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xaa\xc3\x83\xc2\xa0\xc3\x82\xc2\xad\xc3\x82\xc2\x8d\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb0\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\x95\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xbe\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb6\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xa8\xc3\x83\xc2\xa0\xc3\x82\xc2\xa5\xc3\x82\xc2\xa4', 'name': '\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xae\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xbf\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb2\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xbe\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xa8 \xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xa8\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xad\xc3\x83\xc2\xa0\xc3\x82\xc2\xad\xc3\x82\xc2\x8d\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xb0\xc3\x83\xc2\xa0\xc3\x82\xc2\xac\xc3\x82\xc2\xa4\xc3\x83\xc2\xa0\xc3\x82\xc2', 'time': '2015-01-14 18:00:00'}


Expected results:
Should work same way as it works when syncing for RHN (when on connected Satellite)


Additional info:
SYNC ERROR: unhandled exception occurred:

Exception reported from rhndev1.s390.bos.redhat.com
Time: Tue Jan 19 05:25:45 2016
Exception type <class 'psycopg2.DataError'>

Exception Handler Information
Traceback (most recent call last):
  File "/usr/bin/satellite-sync", line 139, in main
    return satsync.Runner().main()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 229, in main
    ret = method()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 344, in _step_packages
    self._affected_channels = self.syncer.import_packages()
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1613, in import_packages
    [sources])
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1587, in _proces_batch
    prompt, nevermorethan, process_function_args)
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1567, in _processWithProgressBar
    process_function(chunk, *process_function_args)
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/satsync.py", line 1594, in _import_packages_process
    sync_handlers.import_packages(batch, sources)
  File "/usr/lib/python2.6/site-packages/spacewalk/satellite_tools/sync_handlers.py", line 391, in import_packages
    importer.run()
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/importLib.py", line 649, in run
    self.fix()
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/packageImport.py", line 297, in fix
    self.backend.processChangeLog(self.changelog_data)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/importlib/backend.py", line 103, in processChangeLog
    h.execute(name=val['name'], time=val['time'], text=val['text'])
  File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 147, in execute
    return self._execute_wrapper(self._execute, *p, **kw)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 286, in _execute_wrapper
    retval = function(*p, **kw)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/sql_base.py", line 201, in _execute
    return self._execute_(args, kwargs)
  File "/usr/lib/python2.6/site-packages/spacewalk/server/rhnSQL/driver_postgresql.py", line 305, in _execute_
    self._real_cursor.execute(self.sql, params)
DataError: invalid byte sequence for encoding "UTF8": 0xc2 0x20


As per initial testing, it looks like syncing on Satellite with Oracle DB backend works (but sync is still running, so not sure).

Comment 1 Jan Hutař 2016-01-19 20:14:57 UTC
Yep, works on Sat570 with Oracle:

[...]
09:40:12 Importing package metadata
09:40:12    Importing *relevant* package metadata: rhel-x86_64-hpc-node-7 (3705)
            ________________________________________
Importing:  ######################################## - complete
[...]

Comment 2 Jan Hutař 2016-01-20 07:13:47 UTC
When I have created RHEL7 dump on Satellite 5.7.0 @ embedded PostgreSQL, we were able to sync it into reproducer Satellite 5.6.0.

Comment 3 jiri.blaha 2016-04-10 12:21:39 UTC
Hi,
tried to import  "RHEL 7 (x86_64) + EUS + RHN Tools + Optional (Base 2015-11-22)" and "RHEL 7 (x86_64) + EUS + RHN Tools + Optional (Base 2015-03-08)" to Satellite 5.7.0@PostgreSQL with the same result:
DataError('invalid byte sequence for encoding "UTF8": 0xc2 0x20\n',)
invalid byte sequence for encoding "UTF8": 0xc2 0x20

Comment 4 Tomas Lestach 2016-04-13 11:57:27 UTC
This BZ looks to me like duplicate of Bug 1264467. Can anybody confirm that?

Comment 5 Jan Dobes 2016-04-13 13:27:03 UTC
Yes, it seem identical. Closing this bug, the second bug is already in MODIFIED state.

*** This bug has been marked as a duplicate of bug 1264467 ***