Bug 911650 - pulp-manage-db fails after upgrade
Summary: pulp-manage-db fails after upgrade
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Pulp
Classification: Retired
Component: z_other
Version: Master
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: ---
Assignee: Todd Sanders
QA Contact: Preethi Thomas
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2013-02-15 14:29 UTC by Preethi Thomas
Modified: 2013-04-08 16:03 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2013-04-08 16:03:25 UTC
Embargoed:


Attachments (Terms of Use)

Description Preethi Thomas 2013-02-15 14:29:43 UTC
Description of problem:
pulp-manage-db erroring out after upgrade

[root@preethi ~]# pulp-manage-db 
Beginning database migrations.
Migration package pulp.server.db.migrations is up to date at version 2
Migration package pulp_puppet.plugins.migrations is up to date at version 0
Applying pulp_rpm.migrations version 2
Applying migration pulp_rpm.migrations.0002_rpm_unit_repodata failed.  See log for details.
2013-02-15 09:27:03,714 db:CRITICAL: Applying migration pulp_rpm.migrations.0002_rpm_unit_repodata failed.
2013-02-15 09:27:03,714 db:CRITICAL: 'ascii' codec can't decode byte 0xc2 in position 341: ordinal not in range(128)
2013-02-15 09:27:03,715 db:CRITICAL: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pulp/server/db/manage.py", line 80, in migrate_database
    update_current_version=not options.test)
  File "/usr/lib/python2.7/site-packages/pulp/server/db/migrate/models.py", line 147, in apply_migration
    migration.migrate()
  File "/usr/lib/python2.7/site-packages/pulp_rpm/migrations/0002_rpm_unit_repodata.py", line 36, in migrate
    _migrate_rpm_unit_repodata()
  File "/usr/lib/python2.7/site-packages/pulp_rpm/migrations/0002_rpm_unit_repodata.py", line 31, in _migrate_rpm_unit_repodata
    rpm_unit["repodata"] = metadata.get_package_xml(rpm_unit['_storage_path'])
  File "/usr/lib/python2.7/site-packages/pulp_rpm/yum_plugin/metadata.py", line 375, in get_package_xml
    primary_xml_snippet = change_location_tag(po.xml_dump_primary_metadata(), pkg_path)
  File "/usr/lib/python2.7/site-packages/pulp_rpm/yum_plugin/metadata.py", line 401, in change_location_tag
    return first_portion + location + end_portion
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 341: ordinal not in range(128)

Database migrations complete.
Loading content types.
Content types loaded.
[root@preethi ~]# 


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 1 Randy Barlow 2013-02-15 14:48:22 UTC
This is a bug in either the RPM migration package (in the second migration), or in the upgrade script from v1 to v2. Looks like we have some encoding issues, somewhere, and the two Pulp versions don't get along. So we could either make the upgrade script change encodings to whatever encoding v2 uses (do we really use ASCII and not UTF-8?), or make this script more robust at dealing with other encodings.

We could consider having the pulp-manage-db script hide the traceback from the user, but if we want to do that, I think it should be a separate ticket.

Comment 2 Pradeep Kilambi 2013-02-18 15:47:18 UTC
The issue doesn't seem to be related to upgrades to me. So this is whats happening, When the migration script starts generating the metadata for rpms, some rpms have special encoding in either changelog or description. The metadata is generated by yum api and returned to us in ascii. Unfortunately yum api does not do any encoding(unless we're getting the xml directly in which case it does). Hopefully once we move away from yum apis in future we wont have this issue, but for now we have to work around this by converting the metadata strings on our end.

Note this wont happen with your latest v2. This only happens when we try to directly interact with yum apis to generate metadata.

Comment 3 Pradeep Kilambi 2013-02-19 18:10:42 UTC
fix merged to master

Comment 4 Jeff Ortel 2013-02-20 16:18:46 UTC
build: 2.1.0-0.12.alpha

Comment 5 Preethi Thomas 2013-02-20 16:34:41 UTC
verified
root@preethi ~]# rpm -q pulp-server
pulp-server-2.1.0-0.12.alpha.noarch
on running pulp-manage-db. (I couldn't copy the first part as I was running screen)

[root@preethi ~]# 



iso-8859-1 encoding on Trond Eivind Glomsr�d <teg>


iso-8859-1 encoding on Trond Eivind Glomsr�d <teg>


iso-8859-1 encoding on Trond Eivind Glomsr�d <teg> 2.0-7


iso-8859-1 encoding on Ville Skytt� <ville.skytta> - 2.8.2-2

Migration to pulp_rpm.migrations version 2 complete.
Applying pulp_rpm.migrations version 3
Migration to pulp_rpm.migrations version 3 complete.
Applying pulp_rpm.migrations version 4
Migration to pulp_rpm.migrations version 4 complete.
Applying pulp_rpm.migrations version 5
Migration to pulp_rpm.migrations version 5 complete.
Database migrations complete.
Loading content types.
Content types loaded.

Comment 91 Preethi Thomas 2013-04-08 16:03:25 UTC
Pulp 2.1 released 


http://www.pulpproject.org/2013/04/05/pulp-2-1-0-released/


Note You need to log in before you can comment on or make changes to this bug.