Bug 1158945 - Pulp cannot publish rpms with non-ascii characters. Falling due gzip
Summary: Pulp cannot publish rpms with non-ascii characters. Falling due gzip
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Pulp
Classification: Retired
Component: rpm-support
Version: 2.4.1
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: 2.6.0
Assignee: Jeremy Cline
QA Contact: Irina Gulina
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-10-30 14:47 UTC by Jindrich Luza
Modified: 2015-02-28 22:42 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-28 22:42:09 UTC
Embargoed:


Attachments (Terms of Use)
patch for plugins/pulp_rpm/plugins/distributors/yum/metadata/primary.py (2.05 KB, patch)
2014-10-30 14:47 UTC, Jindrich Luza
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Pulp Redmine 598 0 None None None Never

Description Jindrich Luza 2014-10-30 14:47:43 UTC
Created attachment 952176 [details]
patch for plugins/pulp_rpm/plugins/distributors/yum/metadata/primary.py

reproducer:
------------

vim dummy.spec
================================================
Summary: Dummy Package to provide tomcat5
Name: dummy-tomcat
Version: 1.0
Release: 1
Group: System Environment/Base
License: Beerware
BuildArch: noarch
Provides: tomcat5, tomcat5-admin-webapps

%description
řšěřčšřšěřě (or some other non-ascii chars)

%files
===============================================

rpmbuild -bb dummy.spec

pulp-admin rpm repo create --repo-id unicode_problem
pulp-admin rpm repo uploads rpm --file dummy* --repo-id unicode_problem
pulp-admin -u admin -p admin rpm repo publish run --repo-id unicode_problem

OUTPUT:
+----------------------------------------------------------------------+
                Publishing Repository [unicode_problem]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... failed
Incorrect length of data produced


Task Failed

Incorrect length of data produced




Affected files:
plugins/pulp_rpm/plugins/distributors/yum/metadata/primary.py
plugins/pulp_rpm/plugins/distributors/yum/metadata/other.py
plugins/pulp_rpm/plugins/distributors/yum/metadata/filelists.py

patch for primary.py in attachement

Comment 1 Jeff Ortel 2014-11-03 17:10:59 UTC
Reproduce using UTF-8 description.

Comment 2 Jeremy Cline 2014-12-01 21:48:58 UTC
https://github.com/pulp/pulp_rpm/pull/605

Comment 3 Chris Duryee 2014-12-23 20:52:44 UTC
fixed in pulp 2.6.0-0.2.beta

Comment 4 Jiri Tyr 2015-01-06 00:55:02 UTC
There is a problem with the proposed patch:
https://github.com/pulp/pulp_rpm/pull/605#issuecomment-68809841

Comment 5 Irina Gulina 2015-01-22 15:22:44 UTC
>> rpm -qa pulp-server
pulp-server-2.6.0-0.5.beta.el6.noarch


>> rpm -qpil /home/ec2-user/rpmbuild/RPMS/x86_64/dummy-client-1.0-el6.x86_64.rpm
Name        : dummy-client                Relocations: /usr 
Version     : 1.0                               Vendor: (none)
Release     : el6                           Build Date: Thu 22 Jan 2015 09:57:34 AM EST
Install Date: (not installed)               Build Host: 
Group       : Development/Tools             Source RPM: dummy-client-1.0-el6.src.rpm
Size        : 0                                License: GPL
Signature   : (none)
Summary     : dummy
Description :
sample řšěřčšřšěřě (or some other non-ascii chars)
/usr/local/dummy-client
/usr/local/dummy-client/test.txt

>> pulp-admin rpm repo create --repo-id unicode_problem
Successfully created repository [unicode_problem]

>> pulp-admin rpm repo uploads rpm --file /home/ec2-user/rpmbuild/RPMS/x86_64/dummy-client-1.0-el6.x86_64.rpm --repo-id unicode_problem
+----------------------------------------------------------------------+
                              Unit Upload
+----------------------------------------------------------------------+

Extracting necessary metadata for each request...
[==================================================] 100%
Analyzing: dummy-client-1.0-el6.x86_64.rpm
... completed

Creating upload requests on the server...
[==================================================] 100%
Initializing: dummy-client-1.0-el6.x86_64.rpm
... completed

Starting upload of selected units. If this process is stopped through ctrl+c,
the uploads will be paused and may be resumed later using the resume command or
cancelled entirely using the cancel command.

Uploading: dummy-client-1.0-el6.x86_64.rpm
[==================================================] 100%
2106/2106 bytes
... completed

Importing into the repository...
This command may be exited via ctrl+c without affecting the request.


[\]
Running...

Task Succeeded


Deleting the upload request...
... completed

>> pulp-admin rpm repo list
+----------------------------------------------------------------------+
                            RPM Repositories
+----------------------------------------------------------------------+

Id:                  unicode_problem
Display Name:        unicode_problem
Description:         None
Content Unit Counts: 
  Rpm: 1

>> pulp-admin -u admin -p admin rpm repo publish run --repo-id unicode_problem
+----------------------------------------------------------------------+
                Publishing Repository [unicode_problem]
+----------------------------------------------------------------------+

This command may be exited via ctrl+c without affecting the request.


Initializing repo metadata
[-]
... completed

Publishing Distribution files
[-]
... completed

Publishing RPMs
[-]
... completed

Publishing Delta RPMs
... skipped

Publishing Errata
[-]
... completed

Publishing Comps file
[-]
... completed

Publishing Metadata.
[-]
... completed

Closing repo metadata
[-]
... completed

Generating sqlite files
... skipped

Publishing files to web
[-]
... completed

Writing Listings File
[-]
... completed


Task Succeeded

>> ls -l /var/lib/pulp/published/yum/master/yum_distributor/unicode_problem/1421938997.07/
total 8
lrwxrwxrwx. 1 apache apache  152 Jan 22 10:03 dummy-client-1.0-el6.x86_64.rpm -> /var/lib/pulp/content/rpm/dummy-client/1.0/el6/x86_64/b322e2027529911bb7b5d524ba808b70a8e01e73b1a5f2381ce45130b35a142e/dummy-client-1.0-el6.x86_64.rpm
drwxr-x---. 2 apache apache 4096 Jan 22 10:03 repodata

Comment 6 Brian Bouterse 2015-02-28 22:42:09 UTC
Moved to https://pulp.plan.io/issues/598


Note You need to log in before you can comment on or make changes to this bug.