Bug 890511

Summary:

python error or traceback mail when a UTF-8 character is present in a package files list

Product:

[Community] Spacewalk

Reporter:

Marco Giunta <giun7a>

Component:

Server

Assignee:

Michael Mráka <mmraka>

Status:

CLOSED CURRENTRELEASE

QA Contact:

Red Hat Satellite QA List <satqe-list>

Severity:

unspecified

Docs Contact:

Priority:

unspecified

Version:

1.8

Target Milestone:

---

Target Release:

---

Hardware:

x86_64

OS:

Linux

Whiteboard:

Fixed In Version:

spacewalk-backend-1.8.85-1.el6.noarch rhnpush-5.5.55-1.el6.noarch

Doc Type:

Bug Fix

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2013-01-08 10:05:21 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Bug Depends On:

Bug Blocks:

1484117

Attachments:

Description	Flags
rhnpush traceback mail of Macaulay2-1.3.1-9.el6.x86_64.rpm	none
rhnpush traceback mail of python-pyramid-1.2.7-6.el6.noarch.rpm	none

Description Marco Giunta 2012-12-27 12:22:16 UTC

Created attachment 669575 [details]
rhnpush traceback mail of Macaulay2-1.3.1-9.el6.x86_64.rpm

Description of problem:
I've a Spacewalk server just upgraded to 1.8 (but the problem was the same on 1.7 version) with postgresql backend on a RHEL-like 6.3

I'm not able to upload a package to a channel when in its files list there is a UTF-8 character, like 'é' or 'ü'.

e.g.:

python-pyramid-1.2.7-6.el6.noarch.rpm (from EPEL):
...
/usr/lib/python2.6/site-packages/pyramid/tests/fixtures/static/héhé
/usr/lib/python2.6/site-packages/pyramid/tests/fixtures/static/héhé.html
/usr/lib/python2.6/site-packages/pyramid/tests/fixtures/static/héhé/index.html
...


Macaulay2-1.3.1-9.el6.x86_64.rpm (from EPEL) :
...
/usr/share/doc/Macaulay2/Macaulay2Doc/html/___Gröbner_spbases.html
...

If I try to upload with 'spacewalk-repo-sync', it returns:

1/2 : python-pyramid-1.2.7-6.el6-0.noarch
'ascii' codec can't encode character u'\xe9' in position 64: ordinal not in range(128)
2/2 : Macaulay2-1.3.1-9.el6-0.x86_64
'ascii' codec can't encode character u'\xf6' in position 48: ordinal not in range(128)

and if I try with 'rhnpush', I receive a traceback mail (attached).


Version-Release number of selected component (if applicable):

spacewalk-backend-1.8.85-1.el6.noarch
spacewalk-base-1.8.49-1.el6.noarch
spacewalk-backend-sql-postgresql-1.8.85-1.el6.noarch
spacewalk-backend-package-push-server-1.8.85-1.el6.noarch
spacewalk-postgresql-1.8.6-1.el6.noarch

postgresql-8.4.13-1.el6_3.x86_64
postgresql-server-8.4.13-1.el6_3.x86_64

How reproducible:

Use 'spacewalk-repo-sync' or 'rhnpush' to upload a package to a Spacewalk channel


Steps to Reproduce:

1. wget http://dl.fedoraproject.org/pub/epel/6/x86_64/Macaulay2-1.3.1-9.el6.x86_64.rpm
2. rhnpush -c CHANNELNAME -u USERNAME -p PASSWORD ./Macaulay2-1.3.1-9.el6.x86_64.rpm

  
Actual results:

Traceback mail


Expected results:

Package uploaded

Comment 1 Marco Giunta 2012-12-27 12:23:05 UTC

Created attachment 669576 [details]
rhnpush traceback mail of python-pyramid-1.2.7-6.el6.noarch.rpm

Comment 2 Michael Mráka 2013-01-08 08:13:01 UTC

Trying to reproduce bug on Spacewalk nightly but it looks like it's been already fixed:

# spacewalk-repo-sync -c   epel6-centos6-x86_64 -i Macaulay2
Repo URL: http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64
Packages in repo:              8282
Packages passed filter rules:     7
Packages already synced:          0
Packages to sync:                 7
1/7 : normaliz-2.2-3.el6-0.x86_64
2/7 : ntl-5.5.2-1.el6-0.x86_64
3/7 : gfan-0.3-8.el6-0.x86_64
4/7 : pari-2.3.5-2.el6-0.x86_64
5/7 : 4ti2-1.3.2-6.el6-0.x86_64
6/7 : Macaulay2-1.3.1-9.el6-0.x86_64
7/7 : mpir-1.3.1-4.el6-0.x86_64
Repo http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64 has comps file 7e84a5b73de1fe41d9507a76731256be64c61ee2fec1b03b1b0b2ac3b782801a-comps-el6.xml.gz.
Repo http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64 has 1975 errata.
1974 errata skipped because of empty package list.
Sync completed.
Total time: 0:03:52
# spacewalk-repo-sync -c   epel6-centos6-x86_64 -i python-pyramid
Repo URL: http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64
Packages in repo:              8282
Packages passed filter rules:    10
Packages already synced:          0
Packages to sync:                10
1/10 : python-chameleon-2.5.3-1.el6.1-0.noarch
2/10 : python-zope-configuration-3.7.2-4.el6-0.noarch
3/10 : python-repoze-lru-0.4-2.el6-0.noarch
4/10 : python-webob1.2-1.2.1-4.el6-0.noarch
5/10 : python-zope-deprecation-3.5.1-1.el6-0.noarch
6/10 : python-translationstring-0.4-1.el6-0.noarch
7/10 : python-mako0.4-0.4.2-7.el6-0.noarch
8/10 : python-venusian-1.0-0.4.a3.el6-0.noarch
9/10 : python-zope-component-3.10.0-5.el6-0.noarch
10/10 : python-pyramid-1.2.7-6.el6-0.noarch
Repo http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64 has comps file 7e84a5b73de1fe41d9507a76731256be64c61ee2fec1b03b1b0b2ac3b782801a-comps-el6.xml.gz.
Repo http://mirrors.fedoraproject.org/mirrorlist?repo=epel-6&arch=x86_64 has 1975 errata.
1964 errata skipped because of empty package list.
Sync completed.
Total time: 0:01:59


Also webUI shows correct filenames:
https://<spacewalk>/network/software/packages/file_list.pxt?upper=300&pid=17&lower=276
/usr/lib/python2.6/site-packages/pyramid/tests/fixtures/static/héhé 	(Directory) 	 2012-10-10 01:06:27 	   
/usr/lib/python2.6/site-packages/pyramid/tests/fixtures/static/héhé.html 	SHA256: f93b9625c00cbd11ba1c12ccdff8b04085efbb4fa0552a3cb046736dc6815f60 	 2012-01-19 08:36:51 	23 bytes 
/usr/lib/python2.6/site-packages/pyramid/tests/fixtures/static/héhé/index.html 	SHA256: f7449a0073e930e75a9d0f9bed7409e0380d8d19f613f4117e70d74c276332cd 	 2012-01-19 08:36:51 	18 bytes

https://<spacewalk>/network/software/packages/file_list.pxt?upper=3000&pid=6&lower=2801
/usr/share/doc/Macaulay2/Macaulay2Doc/html/___Gröbner_spbases.html 	SHA256: c150c3c36718cb2f0731f60c69ed5497d46bf686491fba93977b556a27c880b1 	 2011-05-26 19:31:49 	2,720 bytes

Comment 3 Michael Mráka 2013-01-08 08:27:27 UTC

rhnpush on Spacewalk nightly also passes just fine:
# rhnpush -v -c epel6-centos6-i386 Macaulay2-1.3.1-9.el6.i686.rpm
Connecting to https://localhost/APP
Red Hat Network username: admin
Red Hat Network password: 
Package Macaulay2-1.3.1-9.el6.i686.rpm Not Found on RHN Server -- Uploading
Uploading package Macaulay2-1.3.1-9.el6.i686.rpm
Using POST request

https://<spacewalk>/network/software/packages/file_list.pxt?upper=3000&pid=18&lower=2600
/usr/share/doc/Macaulay2/Macaulay2Doc/html/___Gröbner_spbases.html 	SHA256: c150c3c36718cb2f0731f60c69ed5497d46bf686491fba93977b556a27c880b1 	 2011-05-26 19:28:04 	2,720 bytes

Comment 4 Michael Mráka 2013-01-08 10:05:21 UTC

I can see the same (correct) behavior in Spacewalk 1.8.

Might be your PostgreSQL has different than (spacewalk) default settings which leads to this error.

Comment 5 Marco Giunta 2013-01-09 09:49:04 UTC

You are right, there was a problem on my Spacewalk installation.

On a fresh installed RHEL-clone 6.3 clone, default encoding for pgsql database is 'sql_ascii', so when I followed the instruction on Spacewalk wiki to create the database 'spaceschema', it was created with 'SQL_ASCII' encoding:

# su - postgres -c 'psql -l'
                                List of databases
    Name     |  Owner   | Encoding  | Collation | Ctype |    Access privileges    
-------------+----------+-----------+-----------+-------+-------------------------
...
spaceschema  | postgres | SQL_ASCII | C         | C     | 
...

but Spacewalk use UTF8 encoding for db operation. 

To solve my problem, I've dumped my db, dropped, created a new one with UTF8 encoding and restored dumped data:

# su - postgres -c 'pg_dump spaceschema > /tmp/spaceschema.dump'

# su - postgres -c 'dropdb spaceschema'

# su - postgres -c 'createdb -E UTF8 -T template0 spaceschema'

# su - postgres -c 'psql -l'
                                List of databases
    Name     |  Owner   | Encoding  | Collation | Ctype |    Access privileges    
-------------+----------+-----------+-----------+-------+-------------------------
...
spaceschema | postgres | UTF8      | C         | C     | 
...

# su - postgres -c 'psql spaceschema < /tmp/spaceschema.dump'


and now I'm able to upload 'Macaulay2' and 'python-pyramid' without any error message:

# rhnpush -v -c epel-x86_64-el6 /srv/satellite/mirrors/epel-x86_64-el6/Macaulay2-1.3.1-9.el6.x86_64.rpm  
Connecting to https://localhost/APP
Package /srv/satellite/mirrors/epel-x86_64-el6/Macaulay2-1.3.1-9.el6.x86_64.rpm Not Found on RHN Server -- Uploading
Uploading package /srv/satellite/mirrors/epel-x86_64-el6/Macaulay2-1.3.1-9.el6.x86_64.rpm
Using POST request

# rhnpush -v -c epel-x86_64-el6 /srv/satellite/mirrors/epel-x86_64-el6/python-pyramid-1.2.7-6.el6.noarch.rpm 
Connecting to https://localhost/APP
Package /srv/satellite/mirrors/epel-x86_64-el6/python-pyramid-1.2.7-6.el6.noarch.rpm Not Found on RHN Server -- Uploading
Uploading package /srv/satellite/mirrors/epel-x86_64-el6/python-pyramid-1.2.7-6.el6.noarch.rpm
Using POST request


Maybe the instruction on Spacewalk wiki, should be changed to force the creating of pgsql database with UTF8 encoding, something like this:


su - postgres -c 'PGPASSWORD=spacepw; createdb -E UTF8 -T template0 spaceschema ; createlang plpgsql spaceschema ; yes $PGPASSWORD | createuser -P -sDR spaceuser'


Cheers,
  Marco

Comment 6 Michael Mráka 2013-01-09 10:21:04 UTC

Wiki page is fixed. Thanks for your report.

Comment 7 Eric Herget 2017-09-28 17:56:03 UTC

This BZ closed some time during 2.5, 2.6 or 2.7.  Adding to 2.7 tracking bug.