Bug 1255520 - foreman, candlepin, gutterball databases use default encoding vs explicit UTF8
Summary: foreman, candlepin, gutterball databases use default encoding vs explicit UTF8
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Installer
Version: 6.1.0
Hardware: Unspecified
OS: Unspecified
high
high vote
Target Milestone: Unspecified
Assignee: Ivan Necas
QA Contact: Katello QA List
URL: http://projects.theforeman.org/issues...
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-08-20 19:55 UTC by Shannon Hughes
Modified: 2019-08-15 05:11 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-15 18:20:38 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 1592713 0 None None None Never
Red Hat Product Errata RHBA-2015:1911 0 normal SHIPPED_LIVE Satellite 6.1.3 bug fix update 2015-10-15 22:19:19 UTC

Description Shannon Hughes 2015-08-20 19:55:26 UTC
Description of problem:

It appears when we create the foreman, gutterball and candlepin databases we might be using the default postgresql encoding for the db's vs explicitly forcing UTF8. This is causing issues during errata input: 

/opt/rh/ruby193/root/usr/share/ruby/rake/application.rb:63:in `run'
/opt/rh/ruby193/root/usr/bin/rake:32:in `<main>'
caused by: (ActiveRecord::StatementInvalid) PGError: ERROR:  character with byte sequence 0xe2 0x80 0x98 in encoding "UTF8" has no equivalent in encoding "LATIN1"
: UPDATE "katello_errata" SET "issued" = '2011-03-10', "severity" = '', "title" = 'strace bug fix update', "reboot_suggested" = 'f', "updated" = '2011-05-19', "description" = 'The strace program intercepts and records the system calls called and received
by a running process. It can print a record of each system call, its arguments
and its return value.
:::
/opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0.66/lib/katello/tasks/reindex.rake:37:in `index_objects'
    ... skipped 19 lines
caused by: (PGError) ERROR:  character with byte sequence 0xe2 0x80 0x98 in encoding "UTF8" has no equivalent in encoding "LATIN1"
 
    ... skipped 47 lines
Tasks: TOP => katello:reindex
[ lo1-padm-sat-3 ] #
 
I have researched several reports from your knowledge database with errata_import failing on upgrade from 6.0.8 to 6.1 but none of them correspond with the error:
caused by: (PGError) ERROR:  character with byte sequence 0xe2 0x80 0x98 in encoding "UTF8" has no equivalent in encoding "LATIN1"
 
and it seems to be on the latest available packages. The Satellite 6.1 server cannot be used usefully at the moment as there is no errata!!!
 
Is this of assistance:
-bash-4.2$ psql  -l
                                       List of databases
    Name    |  Owner   | Encoding |     Collate     |      Ctype      |    Access privileges
------------+----------+----------+-----------------+-----------------+-------------------------
 candlepin  | postgres | LATIN1   | en_GB.ISO8859-1 | en_GB.ISO8859-1 | =T/postgres            +
            |          |          |                 |                 | postgres=CTc/postgres  +
            |          |          |                 |                 | candlepin=CTc/postgres
 foreman    | foreman  | LATIN1   | en_GB.ISO8859-1 | en_GB.ISO8859-1 | =T/foreman             +
            |          |          |                 |                 | foreman=CTc/foreman
 gutterball | postgres | LATIN1   | en_GB.ISO8859-1 | en_GB.ISO8859-1 | =T/postgres            +
            |          |          |                 |                 | postgres=CTc/postgres  +
            |          |          |                 |                 | gutterball=CTc/postgres
 postgres   | postgres | LATIN1   | en_GB.ISO8859-1 | en_GB.ISO8859-1 |
 template0  | postgres | LATIN1   | en_GB.ISO8859-1 | en_GB.ISO8859-1 | =c/postgres            +
            |          |          |                 |                 | postgres=CTc/postgres
 template1  | postgres | LATIN1   | en_GB.ISO8859-1 | en_GB.ISO8859-1 | =c/postgres            +
            |          |          |                 |                 | postgres=CTc/postgres

From the following I am not seeing an explicit UTF8 creation, 

https://github.com/theforeman/puppet-foreman/blob/master/manifests/database/postgresql.pp

Comment 3 Ivan Necas 2015-09-03 19:11:35 UTC
Created redmine issue http://projects.theforeman.org/issues/11681 from this bug

Comment 4 Ivan Necas 2015-09-03 19:29:45 UTC
How to reproduce:

echo LANG=\"en_US.iso88591\" > /etc/locale.conf
export LANG=en_US.iso88591
export LANGUAGE=en_US.iso88591
export LC_CTYPE="en_US.iso88591
export LC_NUMERIC="en_US.iso88591
export LC_TIME="en_US.iso88591
export LC_COLLATE="en_US.iso88591
export LC_MONETARY="en_US.iso88591
export LC_MESSAGES="en_US.iso88591
export LC_PAPER="en_US.iso88591
export LC_NAME="en_US.iso88591
export LC_ADDRESS="en_US.iso88591
export LC_TELEPHONE="en_US.iso88591
export LC_MEASUREMENT="en_US.iso88591
export LC_IDENTIFICATION="en_US.iso88591
export LC_ALL=en_US.iso88591

katello-installer

# OR foreman-installer

Current results:

foreman, candlepin and gutterball databases have LATIN1 as encoding

Expected results:

su - postgres -c 'psql -l'
foreman, candlepin and gutterball databases have UTF-8 as encoding

Comment 6 Bryan Kearney 2015-09-03 20:04:15 UTC
Upstream bug component is Provisioning

Comment 7 Bryan Kearney 2015-09-04 08:04:52 UTC
Upstream bug component is Installer

Comment 8 Bryan Kearney 2015-09-04 08:04:53 UTC
Moving to POST since upstream bug http://projects.theforeman.org/issues/11681 has been closed

Comment 14 Corey Welton 2015-10-02 22:02:26 UTC
Verified that this patch made it into/works in latest compose

6.1.3 compose #2

Comment 16 errata-xmlrpc 2015-10-15 18:20:38 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2015:1911


Note You need to log in before you can comment on or make changes to this bug.