Bug 1192436

Summary:	invalid byte sequence for encoding "UTF8": 0xfd on setup
Product:	Red Hat Enterprise Virtualization Manager	Reporter:	David Caro <dcaroest>
Component:	ovirt-engine	Assignee:	Eli Mesika <emesika>
Status:	CLOSED CURRENTRELEASE	QA Contact:	Karolína Hajná <khajna>
Severity:	unspecified	Docs Contact:
Priority:	unspecified
Version:	3.6.0	CC:	amureini, bazulay, dcaroest, eedri, gklein, lsurette, pstehlik, rbalakri, Rhev-m-bugs, sbonazzo, srevivo, ykaul
Target Milestone:	ovirt-3.6.0-rc	Keywords:	Regression
Target Release:	3.6.0
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-04-20 01:38:29 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	Infra	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description David Caro 2015-02-13 10:53:43 UTC

Description of problem:
When running the engine setup, the database does not get installed because there's an error when applying the database changes.

Version-Release number of selected component (if applicable):
3.6.0

How reproducible:
Setup engine

Steps to Reproduce:
1.Setup engine
2.
3.

Actual results:
Failed to setup

Expected results:
Setup succeeded

Additional info:

You can find a failing job at http://jenkins-ci.eng.lab.tlv.redhat.com/view/0%20Unstable%203.6/job/rhevm_3.6_el6-engine_el6-host_automation_infra_one_host_restapi_mixed_nfs_rest_factory/

2015-02-13 06:40:00 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.executeRaw:803 execute-result: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20150213063935-03allv.log', '-c', 'apply'], rc=1
2015-02-13 06:40:00 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.execute:861 execute-output: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20150213063935-03allv.log', '-c', 'apply'] stdout:
Creating schema engine@localhost:5432/engine
Creating fresh schema
Creating tables...
Creating functions...
Creating common functions...
Inserting data from /usr/share/ovirt-engine/dbscripts/data/00000_insert_schema_version.sql ...
Inserting data from /usr/share/ovirt-engine/dbscripts/data/00100_insert_users.sql ...

2015-02-13 06:40:00 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.execute:866 execute-output: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20150213063935-03allv.log', '-c', 'apply'] stderr:
psql:/usr/share/ovirt-engine/dbscripts/data/00100_insert_users.sql:10: ERROR:  invalid byte sequence for encoding "UTF8": 0xfd
HINT:  This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".
FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/data/00100_insert_users.sql

2015-02-13 06:40:00 DEBUG otopi.context context._executeMethod:152 method exception
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/otopi/context.py", line 142, in _executeMethod
    method['method']()
  File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/db/schema.py", line 291, in _misc
    oenginecons.EngineDBEnv.PGPASS_FILE
  File "/usr/lib/python2.6/site-packages/otopi/plugin.py", line 871, in execute
    command=args[0],
RuntimeError: Command '/usr/share/ovirt-engine/dbscripts/schema.sh' failed to execute

Comment 1 Sandro Bonazzola 2015-02-13 11:38:10 UTC

Eli, looks like /usr/share/ovirt-engine/dbscripts/data/00100_insert_users.sql contains invalid characters, can you take a look?

Comment 3 Eli Mesika 2015-02-15 09:10:45 UTC

I had created 2 databases 

1) based on latest master named engine_master
2) based on 3.5 branch named engine_3_5

I had compared the admin entry in both databases and even the length of the external_id (to see if we have any hidden chars in the value) and found that both were exactly the same 

In other words, the entry of the admin user in master is exactly the same as in the 3.5 branch 

master database 
---------------

engine_1192436=# \x
Expanded display is on.
engine_1192436=# select * from users;
-[ RECORD 1 ]-----------+-------------------------------------
user_id                 | fdfc627c-d875-11e0-90f0-83df133b58cc
name                    | admin
surname                 |
domain                  | internal
username                | admin
department              |
email                   |
note                    |
last_admin_check_status | t
external_id             | fdfc627c-d875-11e0-90f0-83df133b58cc
_create_date            | 2015-02-15 11:02:54.277046+02
_update_date            |
namespace               | *

engine_1192436=# select length(external_id) from users;
-[ RECORD 1 ]
length | 36


3.5 database
-------------

engine_3_5=# \x
Expanded display is on.
engine_3_5=# select * from users;
-[ RECORD 1 ]-----------+-------------------------------------
user_id                 | fdfc627c-d875-11e0-90f0-83df133b58cc
name                    | admin
surname                 |
domain                  | internal
username                | admin
groups                  |
department              |
role                    |
email                   |
note                    |
last_admin_check_status | t
group_ids               |
external_id             | fdfc627c-d875-11e0-90f0-83df133b58cc
active                  | t
_create_date            | 2015-02-15 11:00:44.027534+02
_update_date            |
namespace               | *

engine_3_5=# select length(external_id) from users;
-[ RECORD 1 ]
length | 36

Comment 5 David Caro 2015-02-16 10:38:10 UTC

Re-adding the needinfo flag, did not want to remove it

Comment 6 David Caro 2015-02-16 10:55:29 UTC

It's most probably related to the commits 2ef088137e27e2182323c8be4a90489d586f8dd1 and 04667f731e48a4781e902d7f1a5a53920810991e

Still looking

Comment 7 David Caro 2015-02-16 11:04:49 UTC

This one might also be involved

146eddc0c4c802a04073cd540f1c170cc08b9b25

Comment 8 David Caro 2015-02-16 11:20:19 UTC

I'm able to reproduce rebasing downstream master-rhev branch on top of latest upstream master branch. I have a slave ready for you to dig in if you want.

Comment 9 David Caro 2015-02-18 09:19:40 UTC

I can confirm that this fixes the issue on our CI env

Comment 10 Karolína Hajná 2015-04-29 10:25:46 UTC

Verified on 3.6.0-0.0.master.20150427173543.git61dec8c.el6