Bug 1254026 - Node unusable after upgrade from F20 to F21 (supported machine types list too long for database field)
Summary: Node unusable after upgrade from F20 to F21 (supported machine types list too...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: General
Version: ---
Hardware: x86_64
OS: Linux
high
high
Target Milestone: ovirt-3.5.5
: 3.5.5
Assignee: Omer Frenkel
QA Contact: Pavol Brilla
URL:
Whiteboard: virt
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-08-16 23:31 UTC by Jillian Morgan
Modified: 2016-02-10 19:23 UTC (History)
11 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2015-10-26 13:43:45 UTC
oVirt Team: Virt
Embargoed:
ylavi: ovirt-3.5.z?
ylavi: planning_ack+
rule-engine: devel_ack+
rule-engine: testing_ack?


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 45310 0 ovirt-engine-3.5 MERGED core: change supported emulated machines field type Never

Description Jillian Morgan 2015-08-16 23:31:58 UTC
Description of problem:

After upgrading a Node from F20 to F21, node would get stuck in state "unassigned" or "connecting", in a continuous loop.

oVirt Manager logs showed no reason for the error, in fact didn't show any error at all beside the node being stuck in "connecting" state.

VDSM logs on the node revealed no ERRORs.

Engine log however revealed:

PL/pgSQL function updatevdsdynamic(integer,integer,character varying,numeric,char
acter varying,boolean,integer,integer,integer,uuid,integer,integer,integer,intege
r,integer,integer,character varying,character varying,character varying,character
 varying,integer,character varying,integer,integer,integer,boolean,character vary
ing,character varying,character varying,character varying,character varying,chara
cter varying,character varying,character varying,character varying,integer,charac
ter varying,integer,character varying,character varying,character varying,charact
er varying,character varying,character varying,character varying,character varyin
g,smallint,integer,smallint,boolean,character varying,boolean,boolean,text) line
5 at SQL statement; nested exception is org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)

But this log failed to provide any detail of the actual parameters being fed to the SQL query, so had no idea WHAT FIELD was too long.


Using definition of the vds_dynamic table (to see which fields were restricted to 255 chars), and the reported capabilities list from the node's VDSM log, I narrowed down the culprit to the emulated machine types list being too long to fit into the supported_emulated_machines field.



Version-Release number of selected component (if applicable):

ovirt-engine 3.5.3
vdsm-4.16.20-0



How reproducible:
Once, so far (first of three nodes being upgraded.)

Steps to Reproduce:
1. Upgrade node to F21

Actual results:
Upgraded node unusable due to above database error.


Expected results:
Node should work anyways. Fix database to expand the field, or silently truncate the value being stored.
At least the Manager should show a report that the Node failed due to an engine database error.


Additional info:

The vds_dynamic table could not be directly altered due to the dependencies upon it by the "vds" view and it's subsidiaries.

engine=> alter table vds_dynamic alter column supported_emulated_machines type character varying(1000);
ERROR:  cannot alter type of a column used by a view or rule
DETAIL:  rule _RETURN on view vds depends on column "supported_emulated_machines"


I had to resort to this:

BEGIN;
LOCK TABLE pg_attribute IN EXCLUSIVE MODE;
update pg_attribute set atttypmod = 1004 where attrelid = 'vds_dynamic'::regclas
s AND attname = 'supported_emulated_machines';
COMMIT;

This expanded the field to 1000 characters, and now the node capabilities were successfully updated and the node works again.


FYI: the returned list of machine types from qemu-system-x86-2.3.0-5.fc21.x86_64 was:

'emulatedMachines': [u'pc', u'pc-1.3', u'pc-0.12', u'pc-q35-1.6',
u'pc-q35-1.5', u'pc-i440fx-1.6', u'pc-q35-2.2', u'pc-i440fx-1.7', u'xenpv',
u'pc-q35-2.1', u'pc-0.11', u'pc-0.10', u'pc-i440fx-2.2', u'pc-1.2',
u'isapc', u'pc-q35-1.4', u'xenfv', u'pc-0.15', u'pc-i440fx-1.5',
u'pc-i440fx-1.4', u'pc-q35-2.0', u'pc-0.14', u'pc-1.1', u'pc-q35-1.7',
u'pc-i440fx-2.1', u'pc-1.0', u'pc-i440fx-2.0', u'q35', u'pc-0.13']

which stringified as 275 characters: 
pc,pc-1.3,pc-0.12,pc-q35-1.6,pc-q35-1.5,pc-i440fx-1.6,pc-q35-2.2,pc-i440fx-1.7,xenpv,pc-q35-2.1,pc-0.11,pc-0.10,pc-i440fx-2.2,pc-1.2,isapc,pc-q35-1.4,xenfv,pc-0.15,pc-i440fx-1.5,pc-i440fx-1.4,pc-q35-2.0,pc-0.14,pc-1.1,pc-q35-1.7,pc-i440fx-2.1,pc-1.0,pc-i440fx-2.0,q35,pc-0.13

Comment 1 Doron Fediuck 2015-08-17 05:50:35 UTC
Oved, can you please do an initial check on this and route to the relevant team if needed?

Comment 2 Oved Ourfali 2015-08-17 08:37:58 UTC
Supported machine types is virt.

Comment 3 Omer Frenkel 2015-08-25 14:28:21 UTC
please note this was fixed in 3.6 by
https://gerrit.ovirt.org/#/c/39145/

so only backport to 3.5 is needed

Comment 4 Pavol Brilla 2015-10-12 12:50:01 UTC
Verified in
# rpm -q rhevm-setup
rhevm-setup-3.5.5-0.1.el6ev.noarch


supported_emulated_machines | text

Comment 5 Sandro Bonazzola 2015-10-26 13:43:45 UTC
oVirt 3.5.5 has been released including fixes for this issue.


Note You need to log in before you can comment on or make changes to this bug.