Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1425198

Summary: vdsm 4.19.x cannot be added to 3.6 engine
Product: [oVirt] ovirt-engine Reporter: Jiri Belka <jbelka>
Component: BLL.InfraAssignee: Oved Ourfali <oourfali>
Status: CLOSED DUPLICATE QA Contact: Pavel Stehlik <pstehlik>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: ---CC: bugs, jbelka, mperina
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-02-21 16:36:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jiri Belka 2017-02-20 19:16:14 UTC
Description of problem:

vdsm 4.19.x cannot be added to 3.6 engine.

2017-02-20 20:05:06,656 DEBUG [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-63) [e12f9cd] Permission check skipped for internal action SetNonOperationalVds.
2017-02-20 20:05:06,698 INFO  [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (DefaultQuartzScheduler_Worker-63) [e12f9cd] Running command: SetNonOperationalVdsCommand(NonOperationalReason = VERSION_INCOM
PATIBLE_WITH_CLUSTER, StorageDomainId = 00000000-0000-0000-0000-000000000000, CustomLogValues = {VdsSupportedVersions=4.19, CompatibilityVersion=[4.9, 4.10, 4.11, 4.12, 4.13, 4.14, 4.15, 4.16, 4.17]}, IsInternal 
= true, StopGlusterService = false, VdsId = a0a854d1-55ba-4918-aa6d-1b9bf79c0edf, RunSilent = false) internal: true. Entities affected :  ID: a0a854d1-55ba-4918-aa6d-1b9bf79c0edf Type: VDS
2017-02-20 20:05:06,700 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) [e12f9cd] START, SetVdsStatusVDSCommand(HostName = dell-r210ii-03, SetVdsStatusVDSCommandP
arameters:{runAsync='true', hostId='a0a854d1-55ba-4918-aa6d-1b9bf79c0edf', status='NonOperational', nonOperationalReason='VERSION_INCOMPATIBLE_WITH_CLUSTER', stopSpmFailureLogged='false', maintenanceReason='null'
}), log id: 2870e77e
2017-02-20 20:05:06,715 INFO  [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (DefaultQuartzScheduler_Worker-63) [e12f9cd] FINISH, SetVdsStatusVDSCommand, log id: 2870e77e
2017-02-20 20:05:06,873 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-63) [e12f9cd] Correlation ID: e12f9cd, Job ID: e949ca7f-e245-4e93-8f90-55d56c1f4
f79, Call Stack: null, Custom Event ID: -1, Message: Host dell-r210ii-03 is installed with VDSM version (4.19) and cannot join cluster upgrade which is compatible with VDSM versions [4.9, 4.10, 4.11, 4.12, 4.13, 
4.14, 4.15, 4.16, 4.17].


engine=# select option_name,option_value from vdc_options where option_name ilike '%version%';
           option_name           |                option_value                 
---------------------------------+---------------------------------------------
 RemoteViewerSupportedVersions   | 
 RemoteViewerNewerVersionUrl     | ${console_client_resources_url}
 MinimalETLVersion               | 3.6.0
 ProductRPMVersion               | 3.6.10.1-0.1.el6
 SupportedVDSMVersions           | 4.9,4.10,4.11,4.12,4.13,4.14,4.15,4.16,4.17
 VdcVersion                      | 3.6.0.0
 CheckMixedRhelVersions          | false
 CheckMixedRhelVersions          | true
 CheckMixedRhelVersions          | true
 CheckMixedRhelVersions          | true
 CheckMixedRhelVersions          | true
 CheckMixedRhelVersions          | true
 BootstrapMinimalVdsmVersion     | 4.9
 OvirtInitialSupportedIsoVersion | 2.5.5:5.8
 CheckMixedRhelVersions          | false
(15 rows)


Version-Release number of selected component (if applicable):
rhevm-3.6.10.1-0.1.el6.noarch

How reproducible:
100%

Steps to Reproduce:
1. add vdsm 4.19.x into 3.6 engine
2.
3.

Actual results:
fails

Expected results:
should work, needed for migration from 3.6DC/3.5 CL to 4.1

Additional info:

Comment 1 Oved Ourfali 2017-02-21 07:01:20 UTC
What's the exact VDSM version you're working with?
4.19.6?

Comment 2 Martin Perina 2017-02-21 07:48:15 UTC
Why do you need to add VDSM 4.19.z from oVirt 4.1 into oVirt 3.6? Shouldn't engine be always upgraded first using below steps?

1. Upgrade all DCs/clusters to 3.6 level (mandatory)
2. Upgrade 3.6 engine to 4.0 engine
3. Upgrade all DCs/cluster to 4.0 level (optional, but recommended)
3. Upgrade 4.0 engine to 4.1 engine

Comment 3 Jiri Belka 2017-02-21 09:15:50 UTC
(In reply to Martin Perina from comment #2)
> Why do you need to add VDSM 4.19.z from oVirt 4.1 into oVirt 3.6? Shouldn't
> engine be always upgraded first using below steps?
> 
> 1. Upgrade all DCs/clusters to 3.6 level (mandatory)
> 2. Upgrade 3.6 engine to 4.0 engine
> 3. Upgrade all DCs/cluster to 4.0 level (optional, but recommended)
> 3. Upgrade 4.0 engine to 4.1 engine

[root@dell-r210ii-03 ~]# rpm -q vdsm ; cat /etc/os-release 
vdsm-4.19.4-1.el7ev.x86_64
NAME="Red Hat Enterprise Linux"
VERSION="7.3"
VERSION_ID="7.3"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Red Hat Virtualization Host"
VARIANT_ID="ovirt-node"
PRETTY_NAME="Red Hat Virtualization Host 4.1 (el7.3)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.3:GA:hypervisor"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

# FIXME
REDHAT_BUGZILLA_PRODUCT="Red Hat Virtualization"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.3
REDHAT_SUPPORT_PRODUCT="Red Hat Virtualization"
REDHAT_SUPPORT_PRODUCT_VERSION=7.3

Why? A customer could have 3.6 engine with 3.5 EL6 hosts. He would need to somehow move to 3.6 EL7 hosts, so he could go via 3.6 NGN. Then we could update 3.6 NGN to 4.1 NGN and he would get this issue.

See vdsm 4.19 supports 3.6 engine!

[root@dell-r210ii-03 ~]# sed -n '/^version_info/,$p' /usr/lib/python2.7/site-packages/vdsm/dsaversion.py
version_info = {
    'version_name': version_name,
    'software_version': software_version,
    'software_revision': software_revision,
    'supportedENGINEs': ['3.6', '4.0', '4.1'],
    'clusterLevels': ['3.6', '4.0', '4.1'],
}
[root@dell-r210ii-03 ~]# rpm -q vdsm
vdsm-4.19.4-1.el7ev.x86_64

Comment 4 Oved Ourfali 2017-02-21 09:40:19 UTC
Can you please text with 4.19.6?

Comment 5 Jiri Belka 2017-02-21 09:44:00 UTC
(In reply to Oved Ourfali from comment #4)
> Can you please text with 4.19.6?

How is this relevant if 4.19 is missing in vdc_options' SupportedVDSMVersions?

Comment 6 Oved Ourfali 2017-02-21 09:53:19 UTC
The validation we do consist of more than this field. 
In 4.19.6 we have re-added 3.6 support in 4.1 vdsm, after we removed it upon request.

Comment 7 Dan Kenigsberg 2017-02-21 14:17:21 UTC
Using  a fresh vdsm-4.19.6 may hide the issue, but I don't recall the importance of this Engine-DB field:

SupportedVDSMVersions           | 4.9,4.10,4.11,4.12,4.13,4.14,4.15,4.16,4.17

It is missing Vdsm 4.18 and 4.19 (ovirt-4.0 and ovirt-4.1 respectively) which *are* (at least should be) supported by Engine 3.6

Comment 8 Oved Ourfali 2017-02-21 15:39:03 UTC
Both fields come to complete one another. 
So, having both isn't mandatory. 
That is for the case where you no longer release engine or vdsm of a specific version.

Comment 9 Jiri Belka 2017-02-21 16:14:53 UTC
(In reply to Oved Ourfali from comment #8)
> Both fields come to complete one another. 
> So, having both isn't mandatory. 
> That is for the case where you no longer release engine or vdsm of a
> specific version.

host with vdsm 4.19.6 is up in 3.6 engine (3.6 clstr compat level). I used latest 4.1 NGN and did yum update from 4.1 internal build to get latest vdsm (as 4.1 NGN build doesn't have 4.19.6 yet).

engine=# select vds_name,rpm_version,software_version,version_name from vds where vds_name = 'dell-r210ii-03';
    vds_name    |     rpm_version     | software_version | version_name 
----------------+---------------------+------------------+--------------
 dell-r210ii-03 | vdsm-4.19.6-1.el7ev | 4.19             | Snow Man
(1 row)

engine=# select * from vdc_options where option_name = 'SupportedVDSMVersions';
 option_id |      option_name      |                option_value                 | version 
-----------+-----------------------+---------------------------------------------+---------
       600 | SupportedVDSMVersions | 4.9,4.10,4.11,4.12,4.13,4.14,4.15,4.16,4.17 | general
(1 row)

Comment 10 Oved Ourfali 2017-02-21 16:36:39 UTC
Closing as duplicate of bug 1403846.
Jiri,shall you move it to verified?

*** This bug has been marked as a duplicate of bug 1403846 ***