Description of problem: Following this article to upgrade 3.5 hosted-engine with 6.7 hosts to 3.6. https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html-single/Self-Hosted_Engine_Guide/index.html#Upgrading_the_Self-Hosted_Engine_from_6_to_7 1. Remove one of the host from RHEV-M 2. Create a new cluster for 7.x host 3. Reinstalled this hypervisor with Red Hat Enterprise Virtualization Hypervisor release 7.2 (20160302.1.el7ev) 4. Re-deploy the host with hosted-engine --deploy and used the same host id 2 5. Change the host into new cluster so that the host become active. However the ha-agent service in the 7.2 host is not coming up and can see below error in the agent.log. MainThread::ERROR::2016-03-20 09:51:01,831::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'Path to volume c23122fe-9b27-4ef6-b7b8-8816df43ae90 not found in /rhev/data-center/mnt' - trying to restart agent MainThread::WARNING::2016-03-20 09:51:06,837::agent::208::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '1' c23122fe-9b27-4ef6-b7b8-8816df43ae90 is conf_volume_UUID from the hosted-engine.conf file. This lv c23122fe-9b27-4ef6-b7b8-8816df43ae90 is not created too. However reinstalling the host with rhevh-7.2-20160219.0.el6ev.iso (latest version before 3.6) and upgrading to rhevh-7.2-20160302.1.el6ev.iso seems to be working good. conf_volume_UUID is also created. MainThread::INFO::2016-03-20 08:29:15,800::upgrade::151::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(_is_conf_volume_there) Looking for conf volume MainThread::ERROR::2016-03-20 08:29:15,918::upgrade::207::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(_is_conf_volume_there) Unable to find HE conf volume MainThread::INFO::2016-03-20 08:29:15,919::upgrade::262::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(_create_shared_conf_volume) Creating hosted-engine configuration volume on the shared storage domain Version-Release number of selected component (if applicable): ovirt-hosted-engine-setup-1.3.3.4-1.el7ev.noarch ovirt-hosted-engine-ha-1.3.4.3-1.el7ev.noarch How reproducible: 100% Steps to Reproduce: 1. Add a new rhevh-7.2-20160302.1.el6ev host to an existing 3.5 hosted engine setup. 2. ha-agent is not starting with below error. MainThread::ERROR::2016-03-20 09:51:01,831::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'Path to volume c23122fe-9b27-4ef6-b7b8-8816df43ae90 not found in /rhev/data-center/mnt' - trying to restart agent Actual results: ha-agent is not starting Expected results: ha-agent should work. Additional info:
Nijin, can you please retry with the next (today ?) rhev-h build which should include ovirt-ha-agente-1.3.5.1 ? I strongly suspect it's just a dusplicate of https://bugzilla.redhat.com/show_bug.cgi?id=1316143
Closing the bug with insufficient data. Please reopen if you can provide needed info.
Sorry for the delay. Reopening as I was able to replicate the issue in latest ha-agent . Tested with below version vdsm-4.17.23.2-1.el7ev.noarch ovirt-hosted-engine-ha-1.3.5.1-1.el7ev.noarch ovirt-hosted-engine-setup-1.3.4.0-1.el7ev.noarch MainThread::WARNING::2016-04-09 07:40:57,783::ovf_store::104::ovirt_hosted_engine_ha.lib.ovf.ovf_store.OVFStore::(scan) Unable to find OVF_STORE MainThread::ERROR::2016-04-09 07:40:57,783::config::234::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config::(refresh_local_conf_file) Unable to get vm.conf from OVF_STORE, falling back to initial vm.conf MainThread::ERROR::2016-04-09 07:40:57,784::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Error: 'Path to volume None not found in /rhev/data-center/mnt' - trying to restart agent MainThread::WARNING::2016-04-09 07:41:02,788::agent::208::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent) Restarting agent, attempt '5' The conf_volume value is none. grep "conf" /etc/ovirt-hosted-engine/hosted-engine.conf conf=/var/run/ovirt-hosted-engine-ha/vm.conf conf_volume_UUID=None conf_image_UUID=None As earlier mentioned, it works when we upgrade from 3.5 el6 version to the 3.5 el7 version as tested in https://bugzilla.redhat.com/show_bug.cgi?id=1306825#c14 . But fails when we directly upgrade from 3.5 el6 version to 3.6 el7 version.
(In reply to nijin ashok from comment #3) > > As earlier mentioned, it works when we upgrade from 3.5 el6 version to the > 3.5 el7 version as tested in > https://bugzilla.redhat.com/show_bug.cgi?id=1306825#c14 . But fails when we > directly upgrade from 3.5 el6 version to 3.6 el7 version. So, closing as not a bug since the direct upgrade from 3.5 el6 to 3.6 el7 is not supported. As reported, 3.5 el6 to 3.5 el7 and then upgrade to el7 is the proper way to perform the upgrade.
(In reply to Sandro Bonazzola from comment #4) > (In reply to nijin ashok from comment #3) > > > > > As earlier mentioned, it works when we upgrade from 3.5 el6 version to the > > 3.5 el7 version as tested in > > https://bugzilla.redhat.com/show_bug.cgi?id=1306825#c14 . But fails when we > > directly upgrade from 3.5 el6 version to 3.6 el7 version. > > So, closing as not a bug since the direct upgrade from 3.5 el6 to 3.6 el7 is > not supported. > As reported, 3.5 el6 to 3.5 el7 and then upgrade to el7 is the proper way to > perform the upgrade. So what will be the case of RHEL host? How we can restrict the vdsm version to the one which is released with 3.5? Also can you please check this doc bug https://bugzilla.redhat.com/show_bug.cgi?id=1328382 .
(In reply to nijin ashok from comment #5) > So what will be the case of RHEL host? Not sure to have understood the question. Hosted engine is supported in 3.5 on both EL6 and EL7 so the upgrade path is still: 3.5 EL6 -> 3.5 EL7 -> 3.6 EL7 > How we can restrict the vdsm version > to the one which is released with 3.5? when installing 3.5 el7 just don't subscribe to the 3.6 channel, that should allow to install clean 3.5 el7. Once it's done, enable 3.6 channel and perform the upgrade. > Also can you please check this doc > bug https://bugzilla.redhat.com/show_bug.cgi?id=1328382 . Done.
(In reply to Sandro Bonazzola from comment #6) > (In reply to nijin ashok from comment #5) > when installing 3.5 el7 just don't subscribe to the 3.6 channel, that should > allow to install clean 3.5 el7. Once it's done, enable 3.6 channel and > perform the upgrade. Got it. But in RHEL 7, I think we don't have any version specific channel for hosted-engine packages. For both 3.5 and 3.6, it comes from rhel-7-server-rhev-mgmt-agent-rpms ovirt-hosted-engine-ha-1.2.4-5.el7ev.noarch : RHEV-M Hosted Engine High Availability Manager Repo : rhel-7-server-rhev-mgmt-agent-rpms ovirt-hosted-engine-ha-1.3.5.3-1.el7ev.noarch : RHEV-M Hosted Engine High Availability Manager Repo : @rhel-7-server-rhev-mgmt-agent-rpms 3.5 Doc : https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html-single/Installation_Guide/index.html#Subscribing_to_the_Required_Entitlements 3.6 Doc : https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.6/html-single/Self-Hosted_Engine_Guide/index.html#Subscribing_to_the_Required_Entitlements > > Also can you please check this doc > > bug https://bugzilla.redhat.com/show_bug.cgi?id=1328382 . > > Done. Thanks!
Ok, understood the issue. See also bug #1337641 and bug #1328382 with regards to the upgrade procedure for 3.5 -> 3.6 with Hosted Engine. Closing this as duplicate of bug #1328382 *** This bug has been marked as a duplicate of bug 1328382 ***