Bug 1151339 - Lost all the configuration for hosted engine after reboot rhevh
Summary: Lost all the configuration for hosted engine after reboot rhevh
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-hosted-engine-setup
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.5.0
Assignee: Sandro Bonazzola
QA Contact: meital avital
URL:
Whiteboard: integration
Depends On:
Blocks: rhevh-7.0
TreeView+ depends on / blocked
 
Reported: 2014-10-10 08:08 UTC by wanghui
Modified: 2016-07-04 00:39 UTC (History)
18 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-11 20:41:36 UTC
oVirt Team: ---
Target Upstream Version:
Embargoed:
mavital: needinfo+
mavital: needinfo+


Attachments (Terms of Use)
hosted-engine log files (3.72 MB, application/x-gzip)
2014-10-10 08:08 UTC, wanghui
no flags Details
/config/files (694 bytes, text/plain)
2014-10-10 08:09 UTC, wanghui
no flags Details
before reboot /var/lib/vdsm/persistence/* shows (198 bytes, application/x-gzip)
2014-10-13 05:30 UTC, wanghui
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2015:0161 0 normal SHIPPED_LIVE ovirt-hosted-engine-setup bug fix and enhancement update 2015-12-07 21:35:11 UTC
oVirt gerrit 34438 0 master MERGED node: added persistent storage configuration Never
oVirt gerrit 34440 0 ovirt-hosted-engine-setup-1.2 MERGED node: added persistent storage configuration Never

Description wanghui 2014-10-10 08:08:48 UTC
Created attachment 945496 [details]
hosted-engine log files

Description of problem:
It lost all the configuration for hosted engine after reboot rhevh.

Version-Release number of selected component (if applicable):
rhev-hypervisor7-7.0-20141006.0.el7ev
ovirt-node-3.1.0-0.20.20141006gitc421e04.el7.noarch.rpm
ovirt-node-plugin-hosted-engine-0.2.0-2.0.el7.x86_64
ovirt-hosted-engine-setup-1.2.1-1.el7.noarch
ovirt-host-deploy-1.3.0-1.el7.noarch
ovirt-hosted-engine-ha-1.2.2-2.el7.noarch

How reproducible:
100%

Steps to Reproduce:
1. Install rhev-hypervisor7-7.0-20141006.0.el7ev.
2. Configure hosted engine using PXE boot engine vm option.
3. After finish configuration, reboot rhevh.

Actual results:
1. After step3, all the configuration for hosted engine lost. 
2. The network of rhevm is lost also.
   #ifconfig can only list em1 and lo. But #brctl show can list em1 is the member of rhevm.

Expected results:
1. It should keep the configuration for hosted engine. And network should be up.

Additional info:

Comment 1 wanghui 2014-10-10 08:09:47 UTC
Created attachment 945497 [details]
/config/files

Comment 2 Ying Cui 2014-10-10 08:26:03 UTC
with this issue, we can not test rhevh upgrade parts via self-hosted engine.

Comment 3 Fabian Deutsch 2014-10-10 09:31:04 UTC
Toni, could it be that the network configuration is not getting persisted?

Comment 4 Antoni Segura Puimedon 2014-10-10 11:42:57 UTC
I see the call for the networks to be persisted:

    Thread-15::DEBUG::2014-10-09 06:11:33,368::BindingXMLRPC::1132::vds::(wrapper) client [127.0.0.1]::call setupNetworks with ({'rhevm': {'nic': 'em1', 'bootproto': 'dhcp', 'blockingdhcp': True}}, {}, {'connectivityCheck': False}) {}
    Thread-15::DEBUG::2014-10-09 06:11:41,256::BindingXMLRPC::1139::vds::(wrapper) return setupNetworks with {'status': {'message': 'Done', 'code': 0}}
    Detector thread::DEBUG::2014-10-09 06:11:41,265::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 127.0.0.1:42765
    Detector thread::DEBUG::2014-10-09 06:11:41,265::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 127.0.0.1:42765
    Detector thread::DEBUG::2014-10-09 06:11:41,265::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 127.0.0.1:42765
    Detector thread::DEBUG::2014-10-09 06:11:41,265::BindingXMLRPC::1172::XmlDetector::(handleSocket) xml over http detected from ('127.0.0.1', 42765)
    Thread-16::DEBUG::2014-10-09 06:11:41,266::BindingXMLRPC::1132::vds::(wrapper) client [127.0.0.1]::call setSafeNetworkConfig with () {}
    Thread-16::DEBUG::2014-10-09 06:11:42,010::BindingXMLRPC::1139::vds::(wrapper) return setSafeNetworkConfig with {'status': {'message': 'Done', 'code': 0}} <-------------------------------------

In the last vdsCapabilities in vdsm.log the rhevm network is still there.

what's the content of /var/lib/vdsm/persistence/netconf/*/*

Comment 5 wanghui 2014-10-13 05:30:19 UTC
Created attachment 946231 [details]
before reboot /var/lib/vdsm/persistence/* shows

Comment 6 wanghui 2014-10-13 05:34:59 UTC
And after reboot, /var/lib/vdsm/persistence/ shows as follows.
#cd /var/lib/vdsm/persistence

#ll
lrwxrwxrwx. 1 root root 53 Oct 10 10:28 netconf -> /var/lib/vdsm/persistence/netconf.1412936886789521463
drwxr-xr-x. 3 root root 1024 Oct 10 10:28 netconf.1412936886789521463

# cd netconf.1412936886789521463/nets
# cat rhevm
{"nic": "em1", "bootproto": "dhcp", "blockingdhcp": true}

Comment 7 Antoni Segura Puimedon 2014-10-20 08:37:24 UTC
That looks like it is persisted all right. On boot, if vdsmd starts, it will configure a rhevm bridge over em1 with dhcp.

Comment 8 Fabian Deutsch 2014-11-17 13:36:09 UTC
Douglas, can you please help verifying the patch?

Comment 9 Douglas Schilling Landgraf 2014-11-17 20:08:12 UTC
(In reply to Fabian Deutsch from comment #8)
> Douglas, can you please help verifying the patch?

I have verified the patch it worked. I have tested with an additional patch inside the plugin http://gerrit.ovirt.org/#/c/35250/.

Thanks Sandro!

Comment 11 Ying Cui 2014-11-26 01:04:58 UTC
There is no RHEVH 7.0 build for this bug testing in brew till now, so I have to change this bug to MODIFIED, when the build is coming to QE, please modify it back to ON_QA, we will test it asap. Thanks.

Comment 12 Sandro Bonazzola 2014-11-26 07:32:42 UTC
Fabian please move back to ON_QA this one as soon as you have a new RHEV-H 7 build.

Comment 15 wanghui 2014-12-05 03:27:13 UTC
Test version:
rhev-hypervisor7-7.0-20141202.0.iso
ovirt-node-3.1.0-0.28.20141126git25ce016.el7.noarch
ovirt-node-plugin-hosted-engine-0.2.0-5.0.el7ev.x86_64
ovirt-hosted-engine-setup-1.2.1-6.el7ev.noarch
ovirt-hosted-engine-ha-1.2.4-2.el7ev.noarch

Test steps:
1. Install rhev-hypervisor7-7.0-20141202.0.iso.
2. Configure hosted engine using ISO install engine vm option.
3. After finish configuration, reboot rhevh. 

Test result:
1. After step3, all the configuration for hosted-engine existed.
2. After step3, network of rhevm still existed.
3. After step3, the engine status shows "Engine Status: Engine is down. Please check 'hosted-engine --vm-status'".

So this issue probably fixed now. But there is another bug#1151344 which is not fixed yet. So I can not say the engine status is right or not. So I will wait bug#1151344's status to verify this bug.

Thanks!
HuiWang

Comment 16 wanghui 2014-12-05 09:17:57 UTC
Due to bug#1170986, so I can not try on rhev-hypervisor6-6.6-20141202.0.iso this time.

Comment 17 Fabian Deutsch 2014-12-08 08:20:31 UTC
(In reply to wanghui from comment #16)
> Due to bug#1170986, so I can not try on rhev-hypervisor6-6.6-20141202.0.iso
> this time.

Please use rhev-hypervisor6-6.6-20141204.0.iso instead

Comment 19 wanghui 2014-12-11 02:27:17 UTC
(In reply to Fabian Deutsch from comment #17)
> (In reply to wanghui from comment #16)
> > Due to bug#1170986, so I can not try on rhev-hypervisor6-6.6-20141202.0.iso
> > this time.
> 
> Please use rhev-hypervisor6-6.6-20141204.0.iso instead

Hi fabian,

Could you give the download link for rhev-hypervisor6-6.6-20141204.0.iso?

Thanks!
Hui Wang

Comment 22 wanghui 2014-12-18 09:11:54 UTC
Test version:
rhevh version:
rhev-hypervisor6-6.6-20141212.0.iso
ovirt-node-3.1.0-0.34.20141210git0c9c493.el6.noarch
vdsm-4.16.8.1-3.el6ev.x86_64
ovirt-node-plugin-vdsm-0.2.0-14.el6ev.noarch
ovirt-node-plugin-hosted-engine-0.2.0-6.0.el7ev.x86_64
ovirt-hosted-engine-setup-1.2.1-8.el6ev.noarch
ovirt-hosted-engine-ha-1.2.4-3.el6ev.noarch

rhevm version:
rhevm-3.5.0-0.25.el6ev.noarch

Test step:
1. Install rhev-hypervisor6-6.6-20141212.0.iso
2. Configure hosted engine using ISO install engine vm option.
3. Shutdown engine vm When process says "Please shutdown the VM allowing the system to launch it as a monitored service".
4. Check whether engine vm is up or not.
5. Reboot rhevh
6. Check configuration about hosted engine.

Test result:
1. After step6, all the configuration for hosted-engine existed.
2. After step6, network of rhevm still existed.
3. After step6, the engine status shows "Engine Status: Engine is running on 10.66.8.211".
4. After step6, the engine can be access through web and host is still up in engine part.

So this issue is fixed in ovirt-hosted-engine-setup-1.2.1-8.el6ev.noarch now.

Comment 23 Artyom 2014-12-23 13:04:21 UTC
Specify "Fixed In Version" and "Build ID"

Comment 24 wanghui 2014-12-25 06:35:34 UTC
Hi Artyom,

According to my verification, this issue is fixed in ovirt-hosted-engine-setup-1.2.1-8.el6ev.noarch. So If you accept that, you can verified this issue now.

Thanks!
Hui Wang

Comment 25 meital avital 2014-12-25 07:18:02 UTC
Moving to verified according to comment 22

Comment 27 errata-xmlrpc 2015-02-11 20:41:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0161.html


Note You need to log in before you can comment on or make changes to this bug.