Bug 1294517 - Unable to start hosted-engine VM due to internal error
Summary: Unable to start hosted-engine VM due to internal error
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-node
Classification: oVirt
Component: General
Version: 3.6
Hardware: Unspecified
OS: Linux
urgent
urgent
Target Milestone: ovirt-3.6.2
: 3.6
Assignee: Douglas Schilling Landgraf
QA Contact: cshao
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-28 16:56 UTC by Charlie Inglese
Modified: 2016-03-11 07:21 UTC (History)
5 users (show)

Fixed In Version: ovirt-node-3.6.1-3
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-03-11 07:21:50 UTC
oVirt Team: Node
Embargoed:
rule-engine: ovirt-3.6.z+
rule-engine: blocker+
bmcclain: planning_ack+
fdeutsch: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
vdsClient_list (3.69 KB, text/plain)
2015-12-28 16:56 UTC, Charlie Inglese
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 51452 0 master MERGED semodule: update ebtables module 2016-01-08 14:32:53 UTC
oVirt gerrit 51774 0 ovirt-3.6 MERGED semodule: update ebtables module 2016-01-13 10:43:29 UTC

Description Charlie Inglese 2015-12-28 16:56:37 UTC
Created attachment 1110045 [details]
vdsClient_list

Description of problem:
Upon installation of oVirt hosted engine appliance, the VM fails to restart after the initial install and shutdown.

Performing a "vdsClient -s 0 list" produces the following error: 
exitMessage = internal error: Failed to apply firewall rules /usr/sbin/ebtables --concurrent -t nat -N libvirt-J-vnet0: Unable to create lock file /var/lib/ebtables/lock.

Version-Release number of selected component (if applicable):
ovirt-vmconsole-1.0.0-1.el7.centos.noarch
ovirt-setup-lib-1.0.0-1.el7.centos.noarch
ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch
ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch
ovirt-hosted-engine-setup-1.3.0-1.el7.centos.noarch
libgovirt-0.3.3-1.el7.x86_64
ovirt-hosted-engine-ha-1.3.2.1-1.el7.centos.noarch
ovirt-engine-appliance-20151104.0-1.el7.centos.noarch
ovirt-iso-uploader-3.6.0-1.el7.centos.noarch
ovirt-host-deploy-1.4.0-1.el7.centos.noarch


How reproducible:
Everytime

Steps to Reproduce:
1. Install oVirt hosted engine
2. Wait for oVirt hosted engine VM to power-down
3. vdsClient -s 0 list

Actual results:


Expected results:
oVirt hosted engine VM should restart and bring up the oVirt engine and associated services.

Additional info:

Permissions of /var/lib/ebtables (ls -la /var/lib/ebtables):
drwx------.  2 root root    6 Dec 28 16:52 .
drwxr-xr-x. 46 root root 4096 Dec 28 16:19 ..

Comment 1 Fabian Deutsch 2016-01-04 13:32:38 UTC
This can rather be a persistence issue related to node, not the appliance, but this needs further clarification.

Comment 2 Charlie Inglese 2016-01-04 14:29:45 UTC
I believe this is an artifact of SELinux and should be closed as NOT A BUG.

Comment 3 Fabian Deutsch 2016-01-04 14:56:38 UTC
Good hint.

Can you please run 

$ audit2allow -a

Maybe there were some logged denials

Comment 4 Charlie Inglese 2016-01-06 15:51:42 UTC
Fabian,

I ran audit2allow on audit.log for ebtables, and ended up creating an SELinux module for it. Here's the content of the module that audit2allow created:

module ebtables 1.0;

require {
        type iptables_t;
        type var_lib_t;
        class dir { write remove_name create add_name };
        class file { write create unlink open };
}

#============= iptables_t ==============
allow iptables_t var_lib_t:dir { write remove_name create add_name };
allow iptables_t var_lib_t:file { write create unlink open };

Comment 5 Fabian Deutsch 2016-01-06 16:05:00 UTC
Thanks Charlie.

We will include it in our policy.

Comment 6 Fabian Deutsch 2016-01-06 16:55:36 UTC
Chen, have you seen this in your testing?

Comment 7 cshao 2016-01-07 03:35:37 UTC
Hi fabian,

We didn't met this issue with latest RHEV-H 7.2 for RHEV 3.6.2 (rhev-hypervisor7-7.2-20151229.0 build.


Test version:
rhev-hypervisor7-7.2-20151229.0
rhevm-appliance-20151216.0-1.3.6.ova

Due to met Bug 1294783 - Failed to setup engine via rhevm-appliance.ova, so we did testing with the old ova (rhevm-appliance-20151216.0-1.3.6.ova).

Detail testing info please refer test report in VIRT-QE mail list:
Summary: RHEV-H 7.2 for RHEV 3.6.2 (rhev-hypervisor7-7.2-20151229.0) - Acceptance Testing - Partial Fail

Thanks!

Comment 12 Red Hat Bugzilla Rules Engine 2016-01-14 16:47:10 UTC
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.

Comment 13 cshao 2016-02-24 12:46:42 UTC
Test version:
rhev-hypervisor7-7.2-20160222.0
ovirt-node-3.6.1-7.0.el7ev.noarch
ovirt-node-plugin-hosted-engine-0.3.0-7.el7ev.noarch
ovirt-node-plugin-vdsm-0.6.1-7.el7ev.noarch
ovirt-hosted-engine-setup-1.3.3.3-1.el7
ovirt-hosted-engine-ha-1.3.4.1-1.el7
rhev-m appliance:20160212.0-1.3.6.ova  

Test steps:
1. TUI clean install rhevh
2. Login rhevh, setup network via dhcp.
3. Switch to HE menu.
4. Finish HE configure.
5. Wait for oVirt hosted engine VM to power-down

Test result:
oVirt hosted engine VM can restart and bring up the oVirt engine and associated services.

So the bug is fixed, change bug status to VERIFIED.


Note You need to log in before you can comment on or make changes to this bug.