Bug 1294517 - Unable to start hosted-engine VM due to internal error
Unable to start hosted-engine VM due to internal error
Status: CLOSED CURRENTRELEASE
Product: ovirt-node
Classification: oVirt
Component: General (Show other bugs)
3.6
Unspecified Linux
urgent Severity urgent (vote)
: ovirt-3.6.2
: 3.6
Assigned To: Douglas Schilling Landgraf
cshao
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-28 11:56 EST by Charlie Inglese
Modified: 2016-03-11 02:21 EST (History)
5 users (show)

See Also:
Fixed In Version: ovirt-node-3.6.1-3
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-11 02:21:50 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Node
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
rule-engine: ovirt‑3.6.z+
rule-engine: blocker+
bmcclain: planning_ack+
fdeutsch: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
vdsClient_list (3.69 KB, text/plain)
2015-12-28 11:56 EST, Charlie Inglese
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 51452 master MERGED semodule: update ebtables module 2016-01-08 09:32 EST
oVirt gerrit 51774 ovirt-3.6 MERGED semodule: update ebtables module 2016-01-13 05:43 EST

  None (edit)
Description Charlie Inglese 2015-12-28 11:56:37 EST
Created attachment 1110045 [details]
vdsClient_list

Description of problem:
Upon installation of oVirt hosted engine appliance, the VM fails to restart after the initial install and shutdown.

Performing a "vdsClient -s 0 list" produces the following error: 
exitMessage = internal error: Failed to apply firewall rules /usr/sbin/ebtables --concurrent -t nat -N libvirt-J-vnet0: Unable to create lock file /var/lib/ebtables/lock.

Version-Release number of selected component (if applicable):
ovirt-vmconsole-1.0.0-1.el7.centos.noarch
ovirt-setup-lib-1.0.0-1.el7.centos.noarch
ovirt-engine-sdk-python-3.6.0.3-1.el7.centos.noarch
ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch
ovirt-hosted-engine-setup-1.3.0-1.el7.centos.noarch
libgovirt-0.3.3-1.el7.x86_64
ovirt-hosted-engine-ha-1.3.2.1-1.el7.centos.noarch
ovirt-engine-appliance-20151104.0-1.el7.centos.noarch
ovirt-iso-uploader-3.6.0-1.el7.centos.noarch
ovirt-host-deploy-1.4.0-1.el7.centos.noarch


How reproducible:
Everytime

Steps to Reproduce:
1. Install oVirt hosted engine
2. Wait for oVirt hosted engine VM to power-down
3. vdsClient -s 0 list

Actual results:


Expected results:
oVirt hosted engine VM should restart and bring up the oVirt engine and associated services.

Additional info:

Permissions of /var/lib/ebtables (ls -la /var/lib/ebtables):
drwx------.  2 root root    6 Dec 28 16:52 .
drwxr-xr-x. 46 root root 4096 Dec 28 16:19 ..
Comment 1 Fabian Deutsch 2016-01-04 08:32:38 EST
This can rather be a persistence issue related to node, not the appliance, but this needs further clarification.
Comment 2 Charlie Inglese 2016-01-04 09:29:45 EST
I believe this is an artifact of SELinux and should be closed as NOT A BUG.
Comment 3 Fabian Deutsch 2016-01-04 09:56:38 EST
Good hint.

Can you please run 

$ audit2allow -a

Maybe there were some logged denials
Comment 4 Charlie Inglese 2016-01-06 10:51:42 EST
Fabian,

I ran audit2allow on audit.log for ebtables, and ended up creating an SELinux module for it. Here's the content of the module that audit2allow created:

module ebtables 1.0;

require {
        type iptables_t;
        type var_lib_t;
        class dir { write remove_name create add_name };
        class file { write create unlink open };
}

#============= iptables_t ==============
allow iptables_t var_lib_t:dir { write remove_name create add_name };
allow iptables_t var_lib_t:file { write create unlink open };
Comment 5 Fabian Deutsch 2016-01-06 11:05:00 EST
Thanks Charlie.

We will include it in our policy.
Comment 6 Fabian Deutsch 2016-01-06 11:55:36 EST
Chen, have you seen this in your testing?
Comment 7 cshao 2016-01-06 22:35:37 EST
Hi fabian,

We didn't met this issue with latest RHEV-H 7.2 for RHEV 3.6.2 (rhev-hypervisor7-7.2-20151229.0 build.


Test version:
rhev-hypervisor7-7.2-20151229.0
rhevm-appliance-20151216.0-1.3.6.ova

Due to met Bug 1294783 - Failed to setup engine via rhevm-appliance.ova, so we did testing with the old ova (rhevm-appliance-20151216.0-1.3.6.ova).

Detail testing info please refer test report in VIRT-QE mail list:
Summary: RHEV-H 7.2 for RHEV 3.6.2 (rhev-hypervisor7-7.2-20151229.0) - Acceptance Testing - Partial Fail

Thanks!
Comment 12 Red Hat Bugzilla Rules Engine 2016-01-14 11:47:10 EST
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.
Comment 13 cshao 2016-02-24 07:46:42 EST
Test version:
rhev-hypervisor7-7.2-20160222.0
ovirt-node-3.6.1-7.0.el7ev.noarch
ovirt-node-plugin-hosted-engine-0.3.0-7.el7ev.noarch
ovirt-node-plugin-vdsm-0.6.1-7.el7ev.noarch
ovirt-hosted-engine-setup-1.3.3.3-1.el7
ovirt-hosted-engine-ha-1.3.4.1-1.el7
rhev-m appliance:20160212.0-1.3.6.ova  

Test steps:
1. TUI clean install rhevh
2. Login rhevh, setup network via dhcp.
3. Switch to HE menu.
4. Finish HE configure.
5. Wait for oVirt hosted engine VM to power-down

Test result:
oVirt hosted engine VM can restart and bring up the oVirt engine and associated services.

So the bug is fixed, change bug status to VERIFIED.

Note You need to log in before you can comment on or make changes to this bug.