Bug 1566373

Summary: Engine setup fails to start service 'openvswitch'
Product: [oVirt] ovirt-engine Reporter: Radim Hrazdil <rhrazdil>
Component: Setup.EngineAssignee: Marcin Mirecki <mmirecki>
Status: CLOSED DUPLICATE QA Contact: Pavel Stehlik <pstehlik>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 4.2.3CC: bugs, charlieott, danken, jbelka, manish2479, myakove, rhrazdil, ylavi
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-21 13:29:56 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
journalctl -xe none

Description Radim Hrazdil 2018-04-12 07:55:00 UTC
Description of problem:
When installing latest ovirt-engine on CentOS 7.4, running engine-setup fails on [ ERROR ] Failed to execute stage 'Misc configuration': Failed to start service 'openvswitch'.


Version-Release number of selected component (if applicable):
4.3.0-0.0.master.20180411184915.git615b419.el7.centos 
CentOS Linux release 7.4.1708

How reproducible:
100%

Steps to Reproduce:
1. yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm
2. engine-setup, use suggested settings
3.

Actual results:
Failed to start service 'openvswitch'.

Expected results:
Execution of setup should succeed

Additional info:

Comment 2 Meni Yakove 2018-04-16 08:39:29 UTC
Which dependency job failed and which version of openvswitch do you have.

"""
A dependency job for openvswitch.service failed. See 'journalctl -xe' for details.
"""

please attach `journalctl -xe` as well.

Comment 3 Radim Hrazdil 2018-04-16 10:15:38 UTC
Created attachment 1422397 [details]
journalctl -xe

Comment 4 Dan Kenigsberg 2018-04-17 11:44:36 UTC
Again, which version of openvswitch do you have?
which version of selinux-policy-targeted ?

We might be seeing bug 1482682 again.

Comment 5 Radim Hrazdil 2018-04-17 12:03:50 UTC
Sorry, I missed that.

openvswitch and ovn:
ovirt-provider-ovn-1.2.11-0.20180409150639.git36de3ab.el7.centos.noarch
openvswitch-ovn-common-2.9.0-3.el7.x86_64
openvswitch-ovn-central-2.9.0-3.el7.x86_64

selinux-policy-targeted:
selinux-policy-targeted-3.13.1-166.el7.noarch

Comment 6 Jiri Belka 2018-04-19 14:02:06 UTC
I got same issue, it's openvswitch dependency - ovsdb-server - which fails and thus makes openvswitch fail in the end too:

Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: Starting Open vSwitch Database Unit...
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com ovs-ctl[30712]: runuser: System error
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com ovs-ctl[30712]: /etc/openvswitch/conf.db does not exist ... (warning).
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com ovs-ctl[30712]: Creating empty database /etc/openvswitch/conf.db runuser: System error
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com ovs-ctl[30712]: [FAILED]
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: ovsdb-server.service: control process exited, code=exited status=1
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: Failed to start Open vSwitch Database Unit.
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: Unit ovsdb-server.service entered failed state.
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: ovsdb-server.service failed.
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: ovsdb-server.service holdoff time over, scheduling restart.
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: start request repeated too quickly for ovsdb-server.service
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: Failed to start Open vSwitch Database Unit.
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: Unit ovsdb-server.service entered failed state.
Apr 19 15:45:44 10-37-137-41.rhev.lab.eng.brq.redhat.com systemd[1]: ovsdb-server.service failed.

openvswitch got installed in step from ovirt-engine-4.1.10.3-0.1.el7.noarch to ovirt-engine-4.2.3.2-0.1.el7.noarch:

# yum history info <id>
...
Command Line   : update ovirt*setup*
Transaction performed with:
...

and I see in yum transaction:

...
Scriptlet output:
   1 warning: user openvswitch does not exist - using root
   2 warning: group openvswitch does not exist - using root
   3 warning: user openvswitch does not exist - using root
   4 warning: group openvswitch does not exist - using root
   5 warning: user openvswitch does not exist - using root
   6 warning: group openvswitch does not exist - using root
...

ovs rpms versions:

Apr 19 14:26:39 Installed: openvswitch-2.9.0-1.el7fdp.x86_64
Apr 19 14:26:39 Installed: openvswitch-ovn-common-2.9.0-1.el7fdp.x86_64
Apr 19 14:26:40 Installed: openvswitch-ovn-central-2.9.0-1.el7fdp.x86_64
Apr 19 14:26:40 Installed: python-openvswitch-2.9.0-1.el7fdp.noarch

Comment 7 Dan Kenigsberg 2018-04-19 19:25:01 UTC
(In reply to Radim Hrazdil from comment #5)
> Sorry, I missed that.
> 
> openvswitch and ovn:
> ovirt-provider-ovn-1.2.11-0.20180409150639.git36de3ab.el7.centos.noarch
> openvswitch-ovn-common-2.9.0-3.el7.x86_64
> openvswitch-ovn-central-2.9.0-3.el7.x86_64
> 
> selinux-policy-targeted:
> selinux-policy-targeted-3.13.1-166.el7.noarch

yes, this seems like bug 1482682. please update selinux-policy-targeted.

If I'm utterly mistaken, please reopen this bug on ovsdb, and attach ovsdb logs (as well as audit.log).

*** This bug has been marked as a duplicate of bug 1482682 ***

Comment 8 Jiri Belka 2018-04-20 08:10:39 UTC
Usual update flow is:

1. yum update ovirt\*setup\*
2. engine-setup
3. yum update

Thus when on 4.1.x and EL 7.4 I won't have selinux-policy-targeted.noarch from 7.5 yet!

IMO you should require this version as dependency or stricly document it - this is not the case.

This is difference between 7.4 and 7.5's selinux-policy-targeted update:

Dependencies Resolved

=============================================================================================================================================================================================================================================
 Package                                                         Arch                                           Version                                                     Repository                                                  Size
=============================================================================================================================================================================================================================================
Updating:
 selinux-policy-targeted                                         noarch                                         3.13.1-192.el7_5.3                                          rhel-7-server-rpms                                         6.6 M
Updating for dependencies:
 libselinux                                                      x86_64                                         2.5-12.el7                                                  rhel-7-server-rpms                                         162 k
 libselinux-python                                               x86_64                                         2.5-12.el7                                                  rhel-7-server-rpms                                         235 k
 libselinux-utils                                                x86_64                                         2.5-12.el7                                                  rhel-7-server-rpms                                         151 k
 libsemanage                                                     x86_64                                         2.5-11.el7                                                  rhel-7-server-rpms                                         150 k
 libsemanage-python                                              x86_64                                         2.5-11.el7                                                  rhel-7-server-rpms                                         112 k
 libsepol                                                        x86_64                                         2.5-8.1.el7                                                 rhel-7-server-rpms                                         297 k
 policycoreutils                                                 x86_64                                         2.5-22.el7                                                  rhel-7-server-rpms                                         867 k
 policycoreutils-python                                          x86_64                                         2.5-22.el7                                                  rhel-7-server-rpms                                         454 k
 selinux-policy                                                  noarch                                         3.13.1-192.el7_5.3                                          rhel-7-server-rpms                                         453 k
 setools-libs                                                    x86_64                                         3.3.8-2.el7                                                 rhel-7-server-rpms                                         619 k

Transaction Summary
=============================================================================================================================================================================================================================================
Upgrade  1 Package (+10 Dependent packages)

Comment 9 Dan Kenigsberg 2018-04-21 13:29:56 UTC
rhel-7.4.z should ship selinux-policy-targeted >= 3.13.1-166.el7_4.9 that solved the problem, and ovs must conflict with anything newer.

I should have been more precise with the bug number I use as a dup (there are tons of them)

*** This bug has been marked as a duplicate of bug 1549673 ***

Comment 10 Manish 2018-05-25 14:42:00 UTC
I have upgraded selinux-policy-targeted to 3.13.1-166.el7_4.9on RHEL7.4 and still I see the issues. So that doesn't fixes the issue for me.

Comment 11 Charlie 2018-06-20 17:56:10 UTC
Same issue.