Bug 1542824

Summary: Undercloud Install fails because openvswitch fails to start
Product: Red Hat OpenStack Reporter: rchugh1
Component: openvswitchAssignee: Open vSwitch development team <ovs-team>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Ofer Blaut <oblaut>
Severity: high Docs Contact:
Priority: medium    
Version: 10.0 (Newton)CC: aconole, amuller, apevec, aschultz, bdobreli, chrisw, emacchi, jlibosva, mburns, rchugh1, rhel-osp-director-maint, rhos-maint, srevivo
Target Milestone: ---Keywords: ZStream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-04-26 18:31:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1548677    
Bug Blocks:    
Attachments:
Description Flags
Sos report for undercloud failure none

Description rchugh1 2018-02-07 05:59:35 UTC
Description of problem:

Undercloud install fails with the error Could not find a suitable provider for mysql_user, mysql_db, etc.

Version-Release number of selected component (if applicable): RHOSP 10


How reproducible:

Get all the RHOSP 10 packages, setup undercloud and during install of undercloud, we get the aforementioned error.

Steps to Reproduce:
1.
2.
3.

Actual results:

Undercloud setup fails

Expected results:

Undercloud setup is successful


Additional info:


Logs of the error

2018-02-07 00:42:14 - Warning: /Firewall[999 drop all]: Skipping because o
f failed dependencies
2018-02-07 00:42:14 - Error: Could not find a suitable provider for mysql_
datadir
2018-02-07 00:42:14 - Error: Could not find a suitable provider for mysql_
database
2018-02-07 00:42:14 - Error: Could not find a suitable provider for mysql_
user
2018-02-07 00:42:14 - Notice: Finished catalog run in 13.57 seconds
+ rc=4
+ set -e
+ echo 'puppet apply exited with exit code 4'
puppet apply exited with exit code 4
+ '[' 4 '!=' 2 -a 4 '!=' 0 ']'
+ exit 4
[2018-02-07 00:42:27,493] (os-refresh-config) [ERROR] during configure pha
se. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.
d']' returned non-zero exit status 1]

[2018-02-07 00:42:27,493] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py"
, line 1256, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py"
, line 1020, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py"
, line 519, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1

Comment 1 Emilien Macchi 2018-02-07 21:33:25 UTC
Please upload sosreport from the undercloud node, you probably don't have mysql installed which is required on the undercloud.

Comment 2 rchugh1 2018-02-07 21:35:47 UTC
Emilien,

I am trying on another VM on RHHI. Hopefully I did something wrong the first time. I will report back if it doesn't work with all the requisite logs.

Thanks
Rushil

Comment 3 rchugh1 2018-02-08 06:11:47 UTC
Created attachment 1393028 [details]
Sos report for undercloud failure

Comment 4 rchugh1 2018-02-08 06:13:49 UTC
Sos report has been attached with this email and I am still seeing the same failure.

Thanks
Rushil

Comment 5 Bogdan Dobrelya 2018-02-13 13:26:40 UTC
Looking into the sosreport's /var/log/messages, it seems the failure of ovsdb-server.service could be the root cause

Comment 6 Bogdan Dobrelya 2018-02-13 13:33:49 UTC
There is also no logs around '2018-02-07 00:42' in the sosreport

Comment 7 Alex Schultz 2018-03-20 21:56:51 UTC
It appears to be failing because openvswitch service is failing to start.


Feb  8 01:01:13 directorb1 systemd: Starting Open vSwitch Database Unit...
Feb  8 01:01:13 directorb1 ovs-ctl: runuser: System error
Feb  8 01:01:13 directorb1 ovs-ctl: /etc/openvswitch/conf.db does not exist ... (warning).
Feb  8 01:01:13 directorb1 ovs-ctl: Creating empty database /etc/openvswitch/conf.db runuser: System error
Feb  8 01:01:13 directorb1 ovs-ctl: [FAILED]


From the audit.log it appears to be an selinux related issue with openvswitch

type=AVC msg=audit(1518069673.992:528): avc:  denied  { create } for  pid=17165 comm="runuser" scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:system_r:openvswitch_t:s0 tclass=netlink_audit_socket
type=SYSCALL msg=audit(1518069673.992:528): arch=c000003e syscall=41 success=no exit=-13 a0=10 a1=3 a2=9 a3=7ffcaf73d3a0 items=0 ppid=17164 pid=17165 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="runuser" exe="/usr/sbin/runuser" subj=system_u:system_r:openvswitch_t:s0 key=(null)


I don't think openstack-selinux manages the openvswitch selinux permissions because we don't even get to that part in the deployment because openvswitch fails to start.  Moving over to the folks who know more about openvswitch for their review.

Comment 8 Aaron Conole 2018-03-21 13:24:40 UTC
selinux-policy-3.13.1-166.el7_4.7.noarch is installed.

Please upgrade to selinux-policy-3.13.1-166.el7_4.9.noarch and retry.

We have bz 1548677 which resolves this in a way to force the package upgrade.

Comment 9 Jakub Libosvar 2018-04-11 13:44:25 UTC
Do you have any updates from the request from the comment 8?

Comment 10 Assaf Muller 2018-04-26 18:31:12 UTC
Please re-open if the bug is still relevant.

Comment 11 Red Hat Bugzilla 2023-09-14 04:16:22 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days