Bug 840300

Summary: 3.1 beta3 - [vdsm][bootstrap]: we do not fail installation when failing to create rhevm network
Product: Red Hat Enterprise Linux 6 Reporter: Dafna Ron <dron>
Component: vdsmAssignee: Douglas Schilling Landgraf <dougsland>
Status: CLOSED ERRATA QA Contact: GenadiC <gcheresh>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.3CC: abaron, acathrow, alonbl, bazulay, cpelland, danken, dfediuck, dyasny, iheim, ilvovsky, jbiddle, lpeer, mavital, Rhev-m-bugs, sgrinber, ykaul
Target Milestone: beta   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: infra network
Fixed In Version: vdsm-4.9.6-38.0 Doc Type: Bug Fix
Doc Text:
Previously, if the libvirt daemon crashed during Red Hat Enterprise Virtualization host setup, the installation would continue regardless. This resulted in a failure to create a bridge to the Red Hat Enterprise Virtualization Manager, making the host unresponsive and unusable. Now, a condition has been set so if the libvirt daemon crashes, the installation fails and can be started again properly.
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-12-04 19:02:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
logs
none
vdsm bootstrap log none

Description Dafna Ron 2012-07-15 14:50:00 UTC
Created attachment 598311 [details]
logs

Description of problem:

libvirt creashed right after it was started (probably because of dbus). 
addNetwork failed and we continues installation without having rhevm network. 
host becomes non-operational after reboot.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. stop dbus before installing host
2.
3.
  
Actual results:

libvirt will crash after it starts. we will not fail host install and continue without configuring rhevm interface and host will become non-operational after reboot

Expected results:

if we fail to configure rhevm interface we should fail host install. 


Additional info: full logs attached 

Sun, 15 Jul 2012 15:41:16 DEBUG    Starting libvirtd daemon: [  OK  ]

un, 15 Jul 2012 15:41:16 DEBUG    libvir: RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/share/vdsm/configNetwork.py", line 1202, in <module>
    main()
  File "/usr/share/vdsm/configNetwork.py", line 1171, in main
    addNetwork(bridge, **kwargs)
  File "/usr/share/vdsm/configNetwork.py", line 582, in addNetwork
    _netinfo = NetInfo()
  File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 307, in __init__
    _netinfo = get()
  File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 239, in get
    nets = networks()
  File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 70, in networks
    conn = libvirtconnection.get()
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 113, in get

Comment 2 Barak 2012-07-15 15:17:06 UTC
This is a bootstrapping issue, moved to VDSM

Comment 3 Douglas Schilling Landgraf 2012-08-03 20:27:06 UTC
Patch upstream available:
http://gerrit.ovirt.org/6875

Comment 5 Alon Bar-Lev 2012-08-28 07:38:42 UTC
Hello,

I agree that current solution would to fail the bootstrap process. Later on we can discuss improvements to the process.

As I discussed previously with dan, the use of libvirt during bootstrap is something that I personally dislike. It is caused by the need to use its persisted database to store the bridge. While libvirt is actually our slave and vdsm is the master, so any configuration should be persisted by vdsm and pushed to libvirt upon vdsm startup.

The fact that libvirt has capabilities to store information, should not imply that it is a database for vdsm, the fact that it is implements persist data store, does not imply that our design should depend on this. If I would design a pure box diagram of components, I would probably put libvirt as a stateless components [although it may store state, but this state gets overridden each startup].

In the above case, all we need to do is to mark the existence of this bridge, then when vdsm will start it will configure the libvirt properly. So no failure during bootstrap.

Alon.

Comment 13 GenadiC 2012-10-02 14:03:53 UTC
Created attachment 620284 [details]
vdsm bootstrap log

Comment 14 GenadiC 2012-10-02 14:05:14 UTC
The installation finished with success (although the rhevm bridge was not created) 
Expected result: The installation should fail. The log is attached

Comment 15 Douglas Schilling Landgraf 2012-10-04 01:26:53 UTC
Patch upstream available for review:
http://gerrit.ovirt.org/#/c/8345/

Comment 16 Douglas Schilling Landgraf 2012-10-04 14:01:41 UTC
Downstream version:
https://gerrit.eng.lab.tlv.redhat.com/#/c/2480/

Comment 18 GenadiC 2012-11-11 15:14:58 UTC
Verified in SI24 (vdsm-4.9.6-41.0.el6_3.x86_64)
In order to verify we changed the ifup to non executable.

Comment 20 errata-xmlrpc 2012-12-04 19:02:42 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-1508.html