Bug 840300 - 3.1 beta3 - [vdsm][bootstrap]: we do not fail installation when failing to create rhevm network
3.1 beta3 - [vdsm][bootstrap]: we do not fail installation when failing to cr...
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: vdsm (Show other bugs)
x86_64 Linux
medium Severity medium
: beta
: ---
Assigned To: Douglas Schilling Landgraf
infra network
Depends On:
  Show dependency treegraph
Reported: 2012-07-15 10:50 EDT by Dafna Ron
Modified: 2012-12-04 14:02 EST (History)
16 users (show)

See Also:
Fixed In Version: vdsm-4.9.6-38.0
Doc Type: Bug Fix
Doc Text:
Previously, if the libvirt daemon crashed during Red Hat Enterprise Virtualization host setup, the installation would continue regardless. This resulted in a failure to create a bridge to the Red Hat Enterprise Virtualization Manager, making the host unresponsive and unusable. Now, a condition has been set so if the libvirt daemon crashes, the installation fails and can be started again properly.
Story Points: ---
Clone Of:
Last Closed: 2012-12-04 14:02:42 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
logs (25.29 KB, application/x-gzip)
2012-07-15 10:50 EDT, Dafna Ron
no flags Details
vdsm bootstrap log (78.92 KB, application/octet-stream)
2012-10-02 10:03 EDT, GenadiC
no flags Details

External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2012:1508 normal SHIPPED_LIVE Important: rhev-3.1.0 vdsm security, bug fix, and enhancement update 2012-12-04 18:48:05 EST

  None (edit)
Description Dafna Ron 2012-07-15 10:50:00 EDT
Created attachment 598311 [details]

Description of problem:

libvirt creashed right after it was started (probably because of dbus). 
addNetwork failed and we continues installation without having rhevm network. 
host becomes non-operational after reboot.

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:
1. stop dbus before installing host
Actual results:

libvirt will crash after it starts. we will not fail host install and continue without configuring rhevm interface and host will become non-operational after reboot

Expected results:

if we fail to configure rhevm interface we should fail host install. 

Additional info: full logs attached 

Sun, 15 Jul 2012 15:41:16 DEBUG    Starting libvirtd daemon: [  OK  ]

un, 15 Jul 2012 15:41:16 DEBUG    libvir: RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
Traceback (most recent call last):
  File "/usr/lib64/python2.6/runpy.py", line 122, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.6/runpy.py", line 34, in _run_code
    exec code in run_globals
  File "/usr/share/vdsm/configNetwork.py", line 1202, in <module>
  File "/usr/share/vdsm/configNetwork.py", line 1171, in main
    addNetwork(bridge, **kwargs)
  File "/usr/share/vdsm/configNetwork.py", line 582, in addNetwork
    _netinfo = NetInfo()
  File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 307, in __init__
    _netinfo = get()
  File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 239, in get
    nets = networks()
  File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 70, in networks
    conn = libvirtconnection.get()
  File "/usr/lib64/python2.6/site-packages/vdsm/libvirtconnection.py", line 113, in get
Comment 2 Barak 2012-07-15 11:17:06 EDT
This is a bootstrapping issue, moved to VDSM
Comment 3 Douglas Schilling Landgraf 2012-08-03 16:27:06 EDT
Patch upstream available:
Comment 5 Alon Bar-Lev 2012-08-28 03:38:42 EDT

I agree that current solution would to fail the bootstrap process. Later on we can discuss improvements to the process.

As I discussed previously with dan, the use of libvirt during bootstrap is something that I personally dislike. It is caused by the need to use its persisted database to store the bridge. While libvirt is actually our slave and vdsm is the master, so any configuration should be persisted by vdsm and pushed to libvirt upon vdsm startup.

The fact that libvirt has capabilities to store information, should not imply that it is a database for vdsm, the fact that it is implements persist data store, does not imply that our design should depend on this. If I would design a pure box diagram of components, I would probably put libvirt as a stateless components [although it may store state, but this state gets overridden each startup].

In the above case, all we need to do is to mark the existence of this bridge, then when vdsm will start it will configure the libvirt properly. So no failure during bootstrap.

Comment 13 GenadiC 2012-10-02 10:03:53 EDT
Created attachment 620284 [details]
vdsm bootstrap log
Comment 14 GenadiC 2012-10-02 10:05:14 EDT
The installation finished with success (although the rhevm bridge was not created) 
Expected result: The installation should fail. The log is attached
Comment 15 Douglas Schilling Landgraf 2012-10-03 21:26:53 EDT
Patch upstream available for review:
Comment 16 Douglas Schilling Landgraf 2012-10-04 10:01:41 EDT
Downstream version:
Comment 18 GenadiC 2012-11-11 10:14:58 EST
Verified in SI24 (vdsm-4.9.6-41.0.el6_3.x86_64)
In order to verify we changed the ifup to non executable.
Comment 20 errata-xmlrpc 2012-12-04 14:02:42 EST
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.