Bug 1180085 - ovirt-hosted-engine-setup failed to execute stage 'Misc configuration': Failed to setup networks
Summary: ovirt-hosted-engine-setup failed to execute stage 'Misc configuration': Faile...
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.5.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: ---
Assignee: Nobody
QA Contact: Meni Yakove
URL:
Whiteboard: network
Depends On:
Blocks: 1105479
TreeView+ depends on / blocked
 
Reported: 2015-01-08 10:18 UTC by Petr Matyáš
Modified: 2016-02-10 19:57 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-01-29 11:12:50 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
VDSM log (59.01 KB, text/plain)
2015-01-08 10:18 UTC, Petr Matyáš
no flags Details
supervdsm log (41.08 KB, text/plain)
2015-01-21 13:22 UTC, Petr Matyáš
no flags Details
libvirtd configuration file (13.04 KB, text/plain)
2015-01-22 14:29 UTC, Petr Matyáš
no flags Details
libvirtd log (1.08 KB, text/plain)
2015-01-22 14:29 UTC, Petr Matyáš
no flags Details
new vdsm log (415.27 KB, text/plain)
2015-01-23 11:30 UTC, Petr Matyáš
no flags Details
VDSM log (351.45 KB, text/plain)
2015-01-28 14:26 UTC, Petr Matyáš
no flags Details
supervdsm log (20.80 KB, text/plain)
2015-01-28 14:27 UTC, Petr Matyáš
no flags Details

Description Petr Matyáš 2015-01-08 10:18:14 UTC
Created attachment 977714 [details]
VDSM log

Description of problem:
When setting up ovirt-hosted-engine I get [ ERROR ] Failed to execute stage 'Misc configuration': Failed to setup networks {'rhevm': {'nic': 'eth0', 'netmask': '255.255.252.0', 'bootproto': 'none', 'ipaddr': '10.34.60.233'}}. Error code: "16" message: "Unexpected exception"

Version-Release number of selected component (if applicable):
vt13.6

How reproducible:
always on my setup

Steps to Reproduce:
1. set BOOTPROTO=none and comment out/remove GATEWAY in network config
2. run ovirt-hosted-engine-setup
3. use edited network

Actual results:
error

Expected results:
successful setup

Additional info:

Comment 2 Sandro Bonazzola 2015-01-20 09:46:25 UTC
 Thread-15::DEBUG::2015-01-08 10:26:42,134::BindingXMLRPC::1142::vds::(wrapper)  client [127.0.0.1]::call setupNetworks with ({'rhevm': {'nic': 'eth0', 'netmask': '255.255.252.0', 'bootproto': 'none', 'ipaddr': '10.34.60.233'}}, {}, {'connectivityCheck': False}) {}
 Thread-15::ERROR::2015-01-08 10:26:42,210::BindingXMLRPC::1161::vds::(wrapper) unexpected error
 Traceback (most recent call last):
  File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 1145, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 563, in setupNetworks
    return api.setupNetworks(networks, bondings, options)
  File "/usr/share/vdsm/API.py", line 1385, in setupNetworks
    supervdsm.getProxy().setupNetworks(networks, bondings, options)
  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
    **kwargs)
  File "<string>", line 2, in setupNetworks
  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 726, in _callmethod
    kind, result = conn.recv()
 EOFError

Looks like an issue within VDSM. moving to that component.

Comment 3 Dan Kenigsberg 2015-01-21 12:45:47 UTC
It seems that supervdsm crashed while setting up the network. Please attach supervdsm.log

Comment 4 Petr Matyáš 2015-01-21 13:22:54 UTC
Created attachment 982327 [details]
supervdsm log

Comment 5 Dan Kenigsberg 2015-01-22 14:15:28 UTC
The plot thickens. supervdsm took itself down since its connection to libvirt was broken.

MainProcess|Thread-15::CRITICAL::2015-01-08 10:26:42,140::libvirtconnection::137::root::(wrapper) taking calling process down.
MainThread::DEBUG::2015-01-08 10:26:42,140::supervdsmServer::453::SuperVdsm.Server::(main) Terminated normally
MainProcess|Thread-15::DEBUG::2015-01-08 10:26:42,142::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 1 edom: 7 level: 2 message: internal error client socket is closed

Could you attach libvirtd.conf, libvirtd.log and supply libvirt's version?

Comment 6 Dan Kenigsberg 2015-01-22 14:17:01 UTC
Does /var/log/message has any trace of a libvirtd crash?

Comment 7 Petr Matyáš 2015-01-22 14:29:21 UTC
Created attachment 982841 [details]
libvirtd configuration file

Comment 8 Petr Matyáš 2015-01-22 14:29:46 UTC
Created attachment 982843 [details]
libvirtd log

Comment 9 Petr Matyáš 2015-01-22 14:33:32 UTC
libvirt-0.10.2-46.el6_6.2.x86_64

/var/log/messages doesn't show any error or crash from that day

Comment 10 Dan Kenigsberg 2015-01-22 16:15:47 UTC
Is this libvirtd.log in its entirety?

2015-01-21 12:16:21.097+0000: 1566: error : virNetSocketNewListenTCP:239 : Unable to resolve address '0.0.0.0' service '16514': Address family for hostname not supported

if so, could you set

 log_outputs="1:file:/var/log/libvirt/libvirtd.log"
 auth_unix_rw="none"

in libvirtd.conf, pkill libvirtd, and try to reproduce?
Before installation, does `virsh list` succeed?

Comment 11 Petr Matyáš 2015-01-23 11:30:09 UTC
[root@pm-upgrade2 ~]# virsh list
error: Failed to reconnect to the hypervisor
error: no valid connection
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

When I changed the two lines I couldn't reproduce this bug, although I found a different one:
[ ERROR ] Failed to execute stage 'Closing up': Cannot set temporary password for console connection. The VM may not have been created: please check VDSM logs

Comment 12 Petr Matyáš 2015-01-23 11:30:36 UTC
Created attachment 983306 [details]
new vdsm log

Comment 13 Dan Kenigsberg 2015-01-24 22:36:28 UTC
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory

means that libvirtd failed to start. we should debug this issue first - does the more verbose libvirtd.log say anything?

Comment 14 Petr Matyáš 2015-01-28 13:49:48 UTC
I tried to reproduce this issue but I couldn't, the virsh list now succeeds although it prints no domains. However I reproduced the issue I found before (comment #11)

Comment 15 Dan Kenigsberg 2015-01-28 14:13:24 UTC
Could you supply the logs of the new reproduction?

Comment 16 Petr Matyáš 2015-01-28 14:26:57 UTC
Created attachment 985194 [details]
VDSM log

Comment 17 Petr Matyáš 2015-01-28 14:27:42 UTC
Created attachment 985195 [details]
supervdsm log

Comment 18 Dan Kenigsberg 2015-01-28 14:56:56 UTC
It seems that qemu process crashed right after startup. Can you attach qemu and libvirt logs? Could it be that your host does not have enough ram+swap space for this VM?

Thread-77::DEBUG::2015-01-28 13:22:08,363::libvirtconnection::143::root::(wrapper) Unknown libvirterror: ecode: 1 edom: 10 level: 2 message: internal error Process exited while reading console log output: char device redirected to /dev/pts/2
Cannot set up guest memory 'pc.ram': Cannot allocate memory

Comment 19 Petr Matyáš 2015-01-29 11:12:50 UTC
Sorry, it really didn't have enough memory. Must have been fixed with something else because it works fine now.


Note You need to log in before you can comment on or make changes to this bug.