Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1221467

Summary: [HC][HE]Failed to create RHEVM-Bridge and stuck in the middle of HC-HE deployment at configuring the management bridge
Product: Red Hat Enterprise Virtualization Manager Reporter: Nikolai Sednev <nsednev>
Component: vdsmAssignee: Nobody <nobody>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Meni Yakove <myakove>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 3.6.0CC: bazulay, danken, dfediuck, ecohen, gklein, lpeer, lsurette, sbonazzo, yeylon, ylavi
Target Milestone: ovirt-4.0.0-alpha   
Target Release: 4.0.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: network
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-10-22 14:39:00 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1175354    
Attachments:
Description Flags
alma02 logs
none
answers-20150513182226.conf none

Description Nikolai Sednev 2015-05-14 06:11:10 UTC
Description of problem:
Failed to create RHEVM-Bridge and stuck in the middle of HC-HE deployment at creating network bridge

Thread-16::ERROR::2015-05-13 18:22:26,736::BindingXMLRPC::1176::vds::(wrapper) unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 1160, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/rpc/BindingXMLRPC.py", line 574, in setupNetworks
    return api.setupNetworks(networks, bondings, options)
  File "/usr/share/vdsm/API.py", line 1421, in setupNetworks
    supervdsm.getProxy().setupNetworks(networks, bondings, options)
  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
    return callMethod()
  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
    **kwargs)
  File "<string>", line 2, in setupNetworks
  File "/usr/lib64/python2.7/multiprocessing/managers.py", line 773, in _callmethod
    raise convert_to_error(kind, result)
KeyError: 'bridged'
periodic.Scheduler::DEBUG::2015-05-13 18:22:26,989::periodic::182::periodic.Operation::(_step) after 2.000000 seconds: VmDispatch
er(<class 'virt.periodic.DriveWatermarkMonitor'>)

Version-Release number of selected component (if applicable):
glusterfs-api-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
glusterfs-server-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
mom-0.4.3-1.el7.noarch
glusterfs-cli-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
glusterfs-fuse-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
glusterfs-libs-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
vdsm-4.17.0-786.git07dec6d.el7.centos.x86_64
qemu-kvm-ev-2.1.2-23.el7_1.2.x86_64
glusterfs-rdma-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
libvirt-client-1.2.8-16.el7_1.3.x86_64
sanlock-3.2.2-2.el7.x86_64
glusterfs-client-xlators-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
glusterfs-geo-replication-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64
glusterfs-3.7.0beta2-0.2.gitc1cd4fa.el7.centos.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Deploy HC-HE on a host.
2.
3.

Actual results:
Deployment stuck at creating network bridge part and host becomes non-responsive unless power-cycled.

Expected results:
Deployment should succeed.

Additional info:
Logs attached.

Comment 1 Nikolai Sednev 2015-05-14 06:32:09 UTC
Created attachment 1025286 [details]
alma02 logs

Comment 2 Nikolai Sednev 2015-05-14 07:35:05 UTC
Created attachment 1025293 [details]
answers-20150513182226.conf

Comment 3 Doron Fediuck 2015-05-17 06:32:44 UTC
The networking issue is irrelevant and probably the same issue you
had in Bug 1200474. I'd suggest resolving your environment network issues.

The one thing which we may want to consider is the way HC setup fails.
Sandro, can you please check if there's a more graceful way to fail?

Comment 4 Nikolai Sednev 2015-05-17 09:03:33 UTC
(In reply to Doron Fediuck from comment #3)
> The networking issue is irrelevant and probably the same issue you
> had in Bug 1200474. I'd suggest resolving your environment network issues.
> 
> The one thing which we may want to consider is the way HC setup fails.
> Sandro, can you please check if there's a more graceful way to fail?

From the second run, network bridge was created successfully, nothing connected to network infrastructure, as for regular HE everything works just fine at this very host.

Comment 5 Sandro Bonazzola 2015-05-18 09:20:06 UTC
(In reply to Doron Fediuck from comment #3)
> The networking issue is irrelevant and probably the same issue you
> had in Bug 1200474. I'd suggest resolving your environment network issues.
> 
> The one thing which we may want to consider is the way HC setup fails.
> Sandro, can you please check if there's a more graceful way to fail?

Above Traceback is from vdsm logs, not from Hosted Engine output.
Moving to vdsm network to be reviewed.

Comment 6 Nikolai Sednev 2015-05-26 16:27:52 UTC
Not being reproduced anymore on these components:
glusterfs-server-3.7.0-2.el7.x86_64
libvirt-daemon-driver-nodedev-1.2.8-16.el7_1.3.x86_64
sanlock-python-3.2.2-2.el7.x86_64
glusterfs-cli-3.7.0-2.el7.x86_64
qemu-kvm-ev-2.1.2-23.el7_1.3.1.x86_64
libvirt-daemon-config-nwfilter-1.2.8-16.el7_1.3.x86_64
vdsm-4.17.0-860.git92219e2.el7.noarch
glusterfs-3.7.0-2.el7.x86_64
libvirt-daemon-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-secret-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-qemu-1.2.8-16.el7_1.3.x86_64
ovirt-release-master-001-0.9.master.noarch
ovirt-host-deploy-1.4.0-0.0.master.20150525194300.git9a06f4b.el7.noarch
glusterfs-libs-3.7.0-2.el7.x86_64
ovirt-engine-sdk-python-3.6.0.0-0.14.20150520.git8420a90.el7.centos.noarch
libvirt-daemon-driver-nwfilter-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-interface-1.2.8-16.el7_1.3.x86_64
mom-0.4.4-0.0.master.20150525150210.git93ec8be.el7.noarch
glusterfs-api-3.7.0-2.el7.x86_64
glusterfs-rdma-3.7.0-2.el7.x86_64
ovirt-hosted-engine-setup-1.3.0-0.0.master.20150518075146.gitdd9741f.el7.noarch
libvirt-python-1.2.8-7.el7_1.1.x86_64
libvirt-daemon-kvm-1.2.8-16.el7_1.3.x86_64
glusterfs-client-xlators-3.7.0-2.el7.x86_64
glusterfs-geo-replication-3.7.0-2.el7.x86_64
libvirt-client-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-storage-1.2.8-16.el7_1.3.x86_64
libvirt-daemon-driver-network-1.2.8-16.el7_1.3.x86_64
ovirt-hosted-engine-ha-1.3.0-0.0.master.20150424113553.20150424113551.git7c14f4c.el7.noarch
sanlock-3.2.2-2.el7.x86_64
qemu-kvm-tools-ev-2.1.2-23.el7_1.3.1.x86_64
glusterfs-fuse-3.7.0-2.el7.x86_64
libvirt-lock-sanlock-1.2.8-16.el7_1.3.x86_64
sanlock-lib-3.2.2-2.el7.x86_64
qemu-kvm-common-ev-2.1.2-23.el7_1.3.1.x86_64
qemu-img-ev-2.1.2-23.el7_1.3.1.x86_64

Comment 7 Dan Kenigsberg 2015-10-22 14:39:00 UTC
(In reply to Nikolai Sednev from comment #6)
> Not being reproduced anymore on these components:


Please reopen when this is reproducible.