Bug 1038741

Summary: NICs in unknown or incorrectly down state under Network Interfaces tab for hosts
Product: Red Hat Enterprise Virtualization Manager Reporter: Javier Coscia <jcoscia>
Component: ovirt-engine-webadmin-portalAssignee: Alona Kaplan <alkaplan>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Meni Yakove <myakove>
Severity: medium Docs Contact:
Priority: high    
Version: 3.2.0CC: acathrow, alkaplan, ecohen, iheim, jcoscia, lpeer, lvernia, masayag, nyechiel, Rhev-m-bugs, sraje, ukar, yeylon
Target Milestone: ---Keywords: Triaged, UserExperience
Target Release: 3.4.0   
Hardware: x86_64   
OS: Linux   
Whiteboard: network
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-04-07 08:13:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
Working NIC status
none
Unknown state none

Description Javier Coscia 2013-12-05 17:37:08 UTC
Created attachment 833288 [details]
Working NIC status

NICs in unknown or incorrectly down state under Network Interfaces tab for hosts

Description of problem:
CU is seeing his NICs with a "down" status in the Network interfaces tab and also seeing them with a question mark in front of them while clicking in the "Setup Host Network" button.

Version-Release number of selected component (if applicable):
rhevm-3.2.4-0.44.el6ev.noarch

RHEL host as hypervisor
kernel-2.6.32-358.el6.x86_64
libvirt-0.10.2-29.el6.1.x86_64
vdsm-4.10.2-27.0.el6ev.x86_64
qemu-kvm-rhev-0.12.1.2-2.415.el6_5.3.x86_64

How reproducible:
Haven't tested internally yet

Steps to Reproduce:
1. Install a 6.4 RHEL and configure it as an hypervisor
2. Add it into your RHEV Manager
3. Click on hosts -> Network Interfaces (below) -> Setup Host Network 

Actual results:
Arrows are in red and down, also if you click on Setup Host Network the NIC state is with a question mark "?"

Expected results:
Arrows should be up in green if the NICs are actually working and insted of a question mark at front of them, they should show a green or red circle

Additional info:
Images are attached to the BZ

This is not happening on a different hypervisor which is a RHEV-H iso image (20131016.0.el6) with:
kernel-2.6.32-358.23.2.el6.x86_64
libvirt-0.10.2-18.el6_4.14.x86_64
vdsm-4.10.2-27.0.el6ev.x86_64
qemu-kvm-rhev-0.12.1.2-2.355.el6_4.9.x86_64

Comment 1 Javier Coscia 2013-12-05 17:37:57 UTC
Created attachment 833289 [details]
Unknown state

Comment 2 Moti Asayag 2013-12-08 11:58:08 UTC
Javier,
Could you provide the output of 'vdsClient -s localhost getVdsStats' 
and also vdsm.log so we'll get an indication what is the actual data report by vdsm for these interfaces ?

Comment 8 Javier Coscia 2013-12-09 13:51:38 UTC
(In reply to Moti Asayag from comment #2)
> Javier,
> Could you provide the output of 'vdsClient -s localhost getVdsStats' 
> and also vdsm.log so we'll get an indication what is the actual data report
> by vdsm for these interfaces ?

Hi Moti, I've attached the requested information to the BZ, please let me know if you need any other log.

Thanks!!

jcoscia

Comment 9 Lior Vernia 2014-01-29 12:31:58 UTC
*** Bug 1051495 has been marked as a duplicate of this bug. ***

Comment 10 Moti Asayag 2014-02-04 13:50:26 UTC
(In reply to Javier Coscia from comment #8)
> (In reply to Moti Asayag from comment #2)
> > Javier,
> > Could you provide the output of 'vdsClient -s localhost getVdsStats' 
> > and also vdsm.log so we'll get an indication what is the actual data report
> > by vdsm for these interfaces ?
> 
> Hi Moti, I've attached the requested information to the BZ, please let me
> know if you need any other log.
> 
> Thanks!!
> 
> jcoscia

Thanks for the addition information, looks the same as in the attachments of Bug 1051495. The vdsm reports the nics state correctly. Since the host is not in Non-operational state, the problem should be somwhere between the engine host-nics query to the webadmin portal.

Comment 12 Alona Kaplan 2014-03-26 13:56:00 UTC
Hi Javier,

Can you please provide the engine.log?

Comment 15 Lior Vernia 2014-04-02 07:48:35 UTC
Just ran into this issue when adding a new host to oVirt. Straight after installation it interfaces appeared as unknown. No errors in the log, but the database looks like this:



engine_33=> SELECT * FROM vds_interface_statistics;
                  id                  |                vds_id                | rx_rate | tx_rate | rx_drop | tx_drop | iface_status | _update_date 
--------------------------------------+--------------------------------------+---------+---------+---------+---------+--------------+--------------
 3cac3e3e-756f-4b8b-acec-707d9c9d81a7 | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
 c36da1a4-18ad-49b4-a2d2-4de94ed73d39 | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
 c53eb94d-c8cb-4fdb-b58c-ef1086b7e105 | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
 48f62e1d-4dca-4249-a71d-171599726622 | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
 0a61cc41-cdc2-4d5a-8a40-3f7a570cb003 | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
 a7fcd025-6826-417c-ad72-a500bd7f16ad | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
 aad1baf7-24ff-44d7-ac91-0a85b518ebaa | dda541df-abc7-48d5-a5e7-616cd2e0e043 |         |         |         |         |              | 
(7 rows)



No statistics had been input. Alona, please investigate...

Comment 17 Lior Vernia 2014-04-07 07:05:56 UTC
It is worth mentioning that in my example the host was never in "up" state - it moved straight to non-operational due to VDSM incompatibility, and then I moved it to maintenance; so it is possible it would have been fixed had the host been in up state and the statistics updated automatically.

Comment 18 Alona Kaplan 2014-04-07 08:13:51 UTC
Cannot reproduce this bug.
Next time the customer issues this problem please attach-
1. output of 'vdsClient -s localhost getVdsStats'
2. vdsm.log
3. engine.log
4. output of the following query on the engine db-
select name, iface_status from vds_interface as iface,vds_interface_statistics as stats where iface.id = stats.id;
5. Is it reproducible?
6. Is there a workaround to solve it? Refreshing the webadmin (browser)? Restarting the engine? Restarting the host?