Bug 1502510

Summary: [downstream clone - 4.1.7] Rest API does not report network statistics host "data.current.tx, data.current.rx"
Product: Red Hat Enterprise Virtualization Manager Reporter: rhev-integ
Component: ovirt-engineAssignee: Dominik Holler <dholler>
Status: CLOSED ERRATA QA Contact: Michael Burman <mburman>
Severity: medium Docs Contact:
Priority: high    
Version: 4.0.3CC: bgraveno, danken, lsurette, mburman, mgoldboi, mjankula, mperina, pstehlik, rbalakri, Rhev-m-bugs, sradco, srevivo, ykaul, ylavi
Target Milestone: ovirt-4.1.7Keywords: ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
The precision of rx_rate, tx_rate, rx_drop, and tx_drop of virtual and host network interfaces have been increased. Network traffic 100 times smaller can now be detected on network interface statistics. If traffic on the network interface is below the precision of the network interface statistics, it is not reflected in the statistics.
Story Points: ---
Clone Of: 1459908 Environment:
Last Closed: 2017-11-07 17:27:54 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Network RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Bug Depends On: 1459908    
Bug Blocks: 1408963    

Description rhev-integ 2017-10-16 07:17:13 UTC
+++ This bug is a downstream clone. The original bug is: +++
+++   bug 1459908 +++
======================================================================

Description of problem:
Rest API does not report network statistics host "data.current.tx, data.current.rx"

Version-Release number of selected component (if applicable):
rhevm 4.0.3-0.1.el7ev
rhvh 4.0.5.2
rhvh 4.0.7.1


How reproducible:
everytime

Steps to Reproduce:
1. install rhevm and hypervisor
2. connect hypervisor to rhevm
3. query app

"curl -s -k -X GET -H "Accept: application/json" -u admin@internal:redhat https://rhevm.example.com/ovirt-engine/api/hosts/host_uuid/nics/nic_uuid/statistics"

Actual results:
   "host_nic" : {
      "href" : "/ovirt-engine/api/hosts/0d6fdb5d-7fc5-40a7-a8fe-dd2ab75143a1/nics/ca159e66-1f76-4d9e-b072-d6146c665722",
      "id" : "ca159e66-1f76-4d9e-b072-d6146c665722"
    },
    "name" : "data.current.rx",
    "description" : "Receive data rate",
    "href" : "/ovirt-engine/api/hosts/0d6fdb5d-7fc5-40a7-a8fe-dd2ab75143a1/nics/ca159e66-1f76-4d9e-b072-d6146c665722/statistics/ecd0559f-e88f-3330-94b4-1f091b0ffdf7",
    "id" : "ecd0559f-e88f-3330-94b4-1f091b0ffdf7"
  }, {
    "kind" : "gauge",
    "type" : "decimal",
    "unit" : "bytes_per_second",
    "values" : {
      "value" : [ {
        "datum" : 0
      } ]
    },
    "host_nic" : {
      "href" : "/ovirt-engine/api/hosts/0d6fdb5d-7fc5-40a7-a8fe-dd2ab75143a1/nics/ca159e66-1f76-4d9e-b072-d6146c665722",
      "id" : "ca159e66-1f76-4d9e-b072-d6146c665722"
    },
    "name" : "data.current.tx",
    "description" : "Transmit data rate",
    "href" : "/ovirt-engine/api/hosts/0d6fdb5d-7fc5-40a7-a8fe-dd2ab75143a1/nics/ca159e66-1f76-4d9e-b072-d6146c665722/statistics/2c6fe105-4913-3f90-9425-07ddfeca5417",
    "id" : "2c6fe105-4913-3f90-9425-07ddfeca5417"
  }, {
    "kind" : "counter",
    "type" : "integer",
    "unit" : "bytes",
    "values" : {
      "value" : [ {
        "datum" : 286967489913
      } ]
    },


Expected results:
values for current statistics

Additional info:

(Originally by Marian Jankular)

Comment 1 rhev-integ 2017-10-16 07:17:23 UTC
RX/TX values in API are computed from RX/TX percentages and max bandwidth of the interface (provided by speed attribute), so if speed is zero, the also RX/TX are zeros.

Could you please take a look at the value of speed attribute? If it's non-zero could you please check RX/TX values in webadmin if they are the same as in API?

(Originally by Martin Perina)

Comment 3 rhev-integ 2017-10-16 07:17:29 UTC
in my setup i have speed 0, and rx/tx values are "<1"

(Originally by Marian Jankular)

Comment 4 rhev-integ 2017-10-16 07:17:35 UTC
(In reply to Marian Jankular from comment #2)
> in my setup i have speed 0, and rx/tx values are "<1"

Can you try and run some traffic over?

(Originally by Yaniv Kaul)

Comment 5 rhev-integ 2017-10-16 07:17:40 UTC
I did tried on our 4.1 environment which has load - 20 vms per node

in web ui i see speed 1000, rx rate is 10, txrate is <1 however when using api i am still getting no value for current 

    "name" : "data.current.rx",
    "description" : "Receive data rate",
    "href" : "/ovirt-engine/api/hosts/f1e96245-7ca3-4729-8abf-980501380a2b/nics/a442b3d7-7d3e-456b-a9cb-900d22171464/statistics/ecd0559f-e88f-3330-94b4-1f091b0ffdf7",
    "id" : "ecd0559f-e88f-3330-94b4-1f091b0ffdf7"
  }, {
    "kind" : "gauge",
    "type" : "decimal",
    "unit" : "bytes_per_second",
    "values" : {
      "value" : [ {
        "datum" : 0
      } ]
    },
    "host_nic" : {
      "href" : "/ovirt-engine/api/hosts/f1e96245-7ca3-4729-8abf-980501380a2b/nics/a442b3d7-7d3e-456b-a9cb-900d22171464",
      "id" : "a442b3d7-7d3e-456b-a9cb-900d22171464"
    },
    "name" : "data.current.tx",
    "description" : "Transmit data rate",
    "href" : "/ovirt-engine/api/hosts/f1e96245-7ca3-4729-8abf-980501380a2b/nics/a442b3d7-7d3e-456b-a9cb-900d22171464/statistics/2c6fe105-4913-3f90-9425-07ddfeca5417",
    "id" : "2c6fe105-4913-3f90-9425-07ddfeca5417"
  }, {
    "kind" : "counter",
    "type" : "integer",
    "unit" : "bytes",
    "values" : {
      "value" : [ {
        "datum" : 199652833504
      } ]

(Originally by Marian Jankular)

Comment 6 rhev-integ 2017-10-16 07:17:47 UTC
I'm not able to reproduce on RHVM 4.1.2, it works as expected (checked on several hosts):

1. When rx/tx rate in webadmin is <1, then it's shown as 0 in restapi
2. When speed is non-zero and rx/tx rate in webadmin are different than <1, non-zero values are shown in restapi

@Dan, is there any known issue around this area?

(Originally by Martin Perina)

Comment 7 rhev-integ 2017-10-16 07:17:53 UTC
Moving to network team.

(Originally by Oved Ourfali)

Comment 8 rhev-integ 2017-10-16 07:18:00 UTC
We are planning to release a metrics store next month that will allow you to collect raw value more accurately. The customer will probably want to move to this for monitoring.

(Originally by ylavi)

Comment 9 rhev-integ 2017-10-16 07:18:05 UTC
If we change the value type then the DWH will need to be changed as well.

(Originally by ylavi)

Comment 10 rhev-integ 2017-10-16 07:18:11 UTC
(In reply to Martin Perina from comment #5)
> I'm not able to reproduce on RHVM 4.1.2, it works as expected (checked on
> several hosts):
> 
> 1. When rx/tx rate in webadmin is <1, then it's shown as 0 in restapi
> 2. When speed is non-zero and rx/tx rate in webadmin are different than <1,
> non-zero values are shown in restapi
> 
> @Dan, is there any known issue around this area?

I remember sradco complaining about something similar in the past. Do you recall a bz#?

(Originally by danken)

Comment 14 rhev-integ 2017-10-16 07:18:33 UTC
Verified on - 4.2.0-0.0.master.20170901193740.git7900511.el7.centos

(Originally by Michael Burman)

Comment 15 rhev-integ 2017-10-16 07:18:40 UTC
Dan - can we push to 4.1.z?
A CFME bug depends on that one.

(Originally by Oved Ourfali)

Comment 16 rhev-integ 2017-10-16 07:18:46 UTC
(In reply to Oved Ourfali from comment #14)
> Dan - can we push to 4.1.z?
> A CFME bug depends on that one.

yes, it should not be hard.

(Originally by danken)

Comment 17 Dan Kenigsberg 2017-10-17 18:45:54 UTC
This sneaked into 4.1.7. Not too bad.

Comment 19 Dominik Holler 2017-10-20 13:27:47 UTC
I got unplausible values for rx and tx data rate in my local virtual environment. Would be good if the plausibility of the data rates can be checked in a physical environment.

Comment 20 Michael Burman 2017-10-23 10:40:47 UTC
(In reply to Dominik Holler from comment #19)
> I got unplausible values for rx and tx data rate in my local virtual
> environment. Would be good if the plausibility of the data rates can be
> checked in a physical environment.

Dominik, you are absolutely right, api report wrong data rate(tested with physical hosts). I will report a new bug for this issue, this should be fixed for both master and 4.1.7.

The current report can be verified as we do report the current rx and tx, but report wrong data rates. 

Verified on - 4.2.0-0.0.master.20171022103432.gitaf9d8b6.el7.centos

Comment 21 Michael Burman 2017-10-23 10:55:52 UTC
To track the new issue please see BZ 1505328

Comment 23 errata-xmlrpc 2017-11-07 17:27:54 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3138