This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .
Bug 2168470 - VM -> Metrics tab -> Migration: “KV data transfer rate” graph is empty [NEEDINFO]
Summary: VM -> Metrics tab -> Migration: “KV data transfer rate” graph is empty
Keywords:
Status: CLOSED MIGRATED
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: User Experience
Version: 4.12.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: 4.14.0
Assignee: Phillip Bailey
QA Contact: Shijie Xue
URL:
Whiteboard:
Depends On: 2117186 2187304
Blocks:
TreeView+ depends on / blocked
 
Reported: 2023-02-09 06:04 UTC by Guohua Ouyang
Modified: 2023-08-22 11:13 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
: 2187304 (view as bug list)
Environment:
Last Closed: 2023-08-22 11:11:37 UTC
Target Upstream Version:
Embargoed:
phbailey: needinfo?


Attachments (Terms of Use)
vm migration graph empty (141.23 KB, image/png)
2023-02-09 06:04 UTC, Guohua Ouyang
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker   CNV-25210 0 None None None 2023-08-22 11:11:36 UTC

Description Guohua Ouyang 2023-02-09 06:04:11 UTC
Created attachment 1943015 [details]
vm migration graph empty

Description of problem:
VM -> Metrics tab -> Migration: “KV data transfer rate” graph is empty

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. migrate a running vm
2.
3.

Actual results:


Expected results:
"KV data transfer rate” graph is loaded with data

Additional info:

Comment 1 Phillip Bailey 2023-04-12 13:15:05 UTC
@sradco The query[1] used to pull data for this chart isn't returning any data. When I tried looking the metric up in the Observe > Metrics screen, it doesn't show in the list, even though it's in the kubevirt metrics list here: https://github.com/kubevirt/kubevirt/blob/main/docs/metrics.md. Do you have any idea why this metric isn't showing up/returning any data?

[1] sum(sum_over_time(kubevirt_migrate_vmi_disk_transfer_rate_bytes{name='${name}',namespace='${namespace}'}[${duration}]))  BY (name, namespace) 
    Code reference: https://github.com/kubevirt-ui/kubevirt-plugin/blob/bb33bb4ee1c90ad23d72776acebabc952860a5c2/src/utils/components/Charts/utils/queries.ts#L57

Comment 2 Itamar Holder 2023-04-17 09:24:40 UTC
The code says [1]:
```
if jobInfo.DiskBpsSet {
    metrics.pushCommonMetric(
        MigrateVmiDiskTransferRateMetricName,
        "The total VM data processed and migrated.",
        prometheus.GaugeValue,
        float64(jobInfo.DiskBps),
    )
}
```

It seems possible that jobInfo.DiskBpsSet is set to false.

Libvirt docs say [2]:
```
virDomainGetJobStats field: Present only in statistics for a completed job. Optional error message for a failed job.
```

Now, in our code we use `stats, err = dom.GetJobStats(0)` [3], that is 0 as flags. One of the flags is DOMAIN_JOB_STATS_COMPLETED. That makes me wonder - maybe the stat is empty because we don't use the completed flag once the migration is completed?

@bodnopoz, as you are the one who implemented the PR [4], can you confirm that you've tested this metric locally? If so, how did you test it?

Thanks,
Itamar.

[1] https://github.com/kubevirt/kubevirt/blob/v0.60.0-alpha.0/pkg/monitoring/domainstats/prometheus/prometheus.go#L115
[2] https://libvirt.org/html/libvirt-libvirt-domain.html
[3] https://github.com/kubevirt/kubevirt/blob/v0.60.0-alpha.0/pkg/virt-launcher/virtwrap/live-migration-source.go#L719
[4] https://github.com/kubevirt/kubevirt/pull/7946


Note You need to log in before you can comment on or make changes to this bug.