This bug has been migrated to another issue tracking site. It has been closed here and may no longer be being monitored.

If you would like to get updates for this issue, or to participate in it, you may do so at Red Hat Issue Tracker .
Bug 2117186 - missing kubevirt_migrate_vmi_disk_transfer_rate_bytes
Summary: missing kubevirt_migrate_vmi_disk_transfer_rate_bytes
Keywords:
Status: CLOSED MIGRATED
Alias: None
Product: Container Native Virtualization (CNV)
Classification: Red Hat
Component: Virtualization
Version: 4.11.0
Hardware: Unspecified
OS: Unspecified
unspecified
high
Target Milestone: ---
: future
Assignee: João Vilaça
QA Contact: Kedar Bidarkar
URL:
Whiteboard:
: 2187304 (view as bug list)
Depends On:
Blocks: 2168470
TreeView+ depends on / blocked
 
Reported: 2022-08-10 09:32 UTC by Matan Schatzman
Modified: 2023-12-14 16:15 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-12-14 16:15:11 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
missing-query (212.77 KB, image/png)
2022-08-10 09:32 UTC, Matan Schatzman
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Github kubevirt kubevirt pull 10444 0 None closed Fix missing migration transfer rate metric 2023-11-07 13:25:32 UTC
Github kubevirt kubevirt pull 10693 0 None Merged Remove MigrateVmiDiskTransferRateMetric 2023-11-12 13:11:33 UTC
Red Hat Issue Tracker   CNV-20405 0 None None None 2023-12-14 16:15:11 UTC

Description Matan Schatzman 2022-08-10 09:32:16 UTC
Created attachment 1904648 [details]
missing-query

Description of problem:
When migrating a VM, I can see data for other migration queries but not for kubevirt_migrate_vmi_disk_transfer_rate_bytes.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Migrate a vm
2. No results for kubevirt_migrate_vmi_disk_transfer_rate_bytes
3.

Actual results:


Expected results:
data from kubevirt_migrate_vmi_disk_transfer_rate_bytes query

Additional info:
according to https://github.com/kubevirt/monitoring/blob/main/docs/metrics.md - it should provide data

Comment 2 Igor Bezukh 2022-08-15 10:39:02 UTC
I had a similar issue but with the kubevirt_migrate_vmi_succeeded_total metric, following are the steps for reproduction:

1. kubectl apply examples/vmi-migratable.yaml
2. kubectl apply examples/migration-job.yaml
3. kubectl exec -it /bin/bash <source_virt_handler_pod_name> -n {kubevirt, openshift-cnv}
4. (From inside the virt-handler pod) curl -kL https://locahost:8443/metrics | grep kubevirt_migrate_vmi_succeeded_total

Comment 5 Krzysztof Majcher 2022-08-23 12:37:16 UTC
Please update 'target release' according to your plans.

Comment 6 Dominik Holler 2022-08-24 06:46:54 UTC
Even this bug is related to metrics, I change the component to Infra to keep it on the radar of the infra team.

Comment 8 Dominik Holler 2022-10-27 15:31:35 UTC
Restoring the correct state of the bug.

Comment 11 Dominik Holler 2023-04-24 15:27:56 UTC
*** Bug 2187304 has been marked as a duplicate of this bug. ***

Comment 12 Dominik Holler 2023-04-24 15:28:35 UTC
There is a great analysis at https://bugzilla.redhat.com/show_bug.cgi?id=2187304#c0 .

Comment 13 Fabian Deutsch 2023-08-28 13:27:17 UTC
@dholler can you please summarize in a sentence how the analysis relates to this and means for this bug?

Comment 14 Dominik Holler 2023-08-28 13:35:21 UTC
The bug is that kubevirt does not get the related values from libvirt.
https://bugzilla.redhat.com/show_bug.cgi?id=2187304#c0  points out as one possible reason, that kubevirt is not using the libvirt API like expected.
Another possible reason is, that libvirt itself has a bug.

Comment 15 Fabian Deutsch 2023-08-30 07:57:56 UTC
Thanks, Dominik.

Moving this over to virtualization.

Comment 16 João Vilaça 2023-09-06 14:02:33 UTC
From https://listman.redhat.com/archives/libvir-list/2023-September/241935.html

"This data was available with the old-style disk migration where disk
data was migrated over the migration stream.

As that was deprecated by qemu, libvirt switched to a new approach where
we migrate the disks by using NBD protocol and copying them via
'blockdev-mirror'.

Now the qemu blockjobs themselves do not report the actual bandwidth used,
just the progress. Since libvirt doesn't want to pointlessly poll this
data just to calculate bandwidth, the data is no longer available."

Comment 17 Shirly Radco 2023-09-12 06:58:02 UTC
We are not migrating the disk, since it is supposed to be saved to persistent storage. From what I know we are copying only the memory.
Could it be that we only need the kubevirt_migrate_vmi_memory_transfer_rate_bytes?

Comment 18 João Vilaça 2023-09-12 09:08:02 UTC
(In reply to Shirly Radco from comment #17)
> We are not migrating the disk, since it is supposed to be saved to
> persistent storage. From what I know we are copying only the memory.
> Could it be that we only need the
> kubevirt_migrate_vmi_memory_transfer_rate_bytes?

If I create and migrate a VM with the following config:

---
apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
metadata:
  labels:
    special: vmi-nocloud
  name: vmi-nocloud
spec:
  networks:
    - name: default
      pod: { }
  domain:
    devices:
      interfaces:
        - masquerade: { }
          name: default
      disks:
      - disk:
          bus: virtio
        name: containerdisk
      - disk:
          bus: virtio
        name: cloudinitdisk
      - disk:
          bus: virtio
        name: emptydisk
    resources:
      requests:
        memory: 128Mi
  terminationGracePeriodSeconds: 0
  volumes:
  - containerDisk:
      image: registry:5000/kubevirt/cirros-container-disk-demo:devel
    name: containerdisk
  - cloudInitNoCloud:
      userData: |
        #!/bin/sh

        echo 'printed from cloud-init userdata'
    name: cloudinitdisk
  - emptyDisk:
      capacity: 2Gi
    name: emptydisk

libvirt reports disk being migrated alongside memory

Comment 19 Fabian Deutsch 2023-09-13 11:24:02 UTC
Joao, correct.
containerDisks do include block migration


Note You need to log in before you can comment on or make changes to this bug.