Bug 1441563
| Summary: | [RFE] Indicate domain job kind in job stats | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Milan Zamazal <mzamazal> |
| Component: | libvirt | Assignee: | Jiri Denemark <jdenemar> |
| Status: | CLOSED ERRATA | QA Contact: | yafu <yafu> |
| Severity: | unspecified | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.3 | CC: | dyuan, dzheng, jdenemar, jsuchane, lmen, michal.skrivanek, mtessun, rbalakri, xuzhang, yafu, yalzhang, yanqzhan, zpeng |
| Target Milestone: | rc | Keywords: | FutureFeature |
| Target Release: | --- | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | libvirt-3.2.0-4.el7 | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-08-02 00:05:54 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Milan Zamazal
2017-04-12 08:45:50 UTC
if it's not too difficult a backport would be great, otherwise we have to add workarounds and pollute the metadata section with this information Patches sent upstream for review: https://www.redhat.com/archives/libvir-list/2017-April/msg01292.html This is now implemented upstream by
commit b1c79d785f9385eb4ebd5e71d1d44e67f744f4d9
Refs: v3.2.0-281-gb1c79d785
Author: Jiri Denemark <jdenemar>
AuthorDate: Wed Apr 26 11:59:59 2017 +0200
Commit: Jiri Denemark <jdenemar>
CommitDate: Thu Apr 27 15:06:40 2017 +0200
Add VIR_DOMAIN_JOB_OPERATION typed parameter
The parameter is reported by virDomainGetJobStats API and
VIR_DOMAIN_EVENT_ID_JOB_COMPLETED event and it can be used to identify
the operation (migration, snapshot, ...) to which the reported
statistics belong.
https://bugzilla.redhat.com/show_bug.cgi?id=1441563
Signed-off-by: Jiri Denemark <jdenemar>
commit 2a978269fc9dc31c354a7c9132cb62d52e7613ef
Refs: v3.2.0-282-g2a978269f
Author: Jiri Denemark <jdenemar>
AuthorDate: Wed Apr 26 12:00:09 2017 +0200
Commit: Jiri Denemark <jdenemar>
CommitDate: Thu Apr 27 15:08:12 2017 +0200
qemu: Report VIR_DOMAIN_JOB_OPERATION
Not all async jobs are visible via virDomainGetJobStats (either they are
too fast or getting the stats is not allowed during the job), but
forcing all of them to advertise the operation is easier than hunting
the jobs for which fetching statistics is allowed. And we won't need to
think about this when we add support for getting stats for more jobs.
https://bugzilla.redhat.com/show_bug.cgi?id=1441563
Signed-off-by: Jiri Denemark <jdenemar>
Verify with libvirt-3.2.0-6.virtcov.el7.x86_64. Test steps: 1.Test for start guest: # virsh start rhel7.3-min & virsh domjobinfo rhel7.3-min [12] 27872 Job type: Unbounded Operation: Start Time elapsed: 307 ms 2.Test for save & restore: (1)save # virsh save rhel7.3-min /tmp/save & sleep 0.3 ; virsh domjobinfo rhel7.3-min [11] 25225 Job type: Unbounded Operation: Save Time elapsed: 344 ms Data processed: 1.130 MiB Data remaining: 5.059 GiB Data total: 5.063 GiB Memory processed: 1.130 MiB Memory remaining: 5.059 GiB Memory total: 5.063 GiB Dirty rate: 0 pages/s Iteration: 1 Constant pages: 768 Normal pages: 287 Normal data: 1.121 MiB Expected downtime: 300 ms Setup time: 45 ms (2)restore # virsh restore /tmp/save & sleep 0.3 ; virsh domjobinfo rhel7.3-min [12] 10804 Job type: Unbounded Operation: Restore Time elapsed: 131 ms (3)Check the --completed domjob info after guest restored: # virsh domjobinfo rhel7.3-min --completed Job type: Completed Operation: Save Time elapsed: 1151 ms Data processed: 657.641 MiB Data remaining: 0.000 B Data total: 5.063 GiB Memory processed: 657.641 MiB Memory remaining: 0.000 B Memory total: 5.063 GiB Memory bandwidth: 710.475 MiB/s Dirty rate: 0 pages/s Iteration: 3 Constant pages: 1161836 Normal pages: 165480 Normal data: 646.406 MiB Total downtime: 1142 ms Setup time: 57 ms 3.Test for dump: # virsh dump rhel7.3-min /tmp/dump & sleep 0.1 ; virsh domjobinfo rhel7.3-min [4] 20177 [3]+ Stopped virsh dump rhel7.3-min /tmp/dump Job type: Unbounded Operation: Dump Time elapsed: 142 ms Data processed: 259.271 KiB Data remaining: 5.060 GiB Data total: 5.063 GiB Memory processed: 259.271 KiB Memory remaining: 5.060 GiB Memory total: 5.063 GiB Dirty rate: 0 pages/s Iteration: 1 Constant pages: 8099386 Normal pages: 1192657 Normal data: 4.550 GiB Expected downtime: 300 ms Setup time: 39 ms The domjob kind is also "Dump" when get with '--completed" after the dump completed. 4.Test for snapshot: (1)internal snapshot # virsh snapshot-create rhel7.3-min & sleep 0.3 ; virsh domjobinfo rhel7.3-min [13] 12232 error: Timed out during operation: cannot acquire state change lock (held by remoteDispatchDomainSnapshotCreateXML) [13] Exit 1 virsh snapshot-create rhel7.3-min There is a bug about this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1448306 (2)external snapshot # virsh snapshot-create-as rhel7.3-min s8 --memspec /tmp/test & sleep 1 ; virsh domjobinfo rhel7.3-min [12] 15361 Job type: Unbounded Operation: Snapshot Time elapsed: 1001 ms Data processed: 57.981 MiB Data remaining: 645.770 MiB Data total: 5.063 GiB Memory processed: 57.981 MiB Memory remaining: 645.770 MiB Memory total: 5.063 GiB Memory bandwidth: 37.666 MiB/s Dirty rate: 0 pages/s Iteration: 1 Constant pages: 6954365 Normal pages: 844213 Normal data: 3.220 GiB Expected downtime: 300 ms Setup time: 35 ms The domain job kind is also "Snapshot" with --completed. 5.Test for snapshot-revert(only internal snapshot supported): # virsh snapshot-revert rhel7.3-min 1495871240 & sleep 0.2 ; virsh domjobinfo rhel7.3-min [12] 18279 Job type: Unbounded Operation: Snapshot revert Time elapsed: 918 ms 6.Test for migration: (1)Do migration: virsh migrate rhel7.3-min qemu+ssh://10.66.4.251/system --live --verbose (2)Open another migration to get domain job info #virsh domjobinfo # virsh domjobinfo rhel7.3-min Job type: Unbounded Operation: Outgoing migration Time elapsed: 7069 ms Data processed: 28.041 MiB Data remaining: 1.045 GiB Data total: 5.063 GiB Memory processed: 28.041 MiB Memory remaining: 1.045 GiB Memory total: 5.063 GiB Memory bandwidth: 5.010 MiB/s Dirty rate: 0 pages/s Iteration: 1 Constant pages: 1048517 Normal pages: 4865 Normal data: 19.004 MiB Expected downtime: 300 ms Setup time: 42 ms (3)After migration completed, get domain job info with --completed on source host: # virsh domjobinfo rhel7.3-min --completed Job type: Completed Operation: Outgoing migration Time elapsed: 4240 ms Time elapsed w/o network: 4236 ms Data processed: 226.817 MiB Data remaining: 0.000 B Data total: 5.063 GiB Memory processed: 226.817 MiB Memory remaining: 0.000 B Memory total: 5.063 GiB Memory bandwidth: 87.136 MiB/s Dirty rate: 0 pages/s Iteration: 4 Constant pages: 1293990 Normal pages: 55114 Normal data: 215.289 MiB Total downtime: 262 ms Downtime w/o network: 258 ms Setup time: 58 ms (4)After migration completed, get domain job info with --completed on source host: # virsh domjobinfo rhel7.3-min --completed Job type: Completed Operation: ****Unknown**** Time elapsed: 4220 ms Time elapsed w/o network: 4216 ms Data processed: 226.817 MiB Data remaining: 0.000 B Data total: 5.063 GiB Memory processed: 226.817 MiB Memory remaining: 0.000 B Memory total: 5.063 GiB Memory bandwidth: 87.136 MiB/s Dirty rate: 0 pages/s Iteration: 4 Constant pages: 1293990 Normal pages: 55114 Normal data: 215.289 MiB Total downtime: 262 ms Downtime w/o network: 258 ms Setup time: 58 ms Hi, Jiri, If trying to get the domjob info on target host when doing migration, it will show 3 kinds of info: 1.Execute "virsh domjobinfo" immediately after the migration start: #virsh domjobinfo rhel7.3-min Job type: Unbounded Operation: Incoming migration Time elapsed: 105 ms 2.Execute "virsh domjobinfo" after migration start about few seconds: #virsh domjobinfo rhel7.3-min error: Operation not supported: migration statistics are available only on the s ource host 3.Execute "virsh domjobinfo --completed" after the migration completed: # virsh domjobinfo rhel7.3-min --completed Job type: Completed Operation: ****Unknown**** Time elapsed: 4220 ms Time elapsed w/o network: 4216 ms Data processed: 226.817 MiB Data remaining: 0.000 B Data total: 5.063 GiB Memory processed: 226.817 MiB Memory remaining: 0.000 B Memory total: 5.063 GiB Memory bandwidth: 87.136 MiB/s Dirty rate: 0 pages/s Iteration: 4 Constant pages: 1293990 Normal pages: 55114 Normal data: 215.289 MiB Total downtime: 262 ms Downtime w/o network: 258 ms Setup time: 58 ms The domain job kind shows "Unknown" on target host after the migration completed. Would you help to check the issue please? Thanks a lot. Oh, looks like I forgot to update the operation in completed stats on the destination. Checking completed stats on the source host should work though. And it should also be reported in the migration event sent by the source daemon. (In reply to Jiri Denemark from comment #11) > Oh, looks like I forgot to update the operation in completed stats on the > destination. Checking completed stats on the source host should work though. > And it should also be reported in the migration event sent by the source > daemon. Thanks. I filed a separate bug to track this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1457052 And I found the operation shows the digit not name in the event, does that work as expected? Test stesps: 1.#virsh event --all --loop 2.Open another terminal, do migration: migrate rhel7.3-min qemu+ssh://10.66.4.251/system --live --verbose Migration: [100 %] 3.Check the outputs of event: ... event 'migration-iteration' for domain rhel7.3-min: iteration: '1' event 'migration-iteration' for domain rhel7.3-min: iteration: '2' event 'migration-iteration' for domain rhel7.3-min: iteration: '3' event 'lifecycle' for domain rhel7.3-min: Suspended Migrated event 'migration-iteration' for domain rhel7.3-min: iteration: '4' event 'lifecycle' for domain rhel7.3-min: Stopped Migrated event 'job-completed' for domain rhel7.3-min: operation: ****5***** time_elapsed: 8550 time_elapsed_net: 8547 downtime: 501 downtime_net: 498 setup_time: 38 data_total: 5436686336 data_processed: 702045654 data_remaining: 0 memory_total: 5436686336 memory_processed: 702045654 memory_remaining: 0 memory_bps: 99365940 memory_constant: 1161186 memory_normal: 168517 memory_normal_bytes: 690245632 memory_dirty_rate: 0 memory_iteration: 4 disk_total: 0 disk_processed: 0 disk_remaining: 0 Dump/save/snapshot also show digit operation in event outputs. Yeah, the job-completed event printing is very simple and it just prints raw data from libvirt without translating them into human readable format. Patch sent upstream for review: https://www.redhat.com/archives/libvir-list/2017-June/msg00220.html (In reply to Jiri Denemark from comment #14) > Patch sent upstream for review: > https://www.redhat.com/archives/libvir-list/2017-June/msg00220.html Oops, this obviously applies to another bz. (In reply to Jiri Denemark from comment #15) > (In reply to Jiri Denemark from comment #14) > > Patch sent upstream for review: > > https://www.redhat.com/archives/libvir-list/2017-June/msg00220.html > > Oops, this obviously applies to another bz. Thanks for your reply. According to comment 9 and comment 12, move this bug to verified. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2017:1846 |