Bug 2173843

Summary: Destination host fail to switch active when migration is active on source host
Product: Red Hat Enterprise Linux 9 Reporter: Li Xiaohui <xiaohli>
Component: qemu-kvmAssignee: Juan Quintela <quintela>
qemu-kvm sub component: Live Migration QA Contact: Li Xiaohui <xiaohli>
Status: CLOSED CURRENTRELEASE Docs Contact:
Severity: medium    
Priority: medium CC: chayang, leobras, lijin, nilal, peterx, quintela, virt-maint
Version: 9.2Keywords: Triaged
Target Milestone: rc   
Target Release: ---   
Hardware: aarch64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-08-07 14:46:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Li Xiaohui 2023-02-28 06:40:12 UTC
Description of problem:
Destination host fail to switch active when migration is active on source host


Version-Release number of selected component (if applicable):
the latest rhel 9.2.0 (qemu-kvm-7.2)


How reproducible:
1/50


Steps to Reproduce:
1. Start zerocopy migration (multifd enabled)
2. 
3.

Actual results:
During zerocopy migration, once get migration active status on the src host, then switch to the dst host for checking migration status, check for 20 mins, migration isn't active. 
In 20 mins, the migration status of the dst host is always like the last line.
Shall we file a new bug for the issue?
2023-02-07-07:53:11: Host(10.19.241.182) Sending qmp command   : {"execute": "migrate-incoming", "arguments": {"uri": "tcp:[::]:4000"}, "id": "x0o5K1Uv"}
2023-02-07-07:53:12: Host(10.19.241.182) Responding qmp command: {"return": {}, "id": "x0o5K1Uv"}
2023-02-07-07:53:12: Host(10.19.241.172) Sending qmp command   : {"execute": "migrate", "arguments": {"uri": "tcp:10.19.241.182:4000", "blk": false, "inc": false, "detach": true, "resume": false}, "id": "gcRVq85Q"}
2023-02-07-07:53:12: Host(10.19.241.172) Responding qmp command: {"return": {}, "id": "gcRVq85Q"}
2023-02-07-07:53:12: Host(10.19.241.172) Sending qmp command   : {"execute": "query-migrate", "id": "U9X5ug18"}
2023-02-07-07:53:12: Host(10.19.241.172) Responding qmp command: {"return": {"expected-downtime": 300, "status": "active", "setup-time": 6, "total-time": 7, "ram": {"total": 4429328384, "postcopy-requests": 0, "dirty-sync-count": 1, "multifd-bytes": 5376, "pages-per-second": 0, "downtime-bytes": 0, "page-size": 4096, "remaining": 4429328384, "postcopy-bytes": 0, "mbps": 0, "transferred": 5376, "dirty-sync-missed-zero-copy": 0, "precopy-bytes": 0, "duplicate": 0, "dirty-pages-rate": 0, "skipped": 0, "normal-bytes": 0, "normal": 0}}, "id": "U9X5ug18"}
2023-02-07-07:53:12: Host(10.19.241.182) Sending qmp command   : {"execute": "query-migrate", "id": "9PJKTJ5W"}
2023-02-07-07:53:12: Host(10.19.241.182) Responding qmp command: {"return": {"socket-address": [{"port": "4000", "ipv6": true, "host": "::", "type": "inet"}]}, "id": "9PJKTJ5W"}


Expected results:
Dst host would switch active at once if migration is active on the source host.


Additional info:
Now reproduce this bug on aarch64, will try to reproduce it on x86.

Comment 4 Li Xiaohui 2023-04-18 07:31:01 UTC
Didn't reproduce this bug on x86 (kernel-5.14.0-284.4.1.el9_2.x86_64 && qemu-kvm-7.2.0-14.el9_2.x86_64) with repeating 550 times.


So change Hardware to aarch64.

Comment 5 Juan Quintela 2023-05-08 14:41:15 UTC
Hi
Could you try to see if you can reproduce without zerocopy?  The bug is certainly strange, and will help to triage it if we know that only happens with zerocopy.
Could you also post the command line that you are using for migration and for launching qemu?

Thanks, Juan

Comment 6 Li Xiaohui 2023-07-05 11:19:24 UTC
Hi Juan, sorry for the late. I will try next week.

Comment 7 Li Xiaohui 2023-08-04 02:59:27 UTC
Hi Juan,
As repeating tests on the latest RHEL 9.3.0 (qemu-kvm-8.0.0-9.el9.aarch64) all pass, can we close this bug currentrelease?

1. repeating 600 times with zerocopy migration, test steps same as Description, all pass
2. repeating 300 times with multifd migration, also pass.

Comment 8 Juan Quintela 2023-08-07 14:46:20 UTC
We can close it as current release.  If it happens in the future, we can revisit.