Bug 1883446
Summary: | [Upgrade] migration of VMs from a 4.3 host with rhel-7.9 to v4.4.3 host with rhel-8.3 fails | ||
---|---|---|---|
Product: | [oVirt] vdsm | Reporter: | Roni <reliezer> |
Component: | Core | Assignee: | Milan Zamazal <mzamazal> |
Status: | CLOSED CURRENTRELEASE | QA Contact: | Petr Matyáš <pmatyas> |
Severity: | high | Docs Contact: | |
Priority: | unspecified | ||
Version: | 4.40.31 | CC: | aefrat, ahadas, aoconnor, bugs, jmacku, khakimi, lsurette, michal.skrivanek, msobczyk, mzamazal, pmatyas, reliezer, srevivo, ycui |
Target Milestone: | ovirt-4.4.3 | Keywords: | Automation, AutomationBlocker, Regression |
Target Release: | --- | Flags: | pm-rhel:
ovirt-4.4+
aoconnor: blocker+ |
Hardware: | x86_64 | ||
OS: | Linux | ||
Whiteboard: | |||
Fixed In Version: | vdsm-4.40.33 | Doc Type: | Bug Fix |
Doc Text: |
Migrations of VMs started on hosts < 4.4 with a payload stored on a floppy failed when migrating to 4.4 hosts. This has been fixed and migrations work when migrating to up-to-date 4.4 hosts.
|
Story Points: | --- |
Clone Of: | Environment: | ||
Last Closed: | 2020-11-11 06:42:44 UTC | Type: | Bug |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | Virt | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | |||
Bug Blocks: | 1883817 |
Description
Roni
2020-09-29 08:49:51 UTC
I see that the payload was saved into a file named <vm_id>.<hash>.md: /var/run/vdsm/payload/4809e0c7-f4fd-4ea9-8d86-d452b942929c.5560c76d66146eb12ef8b4165f475c5c.img But then the VM is migrated from a 4.3.11-8 host to a 4.4.3-6 host. In 4.4 the <hash> part was removed from the payload filename (https://gerrit.ovirt.org/#/c/102698/) and so I suspect that on the target host the payload is saved to /var/run/vdsm/payload/4809e0c7-f4fd-4ea9-8d86-d452b942929c.img And therefore the domain cannot access the payload source that is specified in the cd-rom device and the migration fails. This combination of migrating VMs with payload from 4.3 to 4.4 is not that common and this shouldn't block automation. That said, we should preserve backward compatibility for migrating VMs - Milan, what do you think? (In reply to Arik from comment #2) > This combination of migrating VMs with payload from 4.3 to 4.4 is not that > common and this shouldn't block automation. Roni, can you please repeat the tests on a homogeneous environment? I suspect the problem is caused by the fact that we switched from /var/run to /run recently in Vdsm and the error message apparently comes from injectFilesToFs function that would fail exactly on that. We must find a way how to handle migrations with payloads from older Vdsm versions. (In reply to Arik from comment #3) > (In reply to Arik from comment #2) > > This combination of migrating VMs with payload from 4.3 to 4.4 is not that > > common and this shouldn't block automation. > > Roni, can you please repeat the tests on a homogeneous environment? When I shut down the VM that was running on the v4.3 host and then start it on the v4.4 host then I can successfully migrate it to v4.4 host and to v4.3 host and vice versa. It is still an upgrade issue if we want to keep the VMs running during the upgrade (In reply to Roni from comment #5) > When I shut down the VM that was running on the v4.3 host and then start it > on the v4.4 host > then I can successfully migrate it to v4.4 host and to v4.3 host and vice > versa. > It is still an upgrade issue if we want to keep the VMs running during the > upgrade Yes, it seems the VM initially ran with run-once + payload. When shutting down the VM and starting it again, it starts without the payload and then migration from a 4.3 host to a 4.4 host would work (In reply to Milan Zamazal from comment #4) > I suspect the problem is caused by the fact that we switched from /var/run > to /run recently in Vdsm and the error message apparently comes from > injectFilesToFs function that would fail exactly on that. We must find a way > how to handle migrations with payloads from older Vdsm versions. Good, that explains why this issue was reported as a recently introduced regression (the change I've mentioned in comment 2 got in long time ago) Petr, why does it depend on bz 1883817? Because that bug blocks our upgrade flow where this bug happened. (In reply to Petr Matyáš from comment #9) > Because that bug blocks our upgrade flow where this bug happened. So wouldn't it make more sense then to set it the other way around - that this bug blocks bz 1883817 and to verify this one not in the context of upgrade? Verified on vdsm-4.40.35-1.el8ev This bugzilla is included in oVirt 4.4.3 release, published on November 10th 2020. Since the problem described in this bug report should be resolved in oVirt 4.4.3 release, it has been closed with a resolution of CURRENT RELEASE. If the solution does not work for you, please open a new bug report. |