Bug 1636102
| Summary: | Update instance host and task state when post live migration fails | ||
|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Pablo Caruana <pcaruana> |
| Component: | openstack-nova | Assignee: | Artom Lifshitz <alifshit> |
| Status: | CLOSED ERRATA | QA Contact: | OSP DFG:Compute <osp-dfg-compute> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 10.0 (Newton) | CC: | akarlsso, alifshit, amodi, berrange, dasmith, eglynn, jamsmith, jhakimra, joboyer, kchamart, lyarwood, mschuppe, paldeala, rafael.lopezencinas, sbauza, sgordon, tvvcox, vromanso, ykawada |
| Target Milestone: | z10 | Keywords: | Triaged, ZStream |
| Target Release: | 10.0 (Newton) | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | openstack-nova-14.1.0-35.el7ost | Doc Type: | Bug Fix |
| Doc Text: |
This update fixes issues that could leave an instance record in an inconsistent state after failure of a volume API (Cinder) call during a live migration.
Prior to this update, if the volume API call failed, the live migration didn't finish correctly and the instance record was left in an inconsistent state in the database.
With this update, the instance record is updated correctly in such cases. The volume API error is logged. Administrators may need to clean up the instance's volume attachments.
|
Story Points: | --- |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-01-16 17:09:01 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1269615 | ||
|
Description
Pablo Caruana
2018-10-04 13:10:19 UTC
I talked with Joachim on IRC this morning - we mentioned coming up with a workaround, since the upstream fix might be touchy and take a while. Having looked over the sosreports and code, the only workaround I can come up with is manually updating instance.host in the database once a situation like this has been detected. It sucks, but once the system is in an inconsistent state, I don't see another way of fixing it. Also, be aware that because post_live_migration failed on the source, some other things didn't get done: * VIFs on the source weren't unplugged. * Port bindings weren't updated to reflect the instance being on the destination There are a bunch of other bugs that I believe are either identical or similar enough that it's worth it to analyse them all before coming up with a fix. That's my next step, and then I'll post a patch. Since Jaochim asked on IRC, here's the list of BZs that I think are related/duplicates: * bz 1289858 * bz 1630771 * bz 1636280 Bz 1289858 and bz 1630771 are indeed identical, and are both caused by failures when calling Cinder - either Cinder itself, or something "in front" of Cinder, like Keystone or HAProxy. I've proposed [1] upstream. It's really "dumb", but by virtue of being "dumb" it's also simple and minimizes potential side effects. Since the single common root cause in all 3 bugs is the external API call, I've just wrapped it in a try/except. Let's see what the community thinks. [1] https://review.openstack.org/609517 PS: Bz 1636280 is similar but unrelated, and will need a different fix. *** Bug 1630771 has been marked as a duplicate of this bug. *** *** Bug 1289858 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0074 |