Bug 1635621

Summary: HE VM migration fails with libvirtError: Unsafe migration: Migration without shared storage is unsafe
Product: [oVirt] ovirt-engine Reporter: Polina <pagranat>
Component: BLL.HostedEngineAssignee: Doron Fediuck <dfediuck>
Status: CLOSED DUPLICATE QA Contact: Polina <pagranat>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.2.6.2CC: bugs, michal.skrivanek, ratamir, stirabos
Target Milestone: ---Keywords: Automation, Regression
Target Release: ---Flags: ratamir: blocker?
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-10-03 13:34:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1635705    
Bug Blocks:    
Attachments:
Description Flags
engine and vdsm logs none

Description Polina 2018-10-03 11:59:53 UTC
Created attachment 1489960 [details]
engine and vdsm logs

Description of problem: HE VM migration triggered by user fails (no maintenance) with libvirtError: Unsafe migration: Migration without shared storage is unsafe. The regular VM in the same environment is migrated ok.

Version-Release number of selected component:
OS Version:RHEL - 7.6 - 7.0.el7
OS Description:Red Hat Virtualization Host 4.2.7 (el7.6)
Kernel Version: 3.10.0 - 948.el7.x86_64
KVM Version:2.12.0 - 16.el7
LIBVIRT Version:libvirt-4.5.0-9.el7
VDSM Version:vdsm-4.20.40-1.el7ev
ovirt-engine-4.2.7.1-0.1.el7ev.noarch

How reproducible: The problem happens in the automation environment rhv-4.2-ge-runner-tier1 - rhv-4.2.7-3, GE-4 - Build #234 (RHVH TESTING 4.2-20180928.0). 100%

Steps to Reproduce:
1. Migrate the HE VM with no inserting to maintenance either by senting rest API or from webUI (happens in both cases) Hosted Storage is GlusterFS. 

Actual results: migration fails. Trace on vdsm :
2018-10-03 10:48:33,907+0300 ERROR (migsrc/e69dfd8d) [virt.vm] (vmId='e69dfd8d-25f5-441a-929b-5884150392c2') Failed to migrate (migration:455)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 437, in _regular_run
    self._startUnderlyingMigration(time.time())
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 511, in _startUnderlyingMigration
    self._perform_with_downtime_thread(duri, muri)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 580, in _perform_with_downtime_thread
    self._perform_migration(duri, muri)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line 529, in _perform_migration
    self._migration_flags)
  File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 98, in f
    ret = attr(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 130, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 92, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1779, in migrateToURI3
    if ret == -1: raise libvirtError ('virDomainMigrateToURI3() failed', dom=self)
libvirtError: Unsafe migration: Migration without shared storage is unsafe


Expected results: migration succeeds


Additional info: hosted-engine --vm-status
--== Host 1 status ==--

conf_on_shared_storage             : True
Status up-to-date                  : True
Hostname                           : lynx09.lab.eng.tlv2.redhat.com
Host ID                            : 1
Engine status                      : {"health": "good", "vm": "up", "detail": "Up"}
Score                              : 3400
stopped                            : False
Local maintenance                  : False
crc32                              : b57192c9
local_conf_timestamp               : 1024800
Host timestamp                     : 1024800
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1024800 (Wed Oct  3 14:41:12 2018)
	host-id=1
	score=3400
	vm_conf_refresh_time=1024800 (Wed Oct  3 14:41:12 2018)
	conf_on_shared_storage=True
	maintenance=False
	state=EngineUp
	stopped=False


--== Host 2 status ==--

conf_on_shared_storage             : True
Status up-to-date                  : True
Hostname                           : lynx11.lab.eng.tlv2.redhat.com
Host ID                            : 2
Engine status                      : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}
Score                              : 3400
stopped                            : False
Local maintenance                  : False
crc32                              : 8516fbc6
local_conf_timestamp               : 1024808
Host timestamp                     : 1024807
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1024807 (Wed Oct  3 14:41:11 2018)
	host-id=2
	score=3400
	vm_conf_refresh_time=1024808 (Wed Oct  3 14:41:11 2018)
	conf_on_shared_storage=True
	maintenance=False
	state=EngineDown
	stopped=False


--== Host 3 status ==--

conf_on_shared_storage             : True
Status up-to-date                  : True
Hostname                           : lynx10.lab.eng.tlv2.redhat.com
Host ID                            : 3
Engine status                      : {"reason": "vm not running on this host", "health": "bad", "vm": "down", "detail": "unknown"}
Score                              : 3400
stopped                            : False
Local maintenance                  : False
crc32                              : 4925dddd
local_conf_timestamp               : 1024806
Host timestamp                     : 1024806
Extra metadata (valid at timestamp):
	metadata_parse_version=1
	metadata_feature_version=1
	timestamp=1024806 (Wed Oct  3 14:41:07 2018)
	host-id=3
	score=3400
	vm_conf_refresh_time=1024806 (Wed Oct  3 14:41:07 2018)
	conf_on_shared_storage=True
	maintenance=False
	state=EngineDown

Comment 1 Simone Tiraboschi 2018-10-03 13:34:53 UTC
(In reply to Polina from comment #0)
> Steps to Reproduce:
> 1. Migrate the HE VM with no inserting to maintenance either by senting rest
> API or from webUI (happens in both cases) Hosted Storage is GlusterFS. 

This is a libvirt regression introduced with rhel 7.6.
It affects all the VMs on GlusterFS, not just the HE one.

*** This bug has been marked as a duplicate of bug 1632711 ***