Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 907431

Summary: 3.0 - Migration from vdsm-4.9-113 to vdsm-4.9.6-44 fails
Product: Red Hat Enterprise Virtualization Manager Reporter: Tomas Dosek <tdosek>
Component: DocumentationAssignee: Cheryn Tan <chetan>
Status: CLOSED WORKSFORME QA Contact: ecs-bugs
Severity: high Docs Contact:
Priority: high    
Version: 3.0.8CC: abaron, acathrow, asegundo, bazulay, dyasny, hateya, lpeer, michal.skrivanek, mkalinin, rlandman, thildred, ykaul
Target Milestone: ---Keywords: Regression, Reopened
Target Release: 3.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2013-04-06 18:43:31 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Host sosreport - from host with vdsm 4.9.6-44 none

Description Tomas Dosek 2013-02-04 11:32:48 UTC 
Description of problem:
If User performs update of host in cluster to the latest vdsm (4.9.6-44.xx) and tried to migrate VMs from hosts with previous version (4.9-113) the migration fails with libvirt error.

However if the very same VM is migrated to another hosts with equal vdsm version it works fine.

This prevents customers from doing sort of "live upgrade" of hosts even though it should be possible.

If the migration is performed from the host with newer vdsm to the one with older vdsm, it works fine.

Version-Release number of selected component (if applicable):
vdsm-4.9.6-44

How reproducible:
100 %

Steps to Reproduce:
1. Have two hosts - one with vdsm-4.9.113 and one with vdsm-4.9.6-44
2. Run a VM on the host with older vdsm version
3. Try to migrate the VM to the host with newer vdsm version
  
Actual results: Migration fails


Expected results: Migration succeeds


Additional info:

Log message:
Thread-22::ERROR::2013-02-04 06:01:50,396::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 3ecf620a-8fa0-485d-8e19-866e546373af monitoring information
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
StorageDomainDoesNotExist: Storage domain does not exist: (u'3ecf620a-8fa0-485d-8e19-866e546373af',)
Thread-20::ERROR::2013-02-04 06:01:50,397::domainMonitor::208::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain 10426e24-13f6-4a01-a823-f857574e897a monitoring information
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/domainMonitor.py", line 182, in _monitorDomain
  File "/usr/share/vdsm/storage/sdc.py", line 97, in produce
  File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain
  File "/usr/share/vdsm/storage/sdc.py", line 121, in _realProduce
  File "/usr/share/vdsm/storage/sdc.py", line 152, in _findDomain
StorageDomainDoesNotExist: Storage domain does not exist: (u'10426e24-13f6-4a01-a823-f857574e897a',)
Comment 1 Tomas Dosek 2013-02-04 11:41:06 UTC 
Created attachment 692681 [details]
Host sosreport - from host with vdsm 4.9.6-44
Comment 3 Ayal Baron 2013-02-06 07:37:28 UTC 
This looks like a FC domain? (SOLR-FC_DataCentre) but the VG is not visible.  Assuming that the relevant domain is indeed the one that cannot be found this is probably a configuration issue (lun mapping in storage side not set for the new host?)
Comment 4 Tomas Dosek 2013-02-06 07:49:28 UTC 
The lun mapping was set up correctly, the host was only reinstalled with newer RHEV-H iso. When we pushed back the older ISO (did the very same settings) it worked fine.
Comment 6 Michal Skrivanek 2013-03-05 10:25:49 UTC 
the log is missing any date regarding the migration - we can only see errors/warning accessing the storage. If that's the case the problem is not related to migration. Please retest or confirm
Comment 7 Tomas Dosek 2013-03-20 15:34:47 UTC 
The time of issue is 2013-02-04 06:01:50,396 - I was connected to the customer and performing the migration. So we may be sure the time mentioned in Description is correct. No storage issues were to be seen at that time, only change was placing newer vdsm to host.
Comment 10 Saveliev Peter 2013-03-25 10:24:28 UTC 
I see no migration issues in the logs, only OS level error that ISO domain can not be mounted, that's possible storage server and/or client setup issue, not VDSM's one. We couldn't reproduce it in ours mixed 3.1/3.0 setups. Theoretically, there is a little chance it could be caused by 3.1 → 3.0 migration logic, but I do not see how.

Since the original setup right now can not be used to reproduce the bug, I see no possibility to trace it.
Comment 11 Tomas Dosek 2013-03-26 08:41:36 UTC 
As per mailing thread about this issue, I'm changing the bug's component to documentation to document this issue.

Content of vdsm documentation should contain note that would warn user that live migration doesn't have to necessarily work in case various versions of vdsm package are deployed on hosts within the same cluster.

Example:

Cluster C contains host A and host B, host A is equipped with vdsm-4.9-113, host B is equipped with vdsm newer than 4.9.6-44. In this scenario it's expected result that some live migration flows would not work correctly.
Comment 12 Tim Hildred 2013-04-02 02:51:28 UTC 
Hey Tomas,

I'm going to change the component of this bug from rhevm-doc (which is the rpm package that ships with RHEVM to provide context sensitive help) to the Guides component, which is where we track issues requiring documentation.


That being said, we're working hard on getting the 3.2 documentation ready for the upcoming 3.2 release, and won't be looking at bugs against older versions of the documentation for a while.
Comment 17 Itamar Heim 2013-04-06 18:43:31 UTC 
per testing by marina - vdsm 4.9-113 live migration works with vdsm 4.10.1-9