Bug 1282054 - Migration broken when using jsonrpc client: JsonRpcNoResponseError: [-32605] No response for JSON-RPC VM.migrationCreate request.
Summary: Migration broken when using jsonrpc client: JsonRpcNoResponseError: [-32605] ...
Keywords:
Status: CLOSED DUPLICATE of bug 1274670
Alias: None
Product: vdsm
Classification: oVirt
Component: Bindings-API
Version: 4.17.11
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: ---
: ---
Assignee: Piotr Kliczewski
QA Contact: Aharon Canan
URL:
Whiteboard: virt
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-11-14 17:14 UTC by Nir Soffer
Modified: 2015-11-16 09:48 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-16 08:11:34 UTC
oVirt Team: ---
Embargoed:
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)
vdsm log from the source host (400.71 KB, application/x-xz)
2015-11-14 17:14 UTC, Nir Soffer
no flags Details
vdsm log from the destination host (119.48 KB, application/x-xz)
2015-11-14 17:26 UTC, Nir Soffer
no flags Details
Vdsm log from source host using master (302.05 KB, text/plain)
2015-11-14 17:41 UTC, Nir Soffer
no flags Details
Vdsm log from destination host using master (417.87 KB, text/plain)
2015-11-14 17:41 UTC, Nir Soffer
no flags Details

Description Nir Soffer 2015-11-14 17:14:19 UTC
Created attachment 1094076 [details]
vdsm log from the source host

Description of problem:

Migration fails when using the jsonrpc client to start the migration:

2015-11-14 18:44:37,277 ERROR [virt.vm] (Thread-206) vmId=`b938182d-7f19-435b-9499-4cb8f1578e1b`::Failed to migrate [migration:310(run)]
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/migration.py", line 294, in run
    self._startUnderlyingMigration(time.time())
  File "/usr/share/vdsm/virt/migration.py", line 331, in _startUnderlyingMigration
    result = self._destServer.migrationCreate(self._machineParams)
  File "/usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py", line 119, in migrationCreate
    params)
  File "/usr/lib/python2.7/site-packages/vdsm/jsonrpcvdscli.py", line 101, in _callMethod
    raise JsonRpcNoResponseError(method)
JsonRpcNoResponseError: [-32605] No response for JSON-RPC VM.migrationCreate request.

On the destination host, we do reply to the VM.migrationCreate:

jsonrpc.Executor/6::DEBUG::2015-11-14 18:44:37,261::API::590::vds::(migrationCreate) Destination VM creation succeeded
jsonrpc.Executor/6::DEBUG::2015-11-14 18:44:37,262::__init__::539::jsonrpc.JsonRpcServer::(_serveRequest) Return 'VM.migrationCreate' in bridge with {'migrationPort': 0, 'param
s': {'username': 'Unknown', 'acpiEnable': 'true', 'emulatedMachine': 'pc-i440fx-rhel7.1.0', 'afterMigrationStatus': u'', 'pid': '0', 'memGuaranteedSize': 512, 'transparentHugeP...

Looks like issue receiving the reply from the destination host.

Version-Release number of selected component (if applicable):
master

How reproducible:
Always

Steps to Reproduce:
1. Start 2 hosts using el 7.1 (updated today)
2. Start vm on host 1
3. Try to migrate to other host

Actual results:
Migration fails

Expected results:
Should succeed

Additional info:

Skipping the jsonrpc client for migration fix the migration:

# diff -u /root/src/vdsm/vdsm/virt/migration.py /usr/share/vdsm/virt/migration.py 
--- /root/src/vdsm/vdsm/virt/migration.py	2015-11-04 14:01:36.836210623 +0200
+++ /usr/share/vdsm/virt/migration.py	2015-11-14 18:57:57.866324608 +0200
@@ -140,6 +140,7 @@
         self.remoteHost, port = hostPort.rsplit(':', 1)
 
         try:
+            raise JsonRpcBindingsError()
             client = self._createClient(port)
             requestQueues = config.get('addresses', 'request_queues')
             requestQueue = requestQueues.split(",")[0]
@@ -147,7 +148,8 @@
             self.log.debug('Initiating connection with destination')
             self._destServer.ping()
 
-        except (JsonRpcBindingsError, JsonRpcNoResponseError):
+        except (JsonRpcBindingsError, JsonRpcNoResponseError) as e:
+            self.log.warning("Cannot use jsonrpc: %s, using xmlrpc", e)
             if config.getboolean('vars', 'ssl'):
                 self._destServer = vdscli.connect(
                     hostPort,

2015-11-14 18:59:39,659 INFO [virt.vm] (Thread-45) vmId=`b938182d-7f19-435b-9499-4cb8f1578e1b`::migration took 11 seconds to complete [migration:369(_startUnderlyingMigration)]

Comment 1 Nir Soffer 2015-11-14 17:26:34 UTC
Created attachment 1094100 [details]
vdsm log from the destination host

Comment 2 Nir Soffer 2015-11-14 17:41:21 UTC
Created attachment 1094101 [details]
Vdsm log from source host using master

Comment 3 Nir Soffer 2015-11-14 17:41:59 UTC
Created attachment 1094102 [details]
Vdsm log from destination host using master

Comment 4 Nir Soffer 2015-11-14 17:50:57 UTC
The first logs were using https://gerrit.ovirt.org/48570 on top of master
using vm with non-ascii vm name.

I have seen this failure before, but to be on the safe side I reproduced
again with master using a vm with ascii name.

See attachment 1094101 [details] and attachment 1094102 [details].

Comment 5 Piotr Kliczewski 2015-11-16 08:11:34 UTC

*** This bug has been marked as a duplicate of bug 1274670 ***


Note You need to log in before you can comment on or make changes to this bug.