Bug 1138986

Summary: UploadStreamVDSCommand fails using JsonRpc
Product: [Retired] oVirt Reporter: Maor <mlipchuk>
Component: ovirt-engine-coreAssignee: Piotr Kliczewski <pkliczew>
Status: CLOSED CURRENTRELEASE QA Contact: Jiri Belka <jbelka>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 3.5CC: amureini, bazulay, danken, ecohen, gklein, iheim, laravot, mlipchuk, nsoffer, oourfali, rbalakri, yeylon
Target Milestone: ---Keywords: Regression
Target Release: 3.5.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: infra
Fixed In Version: ovirt-3.5.0_rc2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-17 12:41:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Infra RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1073943    
Attachments:
Description Flags
engine log none

Description Maor 2014-09-07 08:41:26 UTC
Description of problem:
UploadStreamVDSCommand is used to upload a tar file containing the OVF data of all the VMs/Templates in the system
The following exception has occurred:

2014-09-04 18:06:21,739 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] START, UploadStreamVDSCommand(HostName = venus-vdsa.tlv.redhat.com, HostId = 3dc5ba65-2f7a-414c-
ad2b-635cdb1afbc7), log id: 64a2a25c
2014-09-04 18:06:21,740 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] -- executeVdsBrokerCommand:
2014-09-04 18:06:21,740 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] -- parameters:
                spUUID=8d0c16a9-5c3f-480d-ab70-d4b9eefef3a8
                sdUUID=cde3916e-f6f0-414a-a336-903d869463b9
                imageGUID=c0a635a0-f2da-4656-b425-ee4a9adcc9bc
                volUUID=64c41674-1a6e-4812-92a6-838090c8068b
                size=10,240
 
2014-09-04 18:06:21,741 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] Failed in UploadStreamVDS method, for vds: venus-vdsa.tlv.redhat.com; host: venus-vdsa.tlv.redha
t.com
2014-09-04 18:06:21,742 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] Command UploadStreamVDSCommand(HostName = venus-vdsa.tlv.redhat.com, HostId = 3dc5ba65-2f7a-414c
-ad2b-635cdb1afbc7) execution failed. Exception: ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper
2014-09-04 18:06:21,742 INFO  [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] FINISH, UploadStreamVDSCommand, log id: 64a2a25c
2014-09-04 18:06:21,743 ERROR [org.ovirt.engine.core.bll.UploadStreamCommand] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] Command org.ovirt.engine.core.bll.UploadStreamCommand throw Vdc Bll exception. With error message VdcBLLException: java.lang.ClassCastException: org.ovirt.engine.core.vdsbroker.jsonrpc.JsonRpcVdsServer cannot be cast to org.ovirt.engine.core.vdsbroker.vdsbroker.VdsServerWrapper (Failed with error ENGINE and code 5001)
2014-09-04 18:06:21,761 WARN  [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-1) [6ceb24f4] Trying to release a shared lock for key: 8d0c16a9-5c3f-480d-ab70-d4b9eefef3a8OVF_UPDATE , but lock does not exist

 
Version-Release number of selected component (if applicable):


How reproducible:
100%

Steps to Reproduce:
1. Configure the host to use json
2. Set the configuration value in the DB for uploading the tar file to two minutes instead of 60 minutes (So it would be more easy to reproduce) 
OvfUpdateIntervalInMinutes in table vdc_options
3.

Actual results:
5001 exception occurred, No update for the OVF file happened

Expected results:
No exception, and the OVF should be updated.

Additional info:

Comment 1 Maor 2014-09-07 08:56:06 UTC
Created attachment 935121 [details]
engine log

Comment 2 Nir Soffer 2014-09-07 19:58:45 UTC
This command should never use jsonrpc. It uses the http server in vdsm to stream raw data directly to an image.

This is not rpc call, and jsonrpc should not be involved in such request.

Liron, can you take a look at this?

Comment 3 Piotr Kliczewski 2014-09-08 06:41:32 UTC
This command is using http client. The engine code uses vds communication interface and cast it to xmlrpc implementation to get underlying http client.
It is xmlrpc specific behavior.

Jsonrpc do not use http client at all but the code tries to cast jsonrpc to xmlrpc implementation and the code fails.

Comment 4 Jiri Belka 2014-09-17 12:01:43 UTC
What are verification steps please? Those ones in #0 seem to be incomplete to me.

Comment 5 Maor 2014-09-17 13:06:53 UTC
It is a scheduler that runs every time.
You need to create a Storage Domain on Data Center 3.5
Add a VM and a disk
Wait the time configured in the vdc_options
open the engine.log, and check for calls to UploadStreamVDSCommand.

in comment#0 there are errors when calling it, to verify the fix you should see the call without errors

Comment 6 Jiri Belka 2014-09-18 14:51:21 UTC
ok, no UploadStreamVDSCommand errors in the log.

vdsm-4.16.4-0.el6.x86_64

Comment 7 Sandro Bonazzola 2014-10-17 12:41:04 UTC
oVirt 3.5 has been released and should include the fix for this issue.