Description of problem: Import export fails when trying to Import exported VM after VM removal (iscsi). http://jenkins.eng.lab.tlv.redhat.com/job/rhevm_3.3_automation_storage_export_import_server_factory/ENGINE=rest,HOST_HOSTGROUP=P-RHEVM-3.3-RHEL64-HOSTS,STORAGE_TYPE=iscsi,label=RHEVM-3.3-ENGINE-RHEL64/227/consoleFull Version-Release number of selected component (if applicable): is9.1 Additional info: Additional logs: http://jenkins.eng.lab.tlv.redhat.com/job/rhevm_3.3_automation_storage_export_import_server_factory/ENGINE=rest,HOST_HOSTGROUP=P-RHEVM-3.3-RHEL64-HOSTS,STORAGE_TYPE=iscsi,label=RHEVM-3.3-ENGINE-RHEL64/227/artifact/logs/jenkins-rhevm_3.3_automation_storage_export_import_server_factory-ENGINE=rest,HOST_HOSTGROUP=P-RHEVM-3.3-RHEL64-HOSTS,STORAGE_TYPE=iscsi,label=RHEVM-3.3-ENGINE-RHEL64-227__08112013_18-30-47.tar.bz2 2013-08-11 16:22:21,368 ERROR [org.ovirt.engine.core.bll.CopyImageGroupCommand] (ajp-/127.0.0.1:8702-5) Error during CreateTask for command: org.ovirt.engine.core.bll.CopyImageGroupCommand. Exception org.apache.commons.lang.Serialization Exception: org.codehaus.jackson.map.JsonMappingException: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.util.Map at [Source: java.io.StringReader@3543fc31; line: 157, column: 29] (through reference chain: org.ovirt.engine.core.common.action.ImportVmTemplateParameters["vmTemplate"]->org.ovirt.engine.core.common.businessentities.VmTemplate["diskMap" ]): org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Unexpected token (START_OBJECT), expected START_ARRAY: need JSON Array to contain As.WRAPPER_ARRAY type information for class java.util.Ma p at [Source: java.io.StringReader@3543fc31; line: 157, column: 29] (through reference chain: org.ovirt.engine.core.common.action.ImportVmTemplateParameters["vmTemplate"]->org.ovirt.engine.core.common.businessentities.VmTemplate["diskMap"]) at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.readJsonString(JsonObjectDeserializer.java:90) [utils.jar:] at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.deserialize(JsonObjectDeserializer.java:57) [utils.jar:] at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl$AsyncTaskRowMapper.deserializeParameters(AsyncTaskDAODbFacadeImpl.java:88) [dal.jar:] at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl$AsyncTaskRowMapper.mapRow(AsyncTaskDAODbFacadeImpl.java:71) [dal.jar:] at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl$AsyncTaskRowMapper.mapRow(AsyncTaskDAODbFacadeImpl.java:60) [dal.jar:] at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92) [spring-jdbc.jar:3.1.3.RELEASE] at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1) [spring-jdbc.jar:3.1.3.RELEASE] at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649) [spring-jdbc.jar:3.1.3.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.3.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.3.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.3.RELEASE] at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.3.RELEASE] at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:] at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.3.RELEASE] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:] at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:] at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl.get(AsyncTaskDAODbFacadeImpl.java:134) [dal.jar:] at org.ovirt.engine.core.bll.CommandBase.getAsyncTask(CommandBase.java:1617) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.concreteCreateTask(CommandBase.java:1600) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.createTaskImpl(CommandBase.java:1571) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.createTask(CommandBase.java:1541) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.createTask(CommandBase.java:1456) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.createTask(CommandBase.java:1506) [bll.jar:] at org.ovirt.engine.core.bll.CopyImageGroupCommand.executeCommand(CopyImageGroupCommand.java:100) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1130) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1215) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:1879) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:151) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:118) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1235) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:365) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runAction(Backend.java:408) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:387) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:628) [bll.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_25] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_25] ...
how reproducible? steps to reproduce?
I still see it on ci
*** Bug 996571 has been marked as a duplicate of this bug. ***
I don't know how reproducible it is, since I only saw it on CI job - maybe eedri can give more info about it. Steps to reproduce: Create iSCSI data center (name=iscsiStorageTest) Create iSCSI export storage domain (name=ImpExp_iscsi) Attach export storage domain to data center Activate export storage domain Create shared ISO domain NFS (name=shared-iso-domain) Attach shared ISO domain to data center iscsiStorageTest Activate shared ISO storage domain Create VM with rhel6 (name=iscsiTestVm) Stop running VM (iscsiTestVm) to create template Create template (name=iscsiTestTempl) Create VM from template (name=iscsiTemplVm) Wait till vm is down Try to export VM (iscsiTemplVm) only to data domain (iscsiStorageTest_data_domain0) Try to export VM (iscsiTemplVm) only to ISO domain (shared-iso-domain) Export VM (iscsiTemplVm) to domain (ImpExp_iscsi) checkVmVolume after VM (iscsiTemplVm) export to domain (ImpExp_iscsi) Remove VM (iscsiTemplVm) from export domain (ImpExp_iscsi) checkVmVolume after VM (iscsiTemplVm) removal from export domain (ImpExp_iscsi) Export VM (iscsiTemplVm) after VMs removal from export domain to domain(ImpExp_iscsi) checkVmVolume after the second VM export (iscsiTemplVm) in domain (ImpExp_iscsi) Export template (iscsiTestTempl) to domain (ImpExp_iscsi) checkTemplateVolume after template (iscsiTestTempl) export to (ImpExp_iscsi) Export VM (iscsiTemplVm) after template export to (ImpExp_iscsi) Wait for VM down status (iscsiTemplVm) Remove VM (iscsiTemplVm) after template export Remove template (iscsiTestTempl) Import exported template (iscsiTestTempl) (import_storagedomain='iscsiStorageTest_data_domain0',export_storagedomain='ImpExp_iscsi') Import exported VM after VM removal (iscsiTemplVm) (import_storagedomain='iscsiStorageTest_data_domain0',export_storagedomain='ImpExp_iscsi') <here it failed>
Working on CI.
*** Bug 996309 has been marked as a duplicate of this bug. ***
Is there plan to backport this bug to ovirt-stable? It is really nasty because a single import operation can corrupt the engine and cause it to fail completely.
the import flow is creating a VDSM task to copy in disks from the a template. a JSON structure (the task details) is being saved to the DB. when the task is done the engine is reading the JSON from DB back and trying to de-serialize it into a java object bad serialized form: { diskMap: [ "java.util.Map", { ... the map} ] } while it should be { diskMap: [ "java.util.HashMap", { ... the map} ] } when this is being de-serialized back the Json mapper know which concrete Map type to create. (in detail java.util.Map is an Abstract class) more of this in on the patch http://gerrit.ovirt.org/18001 * the engine isn't corrupted by this. this only mean the task could not be terminate and be cleared properly i.e affecting the template in question. * workaround - UNTESTED, didn't try this but changing the JSON in db directly might make the de-serialization work: psql engine postgres -tq -c "select task_parameters from async_task where task_id = '{taskId}';" | sed 's/java\.util\.Map/java\.util\.HashMap/' | psql engine postgres -c "update async_task set task_parameters = '$(xargs < /dev/stdio)' where task_id = '{task_id}'
Closing - RHEV 3.3 Released