Bug 985008 - Failed to create a VM from a template, image is in locked state forever
Failed to create a VM from a template, image is in locked state forever
Status: CLOSED CURRENTRELEASE
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine (Show other bugs)
3.3.0
Unspecified Unspecified
unspecified Severity urgent
: ---
: 3.3.0
Assigned To: Roy Golan
Tareq Alayan
virt
: Regression, TestBlocker
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-16 10:43 EDT by Tareq Alayan
Modified: 2015-09-22 09 EDT (History)
13 users (show)

See Also:
Fixed In Version: is14
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2014-01-21 17:16:18 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
abaron: Triaged+


Attachments (Terms of Use)
xml for create (1.72 KB, text/plain)
2013-07-16 10:44 EDT, Tareq Alayan
no flags Details
rest xmls (130.65 KB, text/x-log)
2013-07-16 10:45 EDT, Tareq Alayan
no flags Details
engin_n_vdsm_libvirt.logs (804.21 KB, application/x-bzip2)
2013-07-16 10:46 EDT, Tareq Alayan
no flags Details
xml to create vm (1.88 KB, text/plain)
2013-07-17 09:15 EDT, Tareq Alayan
no flags Details
engine.log (1019.09 KB, text/x-log)
2013-07-17 09:16 EDT, Tareq Alayan
no flags Details
enginelog (15.32 MB, text/x-log)
2013-07-17 09:18 EDT, Tareq Alayan
no flags Details
logs (2.33 MB, application/x-bzip2)
2013-08-26 07:59 EDT, Tareq Alayan
no flags Details
engine log after reproducing the bug manually (829.32 KB, text/x-log)
2013-08-27 11:00 EDT, sefi litmanovich
no flags Details
vdsm log from rose07 host (3.53 MB, text/x-log)
2013-08-27 11:02 EDT, sefi litmanovich
no flags Details
vdsm log from monique05 host (472.50 KB, text/x-log)
2013-08-27 11:02 EDT, sefi litmanovich
no flags Details


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 18009 None None None Never
oVirt gerrit 18805 None None None Never

  None (edit)
Description Tareq Alayan 2013-07-16 10:43:34 EDT
Description of problem:
VM is not created.
its state is 'image locked'

Version-Release number of selected component (if applicable):
is5

How reproducible:
always

Steps to Reproduce:
1. create a vm, install rhel
2. create a template out of the vm
3. create a new vm based on template created in step 1

Actual results:
vm with image lock state

Expected results:
vm succesfully created

Additional info:
engine, vdsm, rest xmls attached
Comment 1 Tareq Alayan 2013-07-16 10:44:37 EDT
Created attachment 774354 [details]
xml for create
Comment 2 Tareq Alayan 2013-07-16 10:45:12 EDT
Created attachment 774355 [details]
rest xmls
Comment 3 Tareq Alayan 2013-07-16 10:46:18 EDT
Created attachment 774356 [details]
engin_n_vdsm_libvirt.logs
Comment 4 Ayal Baron 2013-07-17 05:21:02 EDT
Tareq, your log is full of errors that are unrelated or from different tests. in order to correlate the relevant problems with what I see in the logs I need to know when you ran the operation that failed.
There are errors in vdsm.log that have already been solved, others that have nothing to do with storage etc.
Comment 5 Tareq Alayan 2013-07-17 09:14:10 EDT
attaching
1. XML for creating the vm out of the template.
2. engine.log


snippet from log:
================
2013-07-17 15:13:41,096 ERROR [org.ovirt.engine.core.bll.CreateSnapshotFromTemplateCommand] (ajp-/127.0.0.1:8702-2) Error during CreateTask for command: org.ovirt.engine.core.bll.CreateSnaps
hotFromTemplateCommand. Exception org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Can not deserialize instance of java.util.LinkedHashMap out o
f START_ARRAY token
 at [Source: java.io.StringReader@19cdf9a4; line: 332, column: 47] (through reference chain: org.ovirt.engine.core.common.action.VmManagementParametersBase["vmPayload"]->org.ovirt.engine.cor
e.common.businessentities.VmPayload["specParams"]): org.apache.commons.lang.SerializationException: org.codehaus.jackson.map.JsonMappingException: Can not deserialize instance of java.util.L
inkedHashMap out of START_ARRAY token
 at [Source: java.io.StringReader@19cdf9a4; line: 332, column: 47] (through reference chain: org.ovirt.engine.core.common.action.VmManagementParametersBase["vmPayload"]->org.ovirt.engine.cor
e.common.businessentities.VmPayload["specParams"])
        at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.readJsonString(JsonObjectDeserializer.java:93) [utils.jar:]
        at org.ovirt.engine.core.utils.serialization.json.JsonObjectDeserializer.deserialize(JsonObjectDeserializer.java:60) [utils.jar:]
        at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl$AsyncTaskRowMapper.deserializeParameters(AsyncTaskDAODbFacadeImpl.java:88) [dal.jar:]
        at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl$AsyncTaskRowMapper.mapRow(AsyncTaskDAODbFacadeImpl.java:71) [dal.jar:]
        at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl$AsyncTaskRowMapper.mapRow(AsyncTaskDAODbFacadeImpl.java:60) [dal.jar:]
        at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:92) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.springframework.jdbc.core.RowMapperResultSetExtractor.extractData(RowMapperResultSetExtractor.java:1) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:649) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:587) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:637) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:666) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:706) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.executeCallInternal(PostgresDbEngineDialect.java:154) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall.doExecute(PostgresDbEngineDialect.java:120) [dal.jar:]
        at org.springframework.jdbc.core.simple.SimpleJdbcCall.execute(SimpleJdbcCall.java:181) [spring-jdbc.jar:3.1.3.RELEASE]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeImpl(SimpleJdbcCallsHandler.java:147) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadAndReturnMap(SimpleJdbcCallsHandler.java:112) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeReadList(SimpleJdbcCallsHandler.java:103) [dal.jar:]
        at org.ovirt.engine.core.dal.dbbroker.SimpleJdbcCallsHandler.executeRead(SimpleJdbcCallsHandler.java:95) [dal.jar:]
        at org.ovirt.engine.core.dao.AsyncTaskDAODbFacadeImpl.get(AsyncTaskDAODbFacadeImpl.java:134) [dal.jar:]
        at org.ovirt.engine.core.bll.CommandBase.getAsyncTask(CommandBase.java:1585) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.concreteCreateTask(CommandBase.java:1568) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.createTaskImpl(CommandBase.java:1539) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.createTask(CommandBase.java:1509) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.createTask(CommandBase.java:1424) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.createTask(CommandBase.java:1474) [bll.jar:]
        at org.ovirt.engine.core.bll.CreateSnapshotCommand.performImageVdsmOperation(CreateSnapshotCommand.java:118) [bll.jar:]
        at org.ovirt.engine.core.bll.CreateSnapshotCommand.executeCommand(CreateSnapshotCommand.java:57) [bll.jar:]
        at org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1121) [bll.jar:]
Comment 6 Tareq Alayan 2013-07-17 09:15:23 EDT
Created attachment 774772 [details]
xml to create vm
Comment 7 Tareq Alayan 2013-07-17 09:16:12 EDT
Created attachment 774773 [details]
engine.log
Comment 8 Tareq Alayan 2013-07-17 09:18:20 EDT
Created attachment 774784 [details]
enginelog
Comment 12 Michal Skrivanek 2013-08-07 09:07:40 EDT
can you try in is9? is5 cloud-init wasn't fully in and it was touching payload a lot...
Comment 13 Tareq Alayan 2013-08-12 09:42:00 EDT
still reproducible in 9.1.
Comment 14 Roy Golan 2013-08-13 05:24:59 EDT
root cause - Json serialization exception becuase getSpecParams type is Map
see Bug 996005
Json serialization issues are surfacing now. An Infra change for 3.3 is now serializing and de-serializing the task parameters during a command with tasks flow so we hit this at runtime and not only at backend booting up and loading all tasks from db
Comment 15 sefi litmanovich 2013-08-22 09:32:08 EDT
failed qa on is11
Comment 16 Omer Frenkel 2013-08-25 04:43:40 EDT
how did you test and what was the result?
please also attach new logs of the test
Comment 17 Michal Skrivanek 2013-08-26 07:29:40 EDT
or link to failed automation job?
Comment 19 Michal Skrivanek 2013-08-26 07:41:16 EDT
well, I don't see a failure, I see:

2013-08-21 21:39:46,647 - MainThread - plmanagement.bugzilla - [32mINFO[0m - Test marked as Skipped due to: Known issue https://bugzilla.redhat.com/show_bug.cgi?id=#985008 ON_QA      - rgolan@redhat.com - Failed to create a VM from a template, image is in locked state forever
Comment 20 Tareq Alayan 2013-08-26 07:59:03 EDT
Created attachment 790448 [details]
logs
Comment 21 Michal Skrivanek 2013-08-26 07:59:47 EDT
the bug above it is actually in the test not matching rest api?
Comment 22 Tareq Alayan 2013-08-26 08:51:36 EDT
Michal, the test ran in 3.2 and failed in 3.3. 
The bug is not in the test. 
Jenkins didn't fully collect the engine logs (bug in test framework)

You can reproduce it manually (steps to reproduce) are mention in description.
Comment 23 Omer Frenkel 2013-08-26 09:47:26 EDT
can't see any error related to add vm or related to this bug in the attached logs.. can you please provide more info on the failure?
Comment 24 sefi litmanovich 2013-08-27 11:00:25 EDT
Created attachment 791030 [details]
engine log after reproducing the bug manually
Comment 25 sefi litmanovich 2013-08-27 11:02:16 EDT
Created attachment 791042 [details]
vdsm log from rose07 host
Comment 26 sefi litmanovich 2013-08-27 11:02:58 EDT
Created attachment 791043 [details]
vdsm log from monique05 host
Comment 27 sefi litmanovich 2013-09-15 03:27:57 EDT
Varified on rhevm 3.3 is14.
Works also with payload included in vm creation (which initially caused the problem)
Comment 28 Itamar Heim 2014-01-21 17:16:18 EST
Closing - RHEV 3.3 Released
Comment 29 Itamar Heim 2014-01-21 17:23:10 EST
Closing - RHEV 3.3 Released

Note You need to log in before you can comment on or make changes to this bug.