Bug 1266250 - Make template from cinder-ceph based VM errors out.
Summary: Make template from cinder-ceph based VM errors out.
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-engine
Classification: oVirt
Component: General
Version: 3.6.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ovirt-3.6.0-ga
: 3.6.0.3
Assignee: Maor
QA Contact: Ori Gofen
URL:
Whiteboard: storage
Depends On: 1264691
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-09-24 21:19 UTC by joseph.blasi
Modified: 2016-05-26 01:49 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-11-27 07:54:45 UTC
oVirt Team: Storage
Embargoed:
amureini: ovirt-3.6.0?
rule-engine: planning_ack?
rule-engine: devel_ack+
rule-engine: testing_ack+


Attachments (Terms of Use)
engine log file (48.03 KB, text/plain)
2015-09-24 21:19 UTC, joseph.blasi
no flags Details
rc2 log (187.09 KB, text/plain)
2015-10-14 15:36 UTC, joseph.blasi
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 47408 0 master MERGED engine: Update command associated entities instantly Never

Description joseph.blasi 2015-09-24 21:19:02 UTC
Created attachment 1076724 [details]
engine log file

Description of problem:
When I try to make an template from an cinder-ceph based VM it processes for some time and then errors out with "Error while executing action New VM Template: Internal Engine Error"

Version-Release number of selected component (if applicable):


How reproducible:
all the time

Steps to Reproduce:
1. Make a new vm with cinder-ceph
2. Install os and shutdown the VM
3. Make a template from the VM

Actual results:

Get a Error while executing action New VM Template: Internal Engine Error and see the copied volume in the cinder store.

The new volume is able to be booted up and ran if attached to a new VM.

Expected results:
To have the VM show in a Templates list.

Additional info:
Log clipping attached

Comment 1 Allon Mureinik 2015-10-14 07:50:30 UTC
Maor - please take a look?

Comment 2 Allon Mureinik 2015-10-14 07:54:34 UTC
Joseph, thanks for reporting this bug!
Can you specify which RPM version you're using?

Comment 3 Maor 2015-10-14 08:09:02 UTC
looks like it is the same issue described in https://bugzilla.redhat.com/1264677

The transaction got timed out  after 10 minutes
2015-09-24 15:54:46,930 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-113) [51809cd7] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{runAsync='true', vmId='48b3a875-94d2-4239-abf7-e42e0e240843', status='ImageLocked', exitStatus='Normal'}), log id: 13d06319
2015-09-24 15:54:46,933 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-113) [51809cd7] FINISH, SetVmStatusVDSCommand, log id: 13d06319
2015-09-24 16:04:46,994 INFO  [org.ovirt.engine.core.bll.storage.CloneCinderDisksCommand] (pool-7-thread-1) [332a1ca] Running command: CloneCinderDisksCommand internal: true.

Comment 4 joseph.blasi 2015-10-14 15:16:23 UTC
After updating to RC2 templates are showing up in the Templates list.

Comment 5 joseph.blasi 2015-10-14 15:36:22 UTC
Created attachment 1082881 [details]
rc2 log

Comment 6 joseph.blasi 2015-10-14 15:38:09 UTC
Trying to start a new VM from an Template leads to a long wait and errors out.

Comment 7 Maor 2015-10-15 07:04:53 UTC
It looks like this is the same issue mentioned at https://bugzilla.redhat.com/1264691
Looking at your logs it looks that you got a transaction timeout after 10 minutes:

2015-10-14 10:15:22,780 INFO  [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-87) [] FINISH, SetVmStatusVDSCommand, log id: 169ec556
2015-10-14 10:25:22,589 ERROR [org.ovirt.engine.core.bll.AddVmFromTemplateCommand] (Transaction Reaper Worker 0) [] Transaction rolled-back for command 'org.ovirt.engine.core.bll.AddVmFromTemplateCommand'.
2015-10-14 10:25:22,639 INFO  [org.ovirt.engine.core.bll.storage.CloneCinderDisksCommand] (pool-7-thread-3) [355b7e7a] Running command: CloneCinderDisksCommand internal: true.
2015-10-14 10:25:22,689 INFO  [org.ovirt.engine.core.bll.storage.CloneSingleCinderDiskCommand] (pool-7-thread-4) [38241192] Running command: CloneSingleCinderDiskCommand internal: true.

Comment 8 Maor 2015-10-15 07:06:43 UTC
Joseph,

Can u please change the summary of the bug to indicate this is now an issue of creating a VM from Template, or open a separate bug on this specific issue.

Comment 9 Maor 2015-10-15 07:09:02 UTC
Moti,

That looks like the same issue at https://bugzilla.redhat.com/1264691, (see https://bugzilla.redhat.com/show_bug.cgi?id=1266250#c7).
Probably worth to check this once BZ1264691 will be solved.

Comment 10 Moti Asayag 2015-10-16 15:06:15 UTC
The root cause of this bug is a difference between the behavior of postgres 8.4 and 9.1.

In the described scenarios (and other similar to this issue as we faced on AddVmTemplate) we have the following sequence:

On The main thread:
Open new transaction (either explicit or as default command's scope):
CommandCoordinatorUtil.executeAsyncCommand(...)
  |
  +-- coco.persistCommandAssociatedEntities(...)
      which internally updates command_assoc_entities table

On the submitted thread we calls:
  CommandBase.setCommandStatus()
  which updates the command_entities table.

The first update of 'command_assoc_entities' table is done on transaction Tx1 a, which is open. While  Tx1 is open, the update for 'command_entities' is executed in other transaction Tx2.

In postgres 9.1, that scenario is permitted without any locking on the command_entities table.

However in postgres 8.4, an update for 'command_entities' isn't permitted, unless the referred table 'command_assoc_entities' is closed.

In order to make the code compliant for both postgres versions, the first update to 'command_assoc_entities' should be immediately committed.

Comment 11 Red Hat Bugzilla Rules Engine 2015-10-19 10:57:11 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 12 Maor 2015-10-19 15:26:41 UTC
Verified with patch 47408.
Moving to modify

Comment 13 Allon Mureinik 2015-10-29 09:03:08 UTC
(In reply to Maor from comment #12)
> Verified with patch 47408.
> Moving to modify
Moving to ON_QA by that logic.

Comment 14 Red Hat Bugzilla Rules Engine 2015-11-02 12:27:04 UTC
Bug tickets that are moved to testing must have target release set to make sure tester knows what to test. Please set the correct target release before moving to ON_QA.

Comment 15 Ori Gofen 2015-11-11 15:02:22 UTC
Verified on rhevm-3.6-0.3

Comment 16 Sandro Bonazzola 2015-11-27 07:54:45 UTC
Since oVirt 3.6.0 has been released, moving from verified to closed current release.


Note You need to log in before you can comment on or make changes to this bug.