This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 1293689 - Leftovers in compensation table
Leftovers in compensation table
Status: CLOSED CURRENTRELEASE
Product: ovirt-engine
Classification: oVirt
Component: BLL.Storage (Show other bugs)
3.6.1.3
Unspecified Unspecified
unspecified Severity medium (vote)
: ovirt-3.6.2
: 3.6.2.5
Assigned To: Allon Mureinik
Karolína Hajná
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-22 12:17 EST by Simone Tiraboschi
Modified: 2016-05-19 21:24 EDT (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-02-18 06:22:07 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Infra
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
oourfali: ovirt‑3.6.z?
rule-engine: planning_ack?
oourfali: devel_ack+
pstehlik: testing_ack+


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
oVirt gerrit 51167 master MERGED core:clean compensation data upon upgrade 2015-12-30 08:10 EST
oVirt gerrit 51169 ovirt-engine-3.6 MERGED core:clean compensation data upon upgrade 2015-12-30 09:33 EST
oVirt gerrit 51170 ovirt-engine-3.6.2 MERGED core:clean compensation data upon upgrade 2015-12-30 09:35 EST

  None (edit)
Description Simone Tiraboschi 2015-12-22 12:17:19 EST
Description of problem:
After upgrading the engine to 3.6.1 some leftover remain in the business_entity_snapshot table.

engine=# select * from business_entity_snapshot;
          id           |      command_id       |    command_type    |       entity_id       |     entity_type     |    entity_snapshot     | snapshot_class | snapshot_type | insertion_order |     started_at     
-----------------------+-----------------------+--------------------+-----------------------+---------------------+------------------------+----------------+---------------+-----------------+--------------------
 5f84f571-cb2b-4186-b8.| ebe288f6-fdf9-4f27-94.| org.ovirt.engine.c.| 77d0d6b1-376f-4ee6-a8.| org.ovirt.engine.co.| [ "org.ovirt.engine.co.| org.ovirt.engi.|             1 |               1 | 2015-12-22 16:41:4.
.3b-35f30a9d2749       |.28-dc7dbfe3754c       |.ore.bll.storage.Ad.|.a3-c998a20e5d69       |.re.common.businesse.|.re.compat.Guid", {    +|.ne.core.compat.|               |                 |.8.703247+00
                       |                       |.dExistingFileStora.|                       |.ntities.StorageDoma.|   "uuid" : "77d0d6b1-3.|..Guid          |               |                 | 
                       |                       |.geDomainCommand    |                       |.inStatic            |.76f-4ee6-a8a3-c998a20e.|                |               |                 | 
                       |                       |                    |                       |                     |.5d69"                 +|                |               |                 | 
                       |                       |                    |                       |                     | } ]                    |                |               |                 | 
 0b333c6b-23f0-486c-95.| ebe288f6-fdf9-4f27-94.| org.ovirt.engine.c.| 77d0d6b1-376f-4ee6-a8.| org.ovirt.engine.co.| [ "org.ovirt.engine.co.| org.ovirt.engi.|             1 |               2 | 2015-12-22 16:41:4.
.64-7dd625a267db       |.28-dc7dbfe3754c       |.ore.bll.storage.Ad.|.a3-c998a20e5d69       |.re.common.businesse.|.re.compat.Guid", {    +|.ne.core.compat.|               |                 |.8.703247+00
                       |                       |.dExistingFileStora.|                       |.ntities.StorageDoma.|   "uuid" : "77d0d6b1-3.|..Guid          |               |                 | 
                       |                       |.geDomainCommand    |                       |.inDynamic           |.76f-4ee6-a8a3-c998a20e.|                |               |                 | 
                       |                       |                    |                       |                     |.5d69"                 +|                |               |                 | 
                       |                       |                    |                       |                     | } ]                    |                |               |                 | 
 aeab1aab-f3a5-4bfb-8a.| ebe288f6-fdf9-4f27-94.| org.ovirt.engine.c.| 00000000-0000-0000-00.| org.ovirt.engine.co.| {                     +| org.ovirt.engi.|             0 |               3 | 2015-12-22 16:41:4.
.24-119ae5a266b7       |.28-dc7dbfe3754c       |.ore.bll.storage.Ad.|.00-000000000000       |.re.common.businesse.|   "availableDiskSize" .|.ne.core.common.|               |                 |.9.83925+00
                       |                       |.dExistingFileStora.|                       |.ntities.StorageDoma.|.: null,               +|..businessentit.|               |                 | 
                       |                       |.geDomainCommand    |                       |.inDynamic           |   "externalStatus" : ".|.ies.StorageDom.|               |                 | 
                       |                       |                    |                       |                     |.Ok",                  +|.ainDynamic     |               |                 | 
                       |                       |                    |                       |                     |   "containsUnregistere.|                |               |                 | 
                       |                       |                    |                       |                     |.dEntities" : false,   +|                |               |                 | 
                       |                       |                    |                       |                     |   "id" : [ "org.ovirt..|                |               |                 | 
                       |                       |                    |                       |                     |.engine.core.compat.Gui.|                |               |                 | 
                       |                       |                    |                       |                     |.d", {                 +|                |               |                 | 
                       |                       |                    |                       |                     |     "uuid" : "00000000.|                |               |                 | 
                       |                       |                    |                       |                     |.-0000-0000-0000-000000.|                |               |                 | 
                       |                       |                    |                       |                     |.000000"               +|                |               |                 | 
                       |                       |                    |                       |                     |   } ],                +|                |               |                 | 
                       |                       |                    |                       |                     |   "usedDiskSize" : nul.|                |               |                 | 
                       |                       |                    |                       |                     |.l,                    +|                |               |                 | 
                       |                       |                    |                       |                     |   "freeDiskPercent" : .|                |               |                 | 
                       |                       |                    |                       |                     |.0.0                   +|                |               |                 | 
                       |                       |                    |                       |                     | }                      |                |               |                 | 
(3 rows)



The correspondent command in command_entities was already dropped.
engine=# select * from command_entities;
 command_id | command_type | root_command_id | command_parameters | command_params_class | created_at | status | callback_enabled | callback_notified | return_value | return_value_class | job_id | step_id | exec
uted | user_id 
------------+--------------+-----------------+--------------------+----------------------+------------+--------+------------------+-------------------+--------------+--------------------+--------+---------+-----
-----+---------
(0 rows)

engine-setup will wait for that to complete:
          The following compensations have been found running in the system:
          org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand org.ovirt.engine.core.common.businessentities.StorageDomainStatic
          org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand org.ovirt.engine.core.common.businessentities.StorageDomainDynamic
          org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand org.ovirt.engine.core.common.businessentities.StorageDomainDynamic
          Would you like to try to wait for that?
          (Answering "no" will stop the upgrade (Yes, No) no
[ ERROR ] Failed to execute stage 'Setup validation': Upgrade cannot be completed; asynchronous tasks or commands or compensations are still running. Please make sure that there are no running tasks before you continue.


Version-Release number of selected component (if applicable):
ovirt-engine.noarch                        3.6.1.3-1.el7.centos       @ovirt-3.6

How reproducible:
Seen more than once.

Steps to Reproduce:
1. upgrade to 3.6.1
2. run engine-setup again just after that
3.

Actual results:
After upgrade to 3.6.1 we found leftovers in the compensation table, this locks next run of engine-setup

Expected results:
No leftovers in the compensation table

Additional info:

There is no foreign key between business_entity_snapshot and command_entities

Engine-setup will delete the zombie commands but here there is no command to delete.
Comment 1 Simone Tiraboschi 2015-12-22 12:22:06 EST
Workaround: In order to execute engine-setup again is necessary to manually delete them from the business_entity_snapshot table.
Comment 2 Tal Nisan 2015-12-23 07:53:05 EST
Although the command in the compensation are storage related, it seems to me that the fact they are not cleared is more infra related, Oved, what do you think?
Comment 3 Oved Ourfali 2015-12-23 07:55:20 EST
Moti/Eli - should those be deleted as part of task cleaner?
Comment 4 Moti Asayag 2015-12-23 08:00:47 EST
(In reply to Oved Ourfali from comment #3)
> Moti/Eli - should those be deleted as part of task cleaner?

It makes sense to clear this table as a condition prior to running the upgrade process.

If not, any persisted class/entity in that table which was changed during the upgrade, will fail deserialization from the db.
Comment 5 Eli Mesika 2015-12-24 07:02:07 EST
(In reply to Moti Asayag from comment #4)
> (In reply to Oved Ourfali from comment #3)
> > Moti/Eli - should those be deleted as part of task cleaner?
> 
> It makes sense to clear this table as a condition prior to running the
> upgrade process.
> 
> If not, any persisted class/entity in that table which was changed during
> the upgrade, will fail deserialization from the db.

Agree
Comment 6 Red Hat Bugzilla Rules Engine 2015-12-30 07:55:47 EST
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Comment 7 Karolína Hajná 2016-02-12 05:45:34 EST
Verified in rhevm-3.6.3-0.1.el6.noarch

Note You need to log in before you can comment on or make changes to this bug.