Description of problem: After upgrading the engine to 3.6.1 some leftover remain in the business_entity_snapshot table. engine=# select * from business_entity_snapshot; id | command_id | command_type | entity_id | entity_type | entity_snapshot | snapshot_class | snapshot_type | insertion_order | started_at -----------------------+-----------------------+--------------------+-----------------------+---------------------+------------------------+----------------+---------------+-----------------+-------------------- 5f84f571-cb2b-4186-b8.| ebe288f6-fdf9-4f27-94.| org.ovirt.engine.c.| 77d0d6b1-376f-4ee6-a8.| org.ovirt.engine.co.| [ "org.ovirt.engine.co.| org.ovirt.engi.| 1 | 1 | 2015-12-22 16:41:4. .3b-35f30a9d2749 |.28-dc7dbfe3754c |.ore.bll.storage.Ad.|.a3-c998a20e5d69 |.re.common.businesse.|.re.compat.Guid", { +|.ne.core.compat.| | |.8.703247+00 | |.dExistingFileStora.| |.ntities.StorageDoma.| "uuid" : "77d0d6b1-3.|..Guid | | | | |.geDomainCommand | |.inStatic |.76f-4ee6-a8a3-c998a20e.| | | | | | | | |.5d69" +| | | | | | | | | } ] | | | | 0b333c6b-23f0-486c-95.| ebe288f6-fdf9-4f27-94.| org.ovirt.engine.c.| 77d0d6b1-376f-4ee6-a8.| org.ovirt.engine.co.| [ "org.ovirt.engine.co.| org.ovirt.engi.| 1 | 2 | 2015-12-22 16:41:4. .64-7dd625a267db |.28-dc7dbfe3754c |.ore.bll.storage.Ad.|.a3-c998a20e5d69 |.re.common.businesse.|.re.compat.Guid", { +|.ne.core.compat.| | |.8.703247+00 | |.dExistingFileStora.| |.ntities.StorageDoma.| "uuid" : "77d0d6b1-3.|..Guid | | | | |.geDomainCommand | |.inDynamic |.76f-4ee6-a8a3-c998a20e.| | | | | | | | |.5d69" +| | | | | | | | | } ] | | | | aeab1aab-f3a5-4bfb-8a.| ebe288f6-fdf9-4f27-94.| org.ovirt.engine.c.| 00000000-0000-0000-00.| org.ovirt.engine.co.| { +| org.ovirt.engi.| 0 | 3 | 2015-12-22 16:41:4. .24-119ae5a266b7 |.28-dc7dbfe3754c |.ore.bll.storage.Ad.|.00-000000000000 |.re.common.businesse.| "availableDiskSize" .|.ne.core.common.| | |.9.83925+00 | |.dExistingFileStora.| |.ntities.StorageDoma.|.: null, +|..businessentit.| | | | |.geDomainCommand | |.inDynamic | "externalStatus" : ".|.ies.StorageDom.| | | | | | | |.Ok", +|.ainDynamic | | | | | | | | "containsUnregistere.| | | | | | | | |.dEntities" : false, +| | | | | | | | | "id" : [ "org.ovirt..| | | | | | | | |.engine.core.compat.Gui.| | | | | | | | |.d", { +| | | | | | | | | "uuid" : "00000000.| | | | | | | | |.-0000-0000-0000-000000.| | | | | | | | |.000000" +| | | | | | | | | } ], +| | | | | | | | | "usedDiskSize" : nul.| | | | | | | | |.l, +| | | | | | | | | "freeDiskPercent" : .| | | | | | | | |.0.0 +| | | | | | | | | } | | | | (3 rows) The correspondent command in command_entities was already dropped. engine=# select * from command_entities; command_id | command_type | root_command_id | command_parameters | command_params_class | created_at | status | callback_enabled | callback_notified | return_value | return_value_class | job_id | step_id | exec uted | user_id ------------+--------------+-----------------+--------------------+----------------------+------------+--------+------------------+-------------------+--------------+--------------------+--------+---------+----- -----+--------- (0 rows) engine-setup will wait for that to complete: The following compensations have been found running in the system: org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand org.ovirt.engine.core.common.businessentities.StorageDomainStatic org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand org.ovirt.engine.core.common.businessentities.StorageDomainDynamic org.ovirt.engine.core.bll.storage.AddExistingFileStorageDomainCommand org.ovirt.engine.core.common.businessentities.StorageDomainDynamic Would you like to try to wait for that? (Answering "no" will stop the upgrade (Yes, No) no [ ERROR ] Failed to execute stage 'Setup validation': Upgrade cannot be completed; asynchronous tasks or commands or compensations are still running. Please make sure that there are no running tasks before you continue. Version-Release number of selected component (if applicable): ovirt-engine.noarch 3.6.1.3-1.el7.centos @ovirt-3.6 How reproducible: Seen more than once. Steps to Reproduce: 1. upgrade to 3.6.1 2. run engine-setup again just after that 3. Actual results: After upgrade to 3.6.1 we found leftovers in the compensation table, this locks next run of engine-setup Expected results: No leftovers in the compensation table Additional info: There is no foreign key between business_entity_snapshot and command_entities Engine-setup will delete the zombie commands but here there is no command to delete.
Workaround: In order to execute engine-setup again is necessary to manually delete them from the business_entity_snapshot table.
Although the command in the compensation are storage related, it seems to me that the fact they are not cleared is more infra related, Oved, what do you think?
Moti/Eli - should those be deleted as part of task cleaner?
(In reply to Oved Ourfali from comment #3) > Moti/Eli - should those be deleted as part of task cleaner? It makes sense to clear this table as a condition prior to running the upgrade process. If not, any persisted class/entity in that table which was changed during the upgrade, will fail deserialization from the db.
(In reply to Moti Asayag from comment #4) > (In reply to Oved Ourfali from comment #3) > > Moti/Eli - should those be deleted as part of task cleaner? > > It makes sense to clear this table as a condition prior to running the > upgrade process. > > If not, any persisted class/entity in that table which was changed during > the upgrade, will fail deserialization from the db. Agree
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
Verified in rhevm-3.6.3-0.1.el6.noarch