Bug 873775 - rhevm-upgrade: Goes into infinite loop when it fails to kill tasks found in 'business_entity_snapshot' table.
rhevm-upgrade: Goes into infinite loop when it fails to kill tasks found in '...
Status: CLOSED NOTABUG
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine-setup (Show other bugs)
3.1.0
Unspecified Unspecified
urgent Severity unspecified
: ---
: ---
Assigned To: Moran Goldboim
Pavel Stehlik
infra integration
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2012-11-06 11:49 EST by Tareq Alayan
Modified: 2012-11-27 06:25 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2012-11-19 05:48:55 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
upgrade_log (47.25 KB, text/x-log)
2012-11-06 11:50 EST, Tareq Alayan
no flags Details
vdsm.log.28.xz from 6.Nov (708.50 KB, application/x-xz)
2012-11-13 06:48 EST, Tareq Alayan
no flags Details
vdsm.log.29.x from 6.Nov (770.12 KB, application/x-xz)
2012-11-13 06:49 EST, Tareq Alayan
no flags Details
engine.log (423.00 KB, application/x-gzip)
2012-11-13 06:51 EST, Tareq Alayan
no flags Details

  None (edit)
Description Tareq Alayan 2012-11-06 11:49:44 EST
Description of problem:
Unable to upgrade from si22 to si24


Steps to Reproduce:
1. rhevm-upgrade 
2. rhevm-upgrade requests to stop the following tasks:
tasks given by the sql:
psql -U engine -c 'select command_type,entity_type from business_entity_snapshot';

 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_domain_static
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.Network
 org.ovirt.engine.core.bll.storage.RemoveStoragePoolCommand | org.ovirt.engine.core.common.businessentities.storage_pool_iso_map

(a) I say YES. 

(b) and wait ~minutes while rhevm-upgrade tries to clear these tasks. 

(c) rhevm-upgrade fails to stop the tasks and again says that there are tasks need to be stopped.

(d) back to (a) [infinite loop]. 

Expected results:
Stop and exit with appropriate message.

Additional info:
i manually truncated business_entity_snapshot;
then ran rhevm-upgrade successfully.

another info: I have no idea from where these tasks come from, but i guess maybe they are there because i run automatic tests on the machine.
Comment 1 Tareq Alayan 2012-11-06 11:50:59 EST
Created attachment 639447 [details]
upgrade_log
Comment 2 Simon Grinberg 2012-11-08 06:02:39 EST
Tareq, this is the design, to break the loop and exit with appropriate error message you should answer no, did you try that and it failed? 

Looking at your log it looks like it. So the problem is that selecting no does not exit in a clean manner and an error message sending the customer to get support from Red Hat.

 Nov 06 17:32:48 ] Would you like to proceed and try to stop tasks automatically?
(Answering 'no' will stop the upgrade)? (yes|no): 
2012-11-06 17:52:26::ERROR::rhevm-upgrade::1001::root:: Traceback (most recent call last):
  File "/usr/bin/rhevm-upgrade", line 986, in checkRunningTasks
    answerYes = utils.askYesNo(stopTasksQuestion)
  File "/usr/share/ovirt-engine/scripts/common_utils.py", line 867, in askYesNo
    rawAnswer = raw_input(message.read())
EOFError

2012-11-06 17:52:26::DEBUG::rhevm-upgrade::1007::root:: Stopping ovirt-engine service...
Comment 3 Tareq Alayan 2012-11-12 12:14:32 EST
when i said no, it broke the loop and exit.
Comment 6 Tareq Alayan 2012-11-13 06:48:55 EST
Created attachment 644069 [details]
vdsm.log.28.xz from 6.Nov
Comment 7 Tareq Alayan 2012-11-13 06:49:53 EST
Created attachment 644070 [details]
vdsm.log.29.x from 6.Nov
Comment 8 Tareq Alayan 2012-11-13 06:51:34 EST
Created attachment 644071 [details]
engine.log
Comment 13 Itamar Heim 2012-11-19 05:48:55 EST
closing on this issue.
simon, if you want to change the exit message, please open a new bug specifically for this.

Note You need to log in before you can comment on or make changes to this bug.