1054108 – engine: DeactivateStorageDomainCommand fails with vdsm error: 'Operation not allowed while SPM is active' because we do not actually send SpmStop while there are unknown tasks

Bug 1054108 - engine: DeactivateStorageDomainCommand fails with vdsm error: 'Operation not allowed while SPM is active' because we do not actually send SpmStop while there are unknown tasks

Summary: engine: DeactivateStorageDomainCommand fails with vdsm error: 'Operation not ...

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Virtualization Manager
Classification:	Red Hat
Component:	ovirt-engine
Sub Component:
Version:	3.2.0
Hardware:	x86_64
OS:	Linux
Priority:	high
Severity:	high
Target Milestone:	---
Target Release:	3.3.3
Assignee:	Liron Aravot
QA Contact:	Aharon Canan
Docs Contact:
URL:
Whiteboard:	storage
Depends On:	921666 1058022
Blocks:
TreeView+	depends on / blocked

Reported:	2014-01-16 09:19 UTC by Tomas Dosek
Modified:	2019-04-28 09:03 UTC (History)
CC List:	16 users (show)
Fixed In Version:	org.ovirt.engine-root-3.3.0-51
Doc Type:	Bug Fix
Doc Text:	Previously, attempts to deactivate the master storage domain in Red Hat Enterprise Virtualization environments in which there were no other domains to which to migrate resulted in an error under certain conditions. The logic used to deactivate storage domains failed to run the SpmStop action on the storage pool manager when zombie tasks were still present. Now, this logic has been revised so that the master storage domain is deactivated only after the storage pool manager has been stopped successfully.
Clone Of:	921666
Environment:
Last Closed:	2014-05-27 09:07:33 UTC
oVirt Team:	Storage
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)

Links
System	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHBA-2014:0547	normal	SHIPPED_LIVE	rhevm 3.3.3 bug fix update	2014-05-27 13:07:15 UTC
oVirt gerrit	22269	None	MERGED	core: order of operations when deactivating the last master domain	2020-06-01 16:04:37 UTC
oVirt gerrit	26051	None	MERGED	core: order of operations when deactivating the last master domain	2020-06-01 16:04:37 UTC

Comment 1 Tomas Dosek 2014-01-16 09:21:21 UTC

Reproduced on:
rhev-guest-tools-iso-3.2-15.noarch                          Tue 22 Oct 2013 07:25:34 PM CEST
rhevm-3.2.3-0.43.el6ev.noarch                               Tue 22 Oct 2013 07:25:56 PM CEST
rhevm-backend-3.2.3-0.43.el6ev.noarch                       Tue 22 Oct 2013 07:25:48 PM CEST
rhevm-cli-3.2.0.9-1.el6ev.noarch                            Tue 22 Oct 2013 07:25:21 PM CEST
rhevm-config-3.2.3-0.43.el6ev.noarch                        Tue 22 Oct 2013 07:25:37 PM CEST
rhevm-dbscripts-3.2.3-0.43.el6ev.noarch                     Tue 22 Oct 2013 07:25:37 PM CEST
rhevm-doc-3.2.1-2.el6eng.noarch                             Tue 22 Oct 2013 07:25:26 PM CEST
rhevm-dwh-3.2.1-2.el6ev.noarch                              Tue 22 Oct 2013 07:34:51 PM CEST
rhevm-genericapi-3.2.3-0.43.el6ev.noarch                    Tue 22 Oct 2013 07:25:34 PM CEST
rhevm-image-uploader-3.2.2-2.el6ev.noarch                   Tue 22 Oct 2013 07:25:22 PM CEST
rhevm-iso-uploader-3.2.2-3.el6ev.noarch                     Tue 22 Oct 2013 07:25:22 PM CEST
rhevm-log-collector-3.2.2-4.el6ev.noarch                    Tue 22 Oct 2013 07:25:22 PM CEST
rhevm-notification-service-3.2.3-0.43.el6ev.noarch          Tue 22 Oct 2013 07:25:47 PM CEST
rhevm-reports-3.2.1-6.el6ev.noarch                          Tue 22 Oct 2013 07:34:53 PM CEST
rhevm-restapi-3.2.3-0.43.el6ev.noarch                       Tue 22 Oct 2013 07:25:34 PM CEST
rhevm-sdk-3.2.1.1-1.el6ev.noarch                            Tue 22 Oct 2013 07:25:20 PM CEST
rhevm-setup-3.2.3-0.43.el6ev.noarch                         Tue 22 Oct 2013 07:22:32 PM CEST
rhevm-spice-client-x64-cab-3.2-13.el6ev.noarch              Tue 22 Oct 2013 07:25:48 PM CEST
rhevm-spice-client-x86-cab-3.2-13.el6ev.noarch              Tue 22 Oct 2013 07:25:55 PM CEST
rhevm-tools-common-3.2.3-0.43.el6ev.noarch                  Tue 22 Oct 2013 07:25:37 PM CEST
rhevm-userportal-3.2.3-0.43.el6ev.noarch                    Tue 22 Oct 2013 07:25:55 PM CEST
rhevm-webadmin-portal-3.2.3-0.43.el6ev.noarch               Tue 22 Oct 2013 07:25:46 PM CEST

Thread-178596::ERROR::2014-01-12 17:06:52,756::task::850::TaskManager.Task::(_setError) Task=`ef41e8b9-449c-4d9f-9c10-d2e6f38a6bcf`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 857, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 41, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 1027, in disconnectStoragePool
    self.validateNotSPM(spUUID)
  File "/usr/share/vdsm/storage/hsm.py", line 310, in validateNotSPM
    raise se.IsSpm(spUUID)
IsSpm: Operation not allowed while SPM is active: ('78074c33-a4a0-44bb-b70b-7a8beada0ff5',)

Attaching relevant engine and vdsm logs in a sec.

Comment 4 Tomas Dosek 2014-01-16 09:28:44 UTC

Reproducer scenario seems to be:

Have an SPM host with tasks in unknown state.
Put the host to maintenance.
Upgrade vdsm and try to activate the host.

Comment 7 Liron Aravot 2014-03-02 07:50:05 UTC

Hi Tomas, Oved
this is merged currently for 3.4,
adding needinfo? on Allon to clarify if we want it also for 3.3 as there's no devel ack at the moment.

Comment 8 Allon Mureinik 2014-03-11 10:13:10 UTC

Pending qa-ack (Aharon - please ack/nack it), yes, we want this backported.

Comment 9 Aharon Canan 2014-03-11 16:02:33 UTC

Allon, back to you following our discussion,

Comment 10 Allon Mureinik 2014-03-11 16:05:40 UTC

My mistake. I missed this was ALREADY a clone.

Aharon - please ack/nack this for 3.3.3.
Liron - if this is acked, we need a backport.

Comment 11 Allon Mureinik 2014-03-16 13:30:34 UTC

(In reply to Allon Mureinik from comment #10)
> My mistake. I missed this was ALREADY a clone.
> 
> Aharon - please ack/nack this for 3.3.3.
> Liron - if this is acked, we need a backport.
qa-ack was given. Liron - please handle backports to ovirt-engine-3.4 and ovirt-engine-3.3

Comment 13 Raz Tamir 2014-05-12 12:43:54 UTC

Verified:
vdsm-4.13.2-0.14.el6ev.x86_64
rhevm-3.3.3-0.51.el6ev.noarch

Create zombie task:
1. On iscsi storage with 200GB create 175GB disk (async task)
2. Delete the task from async_tasks table from db (task become zombie)
3. Put master domain in maintenance

Comment 15 errata-xmlrpc 2014-05-27 09:07:33 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-0547.html

Note You need to log in before you can comment on or make changes to this bug.