Bug 1381418

Summary: Tracebacks are seen in vdsm log when upgrade finishes: Secured object is not in safe state
Product: [oVirt] vdsm Reporter: RamaKasturi <knarra>
Component: CoreAssignee: Tal Nisan <tnisan>
Status: CLOSED DUPLICATE QA Contact: RamaKasturi <knarra>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.18.13CC: amureini, bugs, knarra, nsoffer, tnisan
Target Milestone: ovirt-4.1.0-betaFlags: amureini: ovirt-4.1?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-01-17 13:40:53 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description RamaKasturi 2016-10-04 04:27:23 UTC
Description of problem:
When upgrading a RHV-H node from UI , spm is stopped and due to this trace backs are seen in vdsm logs.

Thread-557::DEBUG::2016-09-30 18:20:25,064::resourceManager::661::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_57ee3a08-004b-02
7b-0395-0000000001d6', Clearing records.
Thread-557::ERROR::2016-09-30 18:20:25,064::utils::375::Storage.StoragePool::(wrapper) Unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 372, in wrapper
    return f(*a, **kw)
  File "/usr/lib/python2.7/site-packages/vdsm/concurrent.py", line 177, in run
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 78, in wrapper
    return method(self, *args, **kwargs)
  File "/usr/share/vdsm/storage/sp.py", line 207, in _upgradePoolDomain
    self._finalizePoolUpgradeIfNeeded()
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 76, in wrapper
    raise SecureError("Secured object is not in safe state")
SecureError: Secured object is not in safe state
b38e7a14-f880-4259-a7dd-3994bae2dbc2::DEBUG::2016-09-30 18:20:25,065::__init__::398::IOProcessClient::(_startCommunication) Communication thread for client ioprocess-7 started
ioprocess communication (22325)::INFO::2016-09-30 18:20:25,067::__init__::447::IOProcess::(_processLogs) Starting ioprocess
ioprocess communication (22325)::INFO::2016-09-30 18:20:25,067::__init__::447::IOProcess::(_processLogs) Starting ioprocess

Version-Release number of selected component (if applicable):
vdsm-4.18.13-1.el7ev.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Upgrade a RHV-H node from UI
2.
3.

Actual results:
Once the upgrade is done, i see that tracebacks mentioned in description are seen in vdsm log.

Expected results:
vdsm log should not throw any trace backs

Additional info:

Comment 1 Nir Soffer 2016-10-04 16:54:12 UTC
These message means that the SPM is stopped when a domain upgrade thread finish.
This may be expected in a multithreaded environment, or may reveal a bug in 
thread synchronization.

RamaKasturi, can you attach complete vdsm log showing the entire flow since the
host was activated?

Comment 3 Nir Soffer 2016-12-22 21:16:52 UTC
This is not related to node, I see it also locally on RHEL 7.3. I think this
is related to a fix we did in 4.0 for stuck upgrades threads.

Comment 4 Tal Nisan 2017-01-17 13:40:53 UTC

*** This bug has been marked as a duplicate of bug 1400534 ***