Bug 844182
| Summary: | [engine-core] reconstruct is not called on engine side in case master domain meta-data is altered | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Haim <hateya> | ||||
| Component: | ovirt-engine | Assignee: | Liron Aravot <laravot> | ||||
| Status: | CLOSED DUPLICATE | QA Contact: | Haim <hateya> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | unspecified | ||||||
| Version: | 3.1.0 | CC: | amureini, dyasny, iheim, lpeer, mkublin, Rhev-m-bugs, sgrinber, yeylon, ykaul | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 3.1.0 | ||||||
| Hardware: | x86_64 | ||||||
| OS: | Linux | ||||||
| Whiteboard: | storage | ||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2012-07-30 11:31:55 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Attachments: |
|
||||||
Created attachment 601048 [details]
engine and vdsm logs
Closing as duplicate , the problem is caused because of an error describe at bug 844306. Reconstruct is not start because we can not find a proxy for vdsm *** This bug has been marked as a duplicate of bug 844306 *** |
Description of problem: 2 hosts topology, 1 storage domain attached and active, change metadata on HSM side (alter master domain) hence cause metadata seal to be broken,engine sends the following to vdsm, getSpmStatus and fails as vdsm returns an error: 2012-07-29 21:56:38,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-88) Failed in SpmStatusVDS method 2012-07-29 21:56:38,941 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-88) Error code StorageDomainDoesNotExist and error message VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Storage domain does not exist: ('4f9154d7-0964-46db-8d88-15ca812eb086',) 2012-07-29 21:56:38,941 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-88) Command SpmStatusVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Storage domain does not exist: ('4f9154d7-0964-46db-8d88-15ca812eb086',) 2012-07-29 21:56:38,945 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (QuartzScheduler_Worker-88) [5658047e] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 77887fd1-3408-44ec-8de4-153948d89cdc Type: StoragePool 2012-07-29 21:56:38,997 ERROR [org.ovirt.engine.core.vdsbroker.ResourceManager] (QuartzScheduler_Worker-88) [5658047e] Cannot get vdsManager for vdsid=78e0bf24-d979-11e1-9e8e-001a4a16970e 2012-07-29 21:56:38,999 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-88) [5658047e] ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: RuntimeException: VdcBLLException: Vds with id: 78e0bf24-d979-11e1-9e8e-001a4a16970e was not found, log id: 386ba6b1: java.lang.RuntimeException: VdcBLLException: Vds with id: 78e0bf24-d979-11e1-9e8e-001a4a16970e was not found at org.ovirt.engine.core.vdsbroker.ResourceManager.CreateCommand(ResourceManager.java:349) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:387) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.gethostFromVds(IrsBrokerCommand.java:652) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.access$400(IrsBrokerCommand.java:124) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData$3.runInTransaction(IrsBrokerCommand.java:550) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData$3.runInTransaction(IrsBrokerCommand.java:547) [engine-vdsbroker.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:168) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:107) [engine-utils.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.getIrsProxy(IrsBrokerCommand.java:547) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.getIrsProxy(IrsBrokerCommand.java:1511) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.ExecuteVDSCommand(IrsBrokerCommand.java:1519) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.VDSCommandBase.ExecuteCommand(VDSCommandBase.java:60) [engine-vdsbroker.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.Execute(VdcCommandBase.java:41) [engine-dal.jar:] at org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:390) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.ProceedStoragePoolStats(IrsBrokerCommand.java:300) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData.access$200(IrsBrokerCommand.java:124) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData$1.runInTransaction(IrsBrokerCommand.java:205) [engine-vdsbroker.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:168) [engine-utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:107) [engine-utils.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand$IrsProxyData._updatingTimer_Elapsed(IrsBrokerCommand.java:195) [engine-vdsbroker.jar:] at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source) [:1.7.0_05-icedtea] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_05-icedtea] at java.lang.reflect.Method.invoke(Method.java:601) [rt.jar:1.7.0_05-icedtea] at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:64) [engine-scheduler.jar:] at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz-2.1.2.jar:] at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz-2.1.2.jar:] Caused by: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: Vds with id: 78e0bf24-d979-11e1-9e8e-001a4a16970e was not found at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.initializeVdsBroker(VdsBrokerCommand.java:52) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.<init>(VdsBrokerCommand.java:45) [engine-vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand.<init>(SpmStatusVDSCommand.java:20) [engine-vdsbroker.jar:] at sun.reflect.GeneratedConstructorAccessor66.newInstance(Unknown Source) [:1.7.0_05-icedtea] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.7.0_05-icedtea] at java.lang.reflect.Constructor.newInstance(Constructor.java:525) [rt.jar:1.7.0_05-icedtea] at org.ovirt.engine.core.vdsbroker.ResourceManager.CreateCommand(ResourceManager.java:344) [engine-vdsbroker.jar:] ... 25 more the end result is SPM is lost on engine side, auto-recover tries to recover pool and send connectStoragePool attempts which fails on wrong master domain on vdsm side, this should trigger reconstruct master, but its not. all logs attached.