Hide Forgot
Description of problem: I now have a ovirt cluster which has one engine and two nodes hardware.The nodes were installed by fedora os system,but I now have a issue when add host into the cluster. If I add these two node into the cluster as hosts,it was successful and working fine,then I set the storage domain in the cluster there were have a issue about uuid occur.(The storage type of storage domain is NFS) The issue message is "VDSM $host2 command failed: can not find master domain :u'spUUID=581bd8ed-0388-00ab-0230-0000000003df,msdUUID=067eecb8-df39-48d5-97ae-cebd6ea570e4". I set the storage domain on $host1,and the $host2 will have this issue.
We'll need both engine and VDSM logs to look into this issue.
(In reply to Yaniv Kaul from comment #1) > We'll need both engine and VDSM logs to look into this issue. hello Yaniv,here is the engine log message,I now have no VDSM log. 2016-11-08 15:36:47,550 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (pool-7-thread-2) [7723568f] Failed in 'GetVolumeInfoVDS' method 2016-11-08 15:36:47,551 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (pool-7-thread-2) [7723568f] Command 'GetVolumeInfoVDSCommand(HostName = fedora138, GetVolumeInfoVDSCommandParameters:{expectedEngineErrors='[VolumeDoesNotExist]', runAsync='true', hostId='25f5f5cc-2791-4fe0-8983-0950a1cf8aba', storagePoolId='581bd8ed-0388-00ab-0230-0000000003df', storageDomainId='c8c44eb2-d43c-4653-bf7f-9f14c65e20be', imageGroupId='87861656-4f4d-4e1c-aafd-2504b8b58a40', imageId='c7b3cb09-0aac-4906-93ae-4c9183318411'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetVolumeInfoVDS, error = Volume does not exist: (u'c7b3cb09-0aac-4906-93ae-4c9183318411',), code = 201 2016-11-08 16:35:48,479 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.TimeBoundPollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [3d15bacb] Error: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request 2016-11-08 16:35:48,480 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.TimeBoundPollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [3d15bacb] Exception: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request 2016-11-08 16:35:48,500 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.TimeBoundPollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [3d15bacb] Command 'TimeBoundPollVDSCommand(HostName = fedora156, TimeBoundPollVDSCommandParameters:{runAsync='true', hostId='40d5aca3-2811-44b9-a94c-88c136fe3f88'})' execution failed: java.util.concurrent.ExecutionException: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request 2016-11-08 16:35:58,991 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [1b237522] Command 'PollVDSCommand(HostName = fedora156, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='40d5aca3-2811-44b9-a94c-88c136fe3f88'})' execution failed: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call 2016-11-08 16:35:58,991 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [1b237522] Timeout waiting for VDSM response: Internal timeout occured 2016-11-08 16:36:01,494 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [1b237522] Command 'PollVDSCommand(HostName = fedora156, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='40d5aca3-2811-44b9-a94c-88c136fe3f88'})' execution failed: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call 2016-11-08 16:36:01,494 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [1b237522] Timeout waiting for VDSM response: Internal timeout occured 2016-11-08 16:36:03,767 ERROR [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able to update response for "93ccb7b4-2eb8-4cf0-ac8d-0caa4121a5ec" 2016-11-08 16:36:03,997 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [1b237522] Command 'PollVDSCommand(HostName = fedora156, VdsIdVDSCommandParametersBase:{runAsync='true', hostId='40d5aca3-2811-44b9-a94c-88c136fe3f88'})' execution failed: VDSGenericException: VDSNetworkException: Timeout during xml-rpc call 2016-11-08 16:36:03,997 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (org.ovirt.thread.pool-8-thread-7) [1b237522] Timeout waiting for VDSM response: Internal timeout occured 2016-11-08 16:36:04,500 ERROR [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able to update response for "104d8c7e-ade2-4544-b161-0b6c37abfdb6" 2016-11-08 16:36:04,519 ERROR [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able to update response for "b4a1e720-567a-44c1-bd35-86e01c9c1f7d" 2016-11-08 16:39:09,751 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler10) [41ae1020] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM fedora156 command failed: Message timeout which can be caused by communication issues 2016-11-08 16:39:09,752 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (DefaultQuartzScheduler10) [41ae1020] Command 'ConnectStorageServerVDSCommand(HostName = fedora156, StorageServerConnectionManagementVDSParameters:{runAsync='true', hostId='40d5aca3-2811-44b9-a94c-88c136fe3f88', storagePoolId='581bd8ed-0388-00ab-0230-0000000003df', storageType='NFS', connectionList='[StorageServerConnections:{id='7dee829c-cf4c-4a95-aad0-1ea5ded28e85', connection='192.168.71.138:/var/nfsshare', iqn='null', vfsType='null', mountOptions='null', nfsVersion='V3', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='40ac547d-6fc7-497d-835d-be4ca0ccf762', connection='192.168.71.138:/var/nfsshare2', iqn='null', vfsType='null', mountOptions='null', nfsVersion='V3', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='b2d71c5e-dc84-48be-bc1d-3c12a9575374', connection='192.168.71.138:/var/iso', iqn='null', vfsType='null', mountOptions='null', nfsVersion='V3', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]'})' execution failed: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues 2016-11-08 16:39:09,753 ERROR [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler10) [41ae1020] Command 'org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues (Failed with error VDS_NETWORK_ERROR and code 5022) 2016-11-08 16:39:12,326 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-30) [] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM fedora156 command failed: Cannot find master domain: u'spUUID=581bd8ed-0388-00ab-0230-0000000003df, msdUUID=067eecb8-df39-48d5-97ae-cebd6ea570e4' 2016-11-08 16:39:12,327 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-30) [] Command 'ConnectStoragePoolVDSCommand(HostName = fedora156, ConnectStoragePoolVDSCommandParameters:{runAsync='true', hostId='40d5aca3-2811-44b9-a94c-88c136fe3f88', vdsId='40d5aca3-2811-44b9-a94c-88c136fe3f88', storagePoolId='581bd8ed-0388-00ab-0230-0000000003df', masterVersion='1'})' execution failed: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: u'spUUID=581bd8ed-0388-00ab-0230-0000000003df, msdUUID=067eecb8-df39-48d5-97ae-cebd6ea570e4' 2016-11-08 16:39:12,327 ERROR [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (org.ovirt.thread.pool-8-thread-30) [] Could not connect host 'fedora156' to pool 'Default': null The host1 is "fedora138",host2 is "fedora156" and cluster use default. I first add a storage domain in host "fedora138". thanks
(In reply to hsiuming from comment #0) > Description of problem: > > I now have a ovirt cluster which has one engine and two nodes hardware.The > nodes were installed by fedora os system,but I now have a issue when add > host into the cluster. > If I add these two node into the cluster as hosts,it was successful and > working fine,then I set the storage domain in the cluster there were have a > issue about uuid occur.(The storage type of storage domain is NFS) > > The issue message is "VDSM $host2 command failed: can not find master domain > :u'spUUID=581bd8ed-0388-00ab-0230-0000000003df,msdUUID=067eecb8-df39-48d5- > 97ae-cebd6ea570e4". > > I set the storage domain on $host1,and the $host2 will have this issue. Hi hsiuming, can you please describe the scenario step by step? what do you mean by "set the storage domain"? I'd appreciate if you could attach the engine.log file from the relevant time (the log you've added is too short). thanks, Liron
Created attachment 1222957 [details] enginelog
(In reply to Liron Aravot from comment #3) > (In reply to hsiuming from comment #0) > > Description of problem: > > > > I now have a ovirt cluster which has one engine and two nodes hardware.The > > nodes were installed by fedora os system,but I now have a issue when add > > host into the cluster. > > If I add these two node into the cluster as hosts,it was successful and > > working fine,then I set the storage domain in the cluster there were have a > > issue about uuid occur.(The storage type of storage domain is NFS) > > > > The issue message is "VDSM $host2 command failed: can not find master domain > > :u'spUUID=581bd8ed-0388-00ab-0230-0000000003df,msdUUID=067eecb8-df39-48d5- > > 97ae-cebd6ea570e4". > > > > I set the storage domain on $host1,and the $host2 will have this issue. > > Hi hsiuming, > can you please describe the scenario step by step? what do you mean by "set > the storage domain"? > > I'd appreciate if you could attach the engine.log file from the relevant > time (the log you've added is too short). > > thanks, > Liron Hello Liron,I already upload the log file. We have three machine ,one installed as engine and two as hosts. In engine web,I first set these two host machine as host. In this step it's OK,but when I select one host NFS server path as storage domain,there were error about uuid in another host.
(In reply to hsiuming from comment #5) > (In reply to Liron Aravot from comment #3) > > (In reply to hsiuming from comment #0) > > > Description of problem: > > > > > > I now have a ovirt cluster which has one engine and two nodes hardware.The > > > nodes were installed by fedora os system,but I now have a issue when add > > > host into the cluster. > > > If I add these two node into the cluster as hosts,it was successful and > > > working fine,then I set the storage domain in the cluster there were have a > > > issue about uuid occur.(The storage type of storage domain is NFS) > > > > > > The issue message is "VDSM $host2 command failed: can not find master domain > > > :u'spUUID=581bd8ed-0388-00ab-0230-0000000003df,msdUUID=067eecb8-df39-48d5- > > > 97ae-cebd6ea570e4". > > > > > > I set the storage domain on $host1,and the $host2 will have this issue. > > > > Hi hsiuming, > > can you please describe the scenario step by step? what do you mean by "set > > the storage domain"? > > > > I'd appreciate if you could attach the engine.log file from the relevant > > time (the log you've added is too short). > > > > thanks, > > Liron > > Hello Liron,I already upload the log file. > > We have three machine ,one installed as engine and two as hosts. > > In engine web,I first set these two host machine as host. > > In this step it's OK,but when I select one host NFS server path as storage > domain,there were error about uuid in another host. I find the problem is that I set the storage domain and host-node on the same machine. If the machine which is set as host-node,another machine can't mount the storage domain(NFS type) on the same machine. If I want to set up storage domain and host-node on the same machine,are there any method?
(In reply to hsiuming from comment #6) > (In reply to hsiuming from comment #5) > > (In reply to Liron Aravot from comment #3) > > > (In reply to hsiuming from comment #0) > > > > Description of problem: > > > > > > > > I now have a ovirt cluster which has one engine and two nodes hardware.The > > > > nodes were installed by fedora os system,but I now have a issue when add > > > > host into the cluster. > > > > If I add these two node into the cluster as hosts,it was successful and > > > > working fine,then I set the storage domain in the cluster there were have a > > > > issue about uuid occur.(The storage type of storage domain is NFS) > > > > > > > > The issue message is "VDSM $host2 command failed: can not find master domain > > > > :u'spUUID=581bd8ed-0388-00ab-0230-0000000003df,msdUUID=067eecb8-df39-48d5- > > > > 97ae-cebd6ea570e4". > > > > > > > > I set the storage domain on $host1,and the $host2 will have this issue. > > > > > > Hi hsiuming, > > > can you please describe the scenario step by step? what do you mean by "set > > > the storage domain"? > > > > > > I'd appreciate if you could attach the engine.log file from the relevant > > > time (the log you've added is too short). > > > > > > thanks, > > > Liron > > > > Hello Liron,I already upload the log file. > > > > We have three machine ,one installed as engine and two as hosts. > > > > In engine web,I first set these two host machine as host. > > > > In this step it's OK,but when I select one host NFS server path as storage > > domain,there were error about uuid in another host. > > I find the problem is that I set the storage domain and host-node on the > same machine. > > If the machine which is set as host-node,another machine can't mount the > storage domain(NFS type) on the same machine. > > If I want to set up storage domain and host-node on the same machine,are > there any method? Indeed - please use Gluster for a hyper-converged solution. I'm closing this issue for now, as this (NFS) is not a supported configuration