Attachment 909918 Details for Bug 1110707 – engine.log

engine.log

engine.log (text/plain), 48.59 KB, created by lily on 2014-06-18 09:54:43 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: lily

Created: 2014-06-18 09:54:43 UTC

Size: 48.59 KB

patch

obsolete

>2014-06-18 16:51:21,168 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) [4a2ec0b9] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 26492a7d
>2014-06-18 16:51:21,225 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) [4a2ec0b9] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3279cff9, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@52347923, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f5312a61, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cca8eba9, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3ad02cfd}, log id: 26492a7d
>2014-06-18 16:51:26,257 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) [52c45176] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 11b9a27f
>2014-06-18 16:51:26,314 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-94) [52c45176] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a1ba4b6d, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@da2043e, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b62a2cb8, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a0fa6947, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6ba7a127}, log id: 11b9a27f
>2014-06-18 16:51:31,350 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 38a278c4
>2014-06-18 16:51:31,408 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@778a9efa, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@777ee21, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6cc2324, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1943249, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7aa56ab9}, log id: 38a278c4
>2014-06-18 16:51:36,441 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 35cdd46b
>2014-06-18 16:51:36,498 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-90) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@aefef0d6, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9624d9b0, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@174bed50, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@317c91, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7ad68f2}, log id: 35cdd46b
>2014-06-18 16:51:41,530 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) [760b38d4] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 44ff2725
>2014-06-18 16:51:41,589 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-51) [760b38d4] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6b0e2296, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5a17dda3, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@371b3b98, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@303e8d67, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ff92353}, log id: 44ff2725
>2014-06-18 16:51:46,627 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) [47e39e7c] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 8cb4bb6
>2014-06-18 16:51:46,705 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-21) [47e39e7c] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1802c584, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b04852cd, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@94085b43, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f4e401d6, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@562a3b4d}, log id: 8cb4bb6
>2014-06-18 16:51:47,416 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-87) START, GlusterTasksListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: dd44ab
>2014-06-18 16:51:47,460 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-87) Command GlusterTasksListVDS execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterTasksList" is not supported
>2014-06-18 16:51:47,461 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-87) FINISH, GlusterTasksListVDSCommand, log id: dd44ab
>2014-06-18 16:51:47,461 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-87) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException
>        at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.7.0_45]
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
>        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
>        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
>        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
>Caused by: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterTasksList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022)
>        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:122) [bll.jar:]
>        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.runVdsCommand(GlusterTasksService.java:60) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:28) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateTasksInCluster(GlusterTasksSyncJob.java:67) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:56) [bll.jar:]
>        ... 6 more
>
>2014-06-18 16:51:51,737 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 24552aee
>2014-06-18 16:51:51,794 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-56) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@68256c37, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8cf156d2, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6615d06e, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@92aeb54b, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@af5ce6fe}, log id: 24552aee
>2014-06-18 16:51:56,827 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 7385bb08
>2014-06-18 16:51:56,887 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-45) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@495dbc06, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cb3de4ee, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5fcebd96, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e7f891ba, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@462f68ab}, log id: 7385bb08
>2014-06-18 16:52:01,915 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) [4a3709dd] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 6fc431e9
>2014-06-18 16:52:01,973 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-9) [4a3709dd] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c7fb5528, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bf25841, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e522ebe3, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cac39f2d, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5b0186e6}, log id: 6fc431e9
>2014-06-18 16:52:07,010 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 172cfde7
>2014-06-18 16:52:07,067 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-40) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a616c12d, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d1bd033, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@87dc29be, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f9fea819, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4ddf8abc}, log id: 172cfde7
>2014-06-18 16:52:12,104 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-54) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 72cc758e
>2014-06-18 16:52:12,161 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-54) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f5794422, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@7d26eafd, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8ff09573, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d758761f, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@96b03a7e}, log id: 72cc758e
>2014-06-18 16:52:17,200 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 7a6610c6
>2014-06-18 16:52:17,257 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-64) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c3971a7a, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bbc46fba, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@dba31bf, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5f5c171d, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a5c8d145}, log id: 7a6610c6
>2014-06-18 16:52:22,292 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) [3b6cadf2] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 59fa76b5
>2014-06-18 16:52:22,348 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-57) [3b6cadf2] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@cb98d6fa, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@75b75aa8, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e5cb9dab, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@55ff414f, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5cfa74b8}, log id: 59fa76b5
>2014-06-18 16:52:27,381 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) [7d2fa683] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 2754a9e0
>2014-06-18 16:52:27,441 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-91) [7d2fa683] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@376647f2, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f538731d, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@91730320, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3abde196, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@faeaf24}, log id: 2754a9e0
>2014-06-18 16:52:27,567 INFO  [org.ovirt.engine.core.bll.LoginAdminUserCommand] (ajp--127.0.0.1-8702-5) Running command: LoginAdminUserCommand internal: false.
>2014-06-18 16:52:27,599 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-5) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
>2014-06-18 16:52:27,655 INFO  [org.ovirt.engine.api.restapi.security.auth.LoginValidator] (ajp--127.0.0.1-8702-17) Validating session failed, reason: Session does not exist.
>2014-06-18 16:52:27,665 INFO  [org.ovirt.engine.core.bll.LoginUserCommand] (ajp--127.0.0.1-8702-11) Running command: LoginUserCommand internal: false.
>2014-06-18 16:52:27,688 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-11) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin@internal logged in.
>2014-06-18 16:52:32,480 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) [40ccb04a] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 3bcb8efb
>2014-06-18 16:52:32,539 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-15) [40ccb04a] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@877f1e0b, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5af0cdf6, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a6131bdc, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9367b2ea, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4bfce8af}, log id: 3bcb8efb
>2014-06-18 16:52:37,576 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 2b09288e
>2014-06-18 16:52:37,634 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-26) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1daeeb4a, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8e511caf, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3f642f24, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a69819c8, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@74fca180}, log id: 2b09288e
>2014-06-18 16:52:42,664 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) [41d82be1] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 34d8ff74
>2014-06-18 16:52:42,721 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) [41d82be1] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@529f648d, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@db96f35a, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@72e8ce3a, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@13c1723f, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ecb46286}, log id: 34d8ff74
>2014-06-18 16:52:42,721 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-79) [41d82be1] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@529f648d, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@db96f35a, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@72e8ce3a, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@13c1723f, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ecb46286}, log id: 34d8ff74
>2014-06-18 16:52:47,471 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) START, GlusterTasksListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 6e515db8
>2014-06-18 16:52:47,515 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) Command GlusterTasksListVDS execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterTasksList" is not supported
>2014-06-18 16:52:47,515 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-68) FINISH, GlusterTasksListVDSCommand, log id: 6e515db8
>2014-06-18 16:52:47,516 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-68) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException
>        at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.7.0_45]
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
>        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
>        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
>        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
>        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
>Caused by: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterTasksList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022)
>        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:122) [bll.jar:]
>        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.runVdsCommand(GlusterTasksService.java:60) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:28) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateTasksInCluster(GlusterTasksSyncJob.java:67) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:56) [bll.jar:]
>        ... 6 more
>
>2014-06-18 16:52:47,757 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 6402e5ad
>2014-06-18 16:52:47,813 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-1) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a59741c5, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@db2dbf30, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c2f9d556, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@dd791229, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5b993fa2}, log id: 6402e5ad
>2014-06-18 16:52:52,846 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [616d0f2f] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 76ef1b00
>2014-06-18 16:52:52,904 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-66) [616d0f2f] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@23ac5acc, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b13afc40, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bb53033e, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5fb8dc01, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6c068f54}, log id: 76ef1b00
>2014-06-18 16:52:57,940 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-30) [3621ef2a] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 5b9e5d8c
>2014-06-18 16:52:58,006 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-30) [3621ef2a] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2c75a205, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@5a516d80, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@79b9e5f2, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ff406913, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@39af97ef}, log id: 5b9e5d8c
>2014-06-18 16:53:03,037 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 5a1bb377
>2014-06-18 16:53:03,093 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-10) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a5be88c1, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6e5e63f2, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4fa8a25, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3d68f0b0, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1d620ff1}, log id: 5a1bb377
>2014-06-18 16:53:08,134 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 6f9e823c
>2014-06-18 16:53:08,191 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-50) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6c73b3af, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@c18ca4bb, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f1def99, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b5ab4a73, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1b11b993}, log id: 6f9e823c
>2014-06-18 16:53:13,224 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) [6c140f14] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 5132c615
>2014-06-18 16:53:13,281 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-98) [6c140f14] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3dcb8ffd, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a8f6b336, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8d5ab84d, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@6f7cdcbd, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ed0cbfb8}, log id: 5132c615
>2014-06-18 16:53:18,314 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-12) [e003dc1] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 38d68b5f
>2014-06-18 16:53:18,370 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-12) [e003dc1] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@654a9e3, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@58df5e90, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a0628820, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@413edea5, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@26465abd}, log id: 38d68b5f
>2014-06-18 16:53:23,407 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) [5ed653] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 4f6973a0
>2014-06-18 16:53:23,463 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-13) [5ed653] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e4d04b2b, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@53f331a9, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@820598bb, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a1a6a2, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a550d0dd}, log id: 4f6973a0
>2014-06-18 16:53:28,498 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) [1b616d6] START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 48b59c54
>2014-06-18 16:53:28,583 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-52) [1b616d6] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b835d69f, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@58e6f286, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f4e21917, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f89422d6, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@caac7274}, log id: 48b59c54
>2014-06-18 16:53:33,618 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: e37a0e7
>2014-06-18 16:53:33,676 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-62) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@45af498f, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@43d59c42, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@4298eaec, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ee84674f, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2d1db7af}, log id: e37a0e7
>2014-06-18 16:53:36,913 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (ajp--127.0.0.1-8702-11) [13b96d31] Lock Acquired to object EngineLock [exclusiveLocks= key: 66e4f53b-682a-4182-bfa8-2121cf36291a value: VM
>, sharedLocks= ]
>2014-06-18 16:53:37,036 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [13b96d31] Running command: MigrateVmCommand internal: false. Entities affected :  ID: 66e4f53b-682a-4182-bfa8-2121cf36291a Type: VM
>2014-06-18 16:53:37,050 INFO  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-32) [13b96d31] START, MigrateVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b, vmId=66e4f53b-682a-4182-bfa8-2121cf36291a, srcHost=10.1.110.174, dstVdsId=7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5, dstHost=10.1.110.173:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 5d8f5881
>2014-06-18 16:53:37,051 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-32) [13b96d31] VdsBroker::migrate::Entered (vm_guid=66e4f53b-682a-4182-bfa8-2121cf36291a, srcHost=10.1.110.174, dstHost=10.1.110.173:54321,  method=online
>2014-06-18 16:53:37,052 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-32) [13b96d31] START, MigrateBrokerVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b, vmId=66e4f53b-682a-4182-bfa8-2121cf36291a, srcHost=10.1.110.174, dstVdsId=7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5, dstHost=10.1.110.173:54321, migrationMethod=ONLINE, tunnelMigration=false), log id: 9c8dadc
>2014-06-18 16:53:37,062 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateBrokerVDSCommand] (pool-6-thread-32) [13b96d31] FINISH, MigrateBrokerVDSCommand, log id: 9c8dadc
>2014-06-18 16:53:37,070 INFO  [org.ovirt.engine.core.vdsbroker.MigrateVDSCommand] (pool-6-thread-32) [13b96d31] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 5d8f5881
>2014-06-18 16:53:37,071 ERROR [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [13b96d31] Command org.ovirt.engine.core.bll.MigrateVmCommand throw exception: java.lang.NullPointerException
>
>2014-06-18 16:53:37,073 ERROR [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [13b96d31] Transaction rolled-back for command: org.ovirt.engine.core.bll.MigrateVmCommand.
>2014-06-18 16:53:37,082 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-32) [13b96d31] Correlation ID: 13b96d31, Job ID: b55052b7-383c-4874-961d-1cd62b4c42d2, Call Stack: null, Custom Event ID: -1, Message: Migration failed  (VM: secvm1, Source: su04, Destination: su03).
>2014-06-18 16:53:37,093 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [13b96d31] Lock freed to object EngineLock [exclusiveLocks= key: 66e4f53b-682a-4182-bfa8-2121cf36291a value: VM
>, sharedLocks= ]
>2014-06-18 16:53:38,341 INFO  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) [590906ea] VM secvm1 66e4f53b-682a-4182-bfa8-2121cf36291a moved from MigratingFrom --> Up
>2014-06-18 16:53:38,342 INFO  [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) [590906ea] Adding VM 66e4f53b-682a-4182-bfa8-2121cf36291a to re-run list
>2014-06-18 16:53:38,372 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-69) [590906ea] Rerun vm 66e4f53b-682a-4182-bfa8-2121cf36291a. Called from vds su04
>2014-06-18 16:53:38,376 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] START, MigrateStatusVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b, vmId=66e4f53b-682a-4182-bfa8-2121cf36291a), log id: 31b32171
>2014-06-18 16:53:38,381 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] Failed in MigrateStatusVDS method
>2014-06-18 16:53:38,382 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] Error code migrateErr and error message VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration
>2014-06-18 16:53:38,382 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] Command org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand return value
> StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=12, mMessage=Fatal error during migration]]
>2014-06-18 16:53:38,383 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] HostName = su04
>2014-06-18 16:53:38,383 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] Command MigrateStatusVDS execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to MigrateStatusVDS, error = Fatal error during migration
>2014-06-18 16:53:38,384 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.MigrateStatusVDSCommand] (pool-6-thread-32) [590906ea] FINISH, MigrateStatusVDSCommand, log id: 31b32171
>2014-06-18 16:53:38,404 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-32) [590906ea] Correlation ID: 13b96d31, Job ID: b55052b7-383c-4874-961d-1cd62b4c42d2, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration. Trying to migrate to another Host (VM: secvm1, Source: su04, Destination: su03).
>2014-06-18 16:53:38,416 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [590906ea] Lock Acquired to object EngineLock [exclusiveLocks= key: 66e4f53b-682a-4182-bfa8-2121cf36291a value: VM
>, sharedLocks= ]
>2014-06-18 16:53:38,424 WARN  [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [590906ea] CanDoAction of action MigrateVm failed. Reasons:VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT
>2014-06-18 16:53:38,425 INFO  [org.ovirt.engine.core.bll.MigrateVmCommand] (pool-6-thread-32) [590906ea] Lock freed to object EngineLock [exclusiveLocks= key: 66e4f53b-682a-4182-bfa8-2121cf36291a value: VM
>, sharedLocks= ]
>2014-06-18 16:53:38,459 INFO  [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (pool-6-thread-32) [590906ea] Correlation ID: 13b96d31, Job ID: b55052b7-383c-4874-961d-1cd62b4c42d2, Call Stack: null, Custom Event ID: -1, Message: Migration failed due to Error: Fatal error during migration (VM: secvm1, Source: su04, Destination: su03).
>2014-06-18 16:53:38,710 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) [4a2ec0b9] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 69fba825
>2014-06-18 16:53:38,765 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-33) [4a2ec0b9] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@1402eab7, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@e45be027, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@ff04d79, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@920e45e4, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d403c9f7}, log id: 69fba825
>2014-06-18 16:53:43,800 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) START, GlusterVolumesListVDSCommand(HostName = su03, HostId = 7e7bc89f-5e1b-43a4-b96b-f62eca36cbf5), log id: 7b33625d
>2014-06-18 16:53:43,856 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-36) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9b01b399, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@bcfdd42b, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@3b1da0fc, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@61a38eb4, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@2b4e6c5c}, log id: 7b33625d
>2014-06-18 16:53:47,526 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-11) START, GlusterTasksListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 64da4786
>2014-06-18 16:53:47,570 ERROR [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-11) Command GlusterTasksListVDS execution failed. Exception: VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterTasksList" is not supported
>2014-06-18 16:53:47,571 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler_Worker-11) FINISH, GlusterTasksListVDSCommand, log id: 64da4786
>2014-06-18 16:53:47,571 ERROR [org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl] (DefaultQuartzScheduler_Worker-11) Failed to invoke scheduled method gluster_async_task_poll_event: java.lang.reflect.InvocationTargetException
>        at sun.reflect.GeneratedMethodAccessor69.invoke(Unknown Source) [:1.7.0_45]
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_45]
>        at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_45]
>        at org.ovirt.engine.core.utils.timer.JobWrapper.execute(JobWrapper.java:60) [scheduler.jar:]
>        at org.quartz.core.JobRunShell.run(JobRunShell.java:213) [quartz.jar:]
>        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
>       at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557) [quartz.jar:]
>Caused by: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: org.apache.xmlrpc.XmlRpcException: <type 'exceptions.Exception'>:method "glusterTasksList" is not supported (Failed with error VDS_NETWORK_ERROR and code 5022)
>        at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:122) [bll.jar:]
>        at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.runVdsCommand(GlusterTasksService.java:60) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.tasks.GlusterTasksService.getTaskListForCluster(GlusterTasksService.java:28) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateTasksInCluster(GlusterTasksSyncJob.java:67) [bll.jar:]
>        at org.ovirt.engine.core.bll.gluster.GlusterTasksSyncJob.updateGlusterAsyncTasks(GlusterTasksSyncJob.java:56) [bll.jar:]
>        ... 6 more
>
>2014-06-18 16:53:48,891 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-67) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 4438116d
>2014-06-18 16:53:48,950 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-67) FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9ada1ea9, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@f9b120cf, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@8e0f3c85, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@93cc9796, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@b3a0650}, log id: 4438116d
>2014-06-18 16:53:53,984 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) [387b7577] START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 54b3a9fa
>2014-06-18 16:53:54,042 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-61) [387b7577] FINISH, GlusterVolumesListVDSCommand, return: {b82339f3-8d72-43fc-ab66-4ae5d26719c2=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@a52461d6, 4fe088dd-1462-4b72-b858-39e0048fb25f=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@87618ceb, a9ed1e07-875c-42c5-a007-11d6d0868a79=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@962cdccf, b7db1063-14e3-4c34-9fde-d8bf46431cb0=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d15b4745, 08f74cf1-f3e8-4115-a535-ee2ac824c066=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@114c879d}, log id: 54b3a9fa
>2014-06-18 16:53:59,076 INFO  [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand] (DefaultQuartzScheduler_Worker-81) START, GlusterVolumesListVDSCommand(HostName = su04, HostId = 8dda55a8-b10c-4d3d-b9fd-28e09abf0d1b), log id: 6c22ed7c

Actions: View

Attachments on bug 1110707: 909918 | 909919 | 910203 | 910204