Attachment 856008 Details for Bug 1058253 – VDSM log

VDSM log

vdsm.out.txt (text/plain), 453.97 KB, created by Jon Archer on 2014-01-27 10:52:27 UTC

(hide)

Description:

Filename:

MIME Type:

Creator: Jon Archer

Created: 2014-01-27 10:52:27 UTC

Size: 453.97 KB

patch

obsolete

>Thread-35::DEBUG::2014-01-27 10:26:25,444::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000902642 s, 610 kB/s\n'; <rc> = 0
>Thread-2215::INFO::2014-01-27 10:26:25,697::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2215::INFO::2014-01-27 10:26:25,697::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000902642', 'lastCheck': '0.2', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000950069', 'lastCheck': '1.9', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000627245', 'lastCheck': '2.2', 'valid': True}}
>Thread-2217::DEBUG::2014-01-27 10:26:28,725::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2217::DEBUG::2014-01-27 10:26:28,748::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2219::DEBUG::2014-01-27 10:26:29,562::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2219::INFO::2014-01-27 10:26:29,563::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2219::INFO::2014-01-27 10:26:29,574::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2220::DEBUG::2014-01-27 10:26:29,577::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2220::INFO::2014-01-27 10:26:29,578::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2220::INFO::2014-01-27 10:26:29,582::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616768000', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:26:33,481::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:26:33,488::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000748295 s, 469 kB/s\n'; <rc> = 0
>Thread-2222::DEBUG::2014-01-27 10:26:33,797::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2222::DEBUG::2014-01-27 10:26:33,820::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-36::DEBUG::2014-01-27 10:26:33,835::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:26:33,841::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000653321 s, 543 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:26:35,461::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:26:35,468::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000880031 s, 626 kB/s\n'; <rc> = 0
>Thread-2225::DEBUG::2014-01-27 10:26:38,874::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2225::DEBUG::2014-01-27 10:26:38,897::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2226::DEBUG::2014-01-27 10:26:39,603::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2226::INFO::2014-01-27 10:26:39,604::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2226::INFO::2014-01-27 10:26:39,613::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2227::DEBUG::2014-01-27 10:26:39,617::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2227::INFO::2014-01-27 10:26:39,618::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2227::INFO::2014-01-27 10:26:39,623::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616768000', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2228::INFO::2014-01-27 10:26:40,769::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2228::INFO::2014-01-27 10:26:40,769::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000880031', 'lastCheck': '5.3', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000653321', 'lastCheck': '6.9', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000748295', 'lastCheck': '7.3', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:26:43,500::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:26:43,507::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000647216 s, 542 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:26:43,857::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:26:43,863::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000846593 s, 419 kB/s\n'; <rc> = 0
>Thread-2231::DEBUG::2014-01-27 10:26:43,943::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2231::DEBUG::2014-01-27 10:26:43,966::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:26:45,487::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:26:45,494::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000971697 s, 567 kB/s\n'; <rc> = 0
>Thread-2233::DEBUG::2014-01-27 10:26:49,005::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2233::DEBUG::2014-01-27 10:26:49,028::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2234::DEBUG::2014-01-27 10:26:49,645::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2234::INFO::2014-01-27 10:26:49,645::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2234::INFO::2014-01-27 10:26:49,654::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2235::DEBUG::2014-01-27 10:26:49,657::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2235::INFO::2014-01-27 10:26:49,658::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2235::INFO::2014-01-27 10:26:49,661::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616768000', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:26:53,519::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:26:53,526::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000721972 s, 486 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:26:53,879::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:26:53,885::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000939488 s, 378 kB/s\n'; <rc> = 0
>Thread-2238::DEBUG::2014-01-27 10:26:54,072::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2238::DEBUG::2014-01-27 10:26:54,095::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:26:55,514::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:26:55,521::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00107179 s, 514 kB/s\n'; <rc> = 0
>Thread-2239::INFO::2014-01-27 10:26:55,847::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2239::INFO::2014-01-27 10:26:55,848::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00107179', 'lastCheck': '0.3', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000939488', 'lastCheck': '2.0', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000721972', 'lastCheck': '2.3', 'valid': True}}
>Thread-2242::DEBUG::2014-01-27 10:26:59,143::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2242::DEBUG::2014-01-27 10:26:59,166::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2243::DEBUG::2014-01-27 10:26:59,680::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2243::INFO::2014-01-27 10:26:59,681::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2243::INFO::2014-01-27 10:26:59,691::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2244::DEBUG::2014-01-27 10:26:59,695::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2244::INFO::2014-01-27 10:26:59,696::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2244::INFO::2014-01-27 10:26:59,699::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616768000', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:27:03,540::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:27:03,547::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000559236 s, 628 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:27:03,901::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:27:03,908::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000876883 s, 405 kB/s\n'; <rc> = 0
>Thread-2246::DEBUG::2014-01-27 10:27:04,204::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2246::DEBUG::2014-01-27 10:27:04,227::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:27:05,539::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:27:05,546::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000928934 s, 593 kB/s\n'; <rc> = 0
>Thread-2249::DEBUG::2014-01-27 10:27:09,279::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2249::DEBUG::2014-01-27 10:27:09,302::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2250::DEBUG::2014-01-27 10:27:09,722::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2250::INFO::2014-01-27 10:27:09,722::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2250::INFO::2014-01-27 10:27:09,732::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2251::DEBUG::2014-01-27 10:27:09,736::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2251::INFO::2014-01-27 10:27:09,736::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2251::INFO::2014-01-27 10:27:09,740::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616768000', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2252::INFO::2014-01-27 10:27:10,921::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2252::INFO::2014-01-27 10:27:10,921::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000928934', 'lastCheck': '5.4', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000876883', 'lastCheck': '7.0', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000559236', 'lastCheck': '7.4', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:27:13,558::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:27:13,565::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.00059059 s, 594 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:27:13,924::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:27:13,930::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000929699 s, 382 kB/s\n'; <rc> = 0
>Thread-2255::DEBUG::2014-01-27 10:27:14,337::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2255::DEBUG::2014-01-27 10:27:14,360::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:27:15,566::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:27:15,573::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000983726 s, 560 kB/s\n'; <rc> = 0
>Thread-2257::DEBUG::2014-01-27 10:27:19,403::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2257::DEBUG::2014-01-27 10:27:19,427::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2258::DEBUG::2014-01-27 10:27:19,761::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2258::INFO::2014-01-27 10:27:19,762::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2258::INFO::2014-01-27 10:27:19,773::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2259::DEBUG::2014-01-27 10:27:19,778::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2259::INFO::2014-01-27 10:27:19,778::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2259::INFO::2014-01-27 10:27:19,782::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616768000', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2262::DEBUG::2014-01-27 10:27:23,067::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('data', '', '') {}
>Thread-2262::DEBUG::2014-01-27 10:27:23,086::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data', 'pid': '1524', 'port': '49159'}], 'nfs': [{'status': 'ONLINE', 'hostname': '192.168.1.107', 'pid': '1560', 'port': '2049'}], 'shd': [], 'name': 'data'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2263::DEBUG::2014-01-27 10:27:23,096::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('fast_data', '', '') {}
>Thread-2263::DEBUG::2014-01-27 10:27:23,113::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data2', 'pid': '1516', 'port': '49160'}], 'nfs': [{'status': 'ONLINE', 'hostname': '192.168.1.107', 'pid': '1560', 'port': '2049'}], 'shd': [], 'name': 'fast_data'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2264::DEBUG::2014-01-27 10:27:23,122::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('ISO_DOMAIN', '', '') {}
>Thread-2264::DEBUG::2014-01-27 10:27:23,138::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data1/iso', 'pid': '1540', 'port': '49158'}], 'nfs': [{'status': 'ONLINE', 'hostname': '192.168.1.107', 'pid': '1560', 'port': '2049'}], 'shd': [], 'name': 'ISO_DOMAIN'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2265::DEBUG::2014-01-27 10:27:23,148::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('test', '', '') {}
>Thread-2265::DEBUG::2014-01-27 10:27:23,164::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data1/test', 'pid': '1530', 'port': '49161'}], 'nfs': [], 'shd': [], 'name': 'test'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2266::DEBUG::2014-01-27 10:27:23,452::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2266::ERROR::2014-01-27 10:27:23,466::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-39::DEBUG::2014-01-27 10:27:23,577::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:27:23,584::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000688317 s, 510 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:27:23,946::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:27:23,952::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00084758 s, 419 kB/s\n'; <rc> = 0
>Thread-2267::DEBUG::2014-01-27 10:27:24,477::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2267::DEBUG::2014-01-27 10:27:24,501::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:27:25,592::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:27:25,599::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00098381 s, 560 kB/s\n'; <rc> = 0
>Thread-2268::INFO::2014-01-27 10:27:25,996::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2268::INFO::2014-01-27 10:27:25,996::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00098381', 'lastCheck': '0.4', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00084758', 'lastCheck': '2.0', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000688317', 'lastCheck': '2.4', 'valid': True}}
>Thread-2271::DEBUG::2014-01-27 10:27:29,551::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2271::DEBUG::2014-01-27 10:27:29,574::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2272::DEBUG::2014-01-27 10:27:29,804::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107] flowID [3833025d]
>Thread-2272::INFO::2014-01-27 10:27:29,804::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2272::INFO::2014-01-27 10:27:29,814::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2273::DEBUG::2014-01-27 10:27:29,818::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107] flowID [3833025d]
>Thread-2273::INFO::2014-01-27 10:27:29,818::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2273::INFO::2014-01-27 10:27:29,822::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616636928', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:27:33,587::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain abf4ff41-eb0a-4580-9ad0-bae53d56c6b0
>Thread-39::DEBUG::2014-01-27 10:27:33,592::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0
>Thread-39::DEBUG::2014-01-27 10:27:33,594::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-39::DEBUG::2014-01-27 10:27:33,603::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=a13a367c-accd-451a-a2f3-4c83660f1719', 'REMOTE_PATH=cauldron.arclab:/ISO_DOMAIN', 'ROLE=Regular', 'SDUUID=abf4ff41-eb0a-4580-9ad0-bae53d56c6b0', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=f9e9b2005f92982aa61eb5738ed4051f4688ea08']
>Thread-39::DEBUG::2014-01-27 10:27:33,606::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-39::INFO::2014-01-27 10:27:33,606::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace abf4ff41-eb0a-4580-9ad0-bae53d56c6b0_imageNS already registered
>Thread-39::INFO::2014-01-27 10:27:33,606::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace abf4ff41-eb0a-4580-9ad0-bae53d56c6b0_volumeNS already registered
>Thread-39::DEBUG::2014-01-27 10:27:33,615::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:27:33,622::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000824992 s, 425 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:27:33,955::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d5a8d1ae-cfd4-483e-bdeb-9649a1134796
>Thread-36::DEBUG::2014-01-27 10:27:33,965::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796
>Thread-36::DEBUG::2014-01-27 10:27:33,966::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-36::DEBUG::2014-01-27 10:27:33,974::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=fast_data', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=a13a367c-accd-451a-a2f3-4c83660f1719', 'REMOTE_PATH=cauldron.arclab:fast_data', 'ROLE=Regular', 'SDUUID=d5a8d1ae-cfd4-483e-bdeb-9649a1134796', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=f86e05b564f9c2cb497ca896ffd782faa386be82']
>Thread-36::DEBUG::2014-01-27 10:27:33,979::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-36::INFO::2014-01-27 10:27:33,979::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace d5a8d1ae-cfd4-483e-bdeb-9649a1134796_imageNS already registered
>Thread-36::INFO::2014-01-27 10:27:33,979::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace d5a8d1ae-cfd4-483e-bdeb-9649a1134796_volumeNS already registered
>Thread-36::DEBUG::2014-01-27 10:27:33,989::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:27:33,995::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000914887 s, 388 kB/s\n'; <rc> = 0
>Thread-2282::DEBUG::2014-01-27 10:27:34,623::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2282::DEBUG::2014-01-27 10:27:34,646::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:27:35,606::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c074a6e3-87be-445e-890f-601c215ba320
>Thread-35::DEBUG::2014-01-27 10:27:35,612::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320
>Thread-35::DEBUG::2014-01-27 10:27:35,613::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-35::DEBUG::2014-01-27 10:27:35,622::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=data_domain', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=arclab', 'POOL_DOMAINS=c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=a13a367c-accd-451a-a2f3-4c83660f1719', 'REMOTE_PATH=cauldron.arclab:/data', 'ROLE=Master', 'SDUUID=c074a6e3-87be-445e-890f-601c215ba320', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=0c3248047b8256dbf9bb7c8b0be9bb835eb1c779']
>Thread-35::DEBUG::2014-01-27 10:27:35,627::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-35::INFO::2014-01-27 10:27:35,627::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c074a6e3-87be-445e-890f-601c215ba320_imageNS already registered
>Thread-35::INFO::2014-01-27 10:27:35,627::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c074a6e3-87be-445e-890f-601c215ba320_volumeNS already registered
>Thread-35::DEBUG::2014-01-27 10:27:35,635::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:27:35,642::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000968316 s, 569 kB/s\n'; <rc> = 0
>Thread-2289::DEBUG::2014-01-27 10:27:39,683::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2289::DEBUG::2014-01-27 10:27:39,706::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2290::DEBUG::2014-01-27 10:27:39,842::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2290::INFO::2014-01-27 10:27:39,843::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2290::INFO::2014-01-27 10:27:39,854::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2291::DEBUG::2014-01-27 10:27:39,858::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2291::INFO::2014-01-27 10:27:39,859::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2291::INFO::2014-01-27 10:27:39,862::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616636928', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2292::INFO::2014-01-27 10:27:41,082::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2292::INFO::2014-01-27 10:27:41,082::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000968316', 'lastCheck': '5.4', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000914887', 'lastCheck': '7.1', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000824992', 'lastCheck': '7.5', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:27:43,633::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:27:43,640::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000702381 s, 500 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:27:44,011::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:27:44,018::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00102063 s, 348 kB/s\n'; <rc> = 0
>Thread-2295::DEBUG::2014-01-27 10:27:44,757::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2295::DEBUG::2014-01-27 10:27:44,780::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:27:45,662::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:27:45,669::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000897156 s, 614 kB/s\n'; <rc> = 0
>Thread-2297::DEBUG::2014-01-27 10:27:49,825::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2297::DEBUG::2014-01-27 10:27:49,848::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2298::DEBUG::2014-01-27 10:27:49,883::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2298::INFO::2014-01-27 10:27:49,884::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2298::INFO::2014-01-27 10:27:49,895::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2299::DEBUG::2014-01-27 10:27:49,899::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2299::INFO::2014-01-27 10:27:49,899::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2299::INFO::2014-01-27 10:27:49,903::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616636928', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:27:53,652::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:27:53,659::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000707431 s, 496 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:27:54,034::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:27:54,041::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00105117 s, 338 kB/s\n'; <rc> = 0
>Thread-2302::DEBUG::2014-01-27 10:27:54,887::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2302::DEBUG::2014-01-27 10:27:54,909::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:27:55,688::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:27:55,696::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000977105 s, 564 kB/s\n'; <rc> = 0
>Thread-2303::INFO::2014-01-27 10:27:56,154::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2303::INFO::2014-01-27 10:27:56,155::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000977105', 'lastCheck': '0.5', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00105117', 'lastCheck': '2.1', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000707431', 'lastCheck': '2.5', 'valid': True}}
>Thread-2306::DEBUG::2014-01-27 10:27:59,924::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2306::INFO::2014-01-27 10:27:59,927::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2306::INFO::2014-01-27 10:27:59,938::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2307::DEBUG::2014-01-27 10:27:59,942::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2307::INFO::2014-01-27 10:27:59,943::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2307::INFO::2014-01-27 10:27:59,946::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616636928', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2308::DEBUG::2014-01-27 10:27:59,956::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2308::DEBUG::2014-01-27 10:27:59,978::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:28:03,673::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:28:03,680::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000677049 s, 518 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:28:04,056::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:28:04,063::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000839621 s, 423 kB/s\n'; <rc> = 0
>Thread-2310::DEBUG::2014-01-27 10:28:05,030::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2310::DEBUG::2014-01-27 10:28:05,052::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:28:05,713::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:28:05,720::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00100349 s, 549 kB/s\n'; <rc> = 0
>Thread-2313::DEBUG::2014-01-27 10:28:09,965::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2313::INFO::2014-01-27 10:28:09,966::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2313::INFO::2014-01-27 10:28:09,977::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2314::DEBUG::2014-01-27 10:28:09,981::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2314::INFO::2014-01-27 10:28:09,982::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2314::INFO::2014-01-27 10:28:09,985::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616636928', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2315::DEBUG::2014-01-27 10:28:10,098::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2315::DEBUG::2014-01-27 10:28:10,121::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2316::INFO::2014-01-27 10:28:11,228::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2316::INFO::2014-01-27 10:28:11,228::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00100349', 'lastCheck': '5.5', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000839621', 'lastCheck': '7.2', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000677049', 'lastCheck': '7.5', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:28:13,692::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:28:13,699::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000536627 s, 654 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:28:14,078::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:28:14,085::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00101616 s, 349 kB/s\n'; <rc> = 0
>Thread-2319::DEBUG::2014-01-27 10:28:15,171::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2319::DEBUG::2014-01-27 10:28:15,194::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:28:15,740::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:28:15,746::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000797677 s, 691 kB/s\n'; <rc> = 0
>Thread-2321::DEBUG::2014-01-27 10:28:20,007::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2321::INFO::2014-01-27 10:28:20,008::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2321::INFO::2014-01-27 10:28:20,018::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2322::DEBUG::2014-01-27 10:28:20,022::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2322::INFO::2014-01-27 10:28:20,022::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2322::INFO::2014-01-27 10:28:20,029::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616505856', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2323::DEBUG::2014-01-27 10:28:20,233::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2323::DEBUG::2014-01-27 10:28:20,256::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2326::DEBUG::2014-01-27 10:28:23,483::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2326::ERROR::2014-01-27 10:28:23,497::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-39::DEBUG::2014-01-27 10:28:23,711::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:28:23,719::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000719123 s, 488 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:28:24,102::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:28:24,109::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000975616 s, 364 kB/s\n'; <rc> = 0
>Thread-2327::DEBUG::2014-01-27 10:28:25,303::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2327::DEBUG::2014-01-27 10:28:25,326::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:28:25,765::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:28:25,772::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000899621 s, 612 kB/s\n'; <rc> = 0
>Thread-2328::INFO::2014-01-27 10:28:26,298::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2328::INFO::2014-01-27 10:28:26,298::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000899621', 'lastCheck': '0.5', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000975616', 'lastCheck': '2.2', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000719123', 'lastCheck': '2.6', 'valid': True}}
>Thread-2331::DEBUG::2014-01-27 10:28:30,057::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2331::INFO::2014-01-27 10:28:30,057::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2331::INFO::2014-01-27 10:28:30,069::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2332::DEBUG::2014-01-27 10:28:30,079::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2332::INFO::2014-01-27 10:28:30,080::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2332::INFO::2014-01-27 10:28:30,084::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616505856', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2333::DEBUG::2014-01-27 10:28:30,362::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2333::DEBUG::2014-01-27 10:28:30,385::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:28:33,733::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:28:33,740::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000553632 s, 634 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:28:34,125::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:28:34,131::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000931401 s, 381 kB/s\n'; <rc> = 0
>Thread-2336::DEBUG::2014-01-27 10:28:35,423::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2336::DEBUG::2014-01-27 10:28:35,447::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:28:35,789::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:28:35,796::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000820743 s, 671 kB/s\n'; <rc> = 0
>Thread-2338::DEBUG::2014-01-27 10:28:40,105::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2338::INFO::2014-01-27 10:28:40,106::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2338::INFO::2014-01-27 10:28:40,115::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2339::DEBUG::2014-01-27 10:28:40,119::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2339::INFO::2014-01-27 10:28:40,119::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2339::INFO::2014-01-27 10:28:40,124::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616505856', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2340::DEBUG::2014-01-27 10:28:40,495::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2340::DEBUG::2014-01-27 10:28:40,518::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2341::INFO::2014-01-27 10:28:41,370::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2341::INFO::2014-01-27 10:28:41,370::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000820743', 'lastCheck': '5.6', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000931401', 'lastCheck': '7.2', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000553632', 'lastCheck': '7.6', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:28:43,751::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:28:43,758::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000708216 s, 496 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:28:44,147::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:28:44,154::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000937559 s, 379 kB/s\n'; <rc> = 0
>Thread-2344::DEBUG::2014-01-27 10:28:45,568::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2344::DEBUG::2014-01-27 10:28:45,591::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:28:45,815::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:28:45,822::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00088096 s, 625 kB/s\n'; <rc> = 0
>Thread-2346::DEBUG::2014-01-27 10:28:50,147::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2346::INFO::2014-01-27 10:28:50,147::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2346::INFO::2014-01-27 10:28:50,158::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2347::DEBUG::2014-01-27 10:28:50,162::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2347::INFO::2014-01-27 10:28:50,162::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2347::INFO::2014-01-27 10:28:50,167::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616505856', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2349::DEBUG::2014-01-27 10:28:50,627::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2349::DEBUG::2014-01-27 10:28:50,649::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:28:53,770::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:28:53,777::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000632248 s, 555 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:28:54,169::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:28:54,176::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000958132 s, 371 kB/s\n'; <rc> = 0
>Thread-2351::DEBUG::2014-01-27 10:28:55,698::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2351::DEBUG::2014-01-27 10:28:55,721::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-35::DEBUG::2014-01-27 10:28:55,841::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:28:55,847::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000720826 s, 764 kB/s\n'; <rc> = 0
>Thread-2352::INFO::2014-01-27 10:28:56,441::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2352::INFO::2014-01-27 10:28:56,441::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000720826', 'lastCheck': '0.6', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000958132', 'lastCheck': '2.3', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000632248', 'lastCheck': '2.7', 'valid': True}}
>Thread-2355::DEBUG::2014-01-27 10:29:00,189::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2355::INFO::2014-01-27 10:29:00,190::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2355::INFO::2014-01-27 10:29:00,198::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2356::DEBUG::2014-01-27 10:29:00,203::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2356::INFO::2014-01-27 10:29:00,203::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2356::INFO::2014-01-27 10:29:00,208::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616505856', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2357::DEBUG::2014-01-27 10:29:00,769::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2357::DEBUG::2014-01-27 10:29:00,792::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:29:03,791::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:29:03,798::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000644569 s, 545 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:29:04,192::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:29:04,198::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000787489 s, 451 kB/s\n'; <rc> = 0
>Thread-2360::DEBUG::2014-01-27 10:29:05,843::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-35::DEBUG::2014-01-27 10:29:05,861::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:29:05,868::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00064855 s, 850 kB/s\n'; <rc> = 0
>Thread-2360::DEBUG::2014-01-27 10:29:05,868::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2362::DEBUG::2014-01-27 10:29:10,229::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2362::INFO::2014-01-27 10:29:10,230::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2362::INFO::2014-01-27 10:29:10,240::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2363::DEBUG::2014-01-27 10:29:10,245::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2363::INFO::2014-01-27 10:29:10,245::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2363::INFO::2014-01-27 10:29:10,250::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616505856', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2364::DEBUG::2014-01-27 10:29:10,912::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2364::DEBUG::2014-01-27 10:29:10,934::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2365::INFO::2014-01-27 10:29:11,511::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2365::INFO::2014-01-27 10:29:11,512::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00064855', 'lastCheck': '5.6', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000787489', 'lastCheck': '7.3', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000644569', 'lastCheck': '7.7', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:29:13,810::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:29:13,817::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000604376 s, 581 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:29:14,214::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:29:14,220::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00089125 s, 398 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:29:15,890::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:29:15,897::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000842963 s, 654 kB/s\n'; <rc> = 0
>Thread-2368::DEBUG::2014-01-27 10:29:15,982::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2368::DEBUG::2014-01-27 10:29:16,005::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2370::DEBUG::2014-01-27 10:29:20,272::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2370::INFO::2014-01-27 10:29:20,272::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2370::INFO::2014-01-27 10:29:20,285::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2371::DEBUG::2014-01-27 10:29:20,289::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2371::INFO::2014-01-27 10:29:20,290::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2371::INFO::2014-01-27 10:29:20,294::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616374784', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2373::DEBUG::2014-01-27 10:29:21,041::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2373::DEBUG::2014-01-27 10:29:21,064::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2374::DEBUG::2014-01-27 10:29:23,515::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2374::ERROR::2014-01-27 10:29:23,530::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-39::DEBUG::2014-01-27 10:29:23,829::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:29:23,836::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000527908 s, 665 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:29:24,236::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:29:24,243::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000846018 s, 420 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:29:25,916::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:29:25,923::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000728142 s, 757 kB/s\n'; <rc> = 0
>Thread-2376::DEBUG::2014-01-27 10:29:26,100::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2376::DEBUG::2014-01-27 10:29:26,123::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2377::INFO::2014-01-27 10:29:26,588::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2377::INFO::2014-01-27 10:29:26,589::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000728142', 'lastCheck': '0.7', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000846018', 'lastCheck': '2.3', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000527908', 'lastCheck': '2.8', 'valid': True}}
>Thread-2380::DEBUG::2014-01-27 10:29:30,316::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2380::INFO::2014-01-27 10:29:30,316::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2380::INFO::2014-01-27 10:29:30,327::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2381::DEBUG::2014-01-27 10:29:30,331::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2381::INFO::2014-01-27 10:29:30,332::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2381::INFO::2014-01-27 10:29:30,337::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616243712', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2382::DEBUG::2014-01-27 10:29:31,163::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2382::DEBUG::2014-01-27 10:29:31,186::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:29:33,850::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:29:33,857::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000748699 s, 469 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:29:34,259::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:29:34,265::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00102307 s, 347 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:29:35,941::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:29:35,949::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000904791 s, 609 kB/s\n'; <rc> = 0
>Thread-2385::DEBUG::2014-01-27 10:29:36,226::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2385::DEBUG::2014-01-27 10:29:36,249::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2387::DEBUG::2014-01-27 10:29:40,358::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2387::INFO::2014-01-27 10:29:40,359::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2387::INFO::2014-01-27 10:29:40,370::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2388::DEBUG::2014-01-27 10:29:40,375::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2388::INFO::2014-01-27 10:29:40,375::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2388::INFO::2014-01-27 10:29:40,380::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616374784', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2389::DEBUG::2014-01-27 10:29:41,298::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2389::DEBUG::2014-01-27 10:29:41,320::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2390::INFO::2014-01-27 10:29:41,658::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2390::INFO::2014-01-27 10:29:41,658::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000904791', 'lastCheck': '5.7', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00102307', 'lastCheck': '7.4', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000748699', 'lastCheck': '7.8', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:29:43,868::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:29:43,875::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000676699 s, 519 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:29:44,280::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:29:44,287::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000936058 s, 379 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:29:45,967::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:29:45,975::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000955897 s, 576 kB/s\n'; <rc> = 0
>Thread-2393::DEBUG::2014-01-27 10:29:46,369::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2393::DEBUG::2014-01-27 10:29:46,391::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2395::DEBUG::2014-01-27 10:29:50,402::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2395::INFO::2014-01-27 10:29:50,403::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2395::INFO::2014-01-27 10:29:50,413::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2396::DEBUG::2014-01-27 10:29:50,417::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2396::INFO::2014-01-27 10:29:50,418::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2396::INFO::2014-01-27 10:29:50,422::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616374784', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2398::DEBUG::2014-01-27 10:29:51,434::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2398::DEBUG::2014-01-27 10:29:51,458::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:29:53,887::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:29:53,894::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000630954 s, 556 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:29:54,303::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:29:54,309::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00076802 s, 462 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:29:55,994::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:29:56,001::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00101097 s, 545 kB/s\n'; <rc> = 0
>Thread-2400::DEBUG::2014-01-27 10:29:56,495::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2400::DEBUG::2014-01-27 10:29:56,518::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2401::INFO::2014-01-27 10:29:56,730::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2401::INFO::2014-01-27 10:29:56,731::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00101097', 'lastCheck': '0.7', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00076802', 'lastCheck': '2.4', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000630954', 'lastCheck': '2.8', 'valid': True}}
>Thread-2404::DEBUG::2014-01-27 10:30:00,444::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2404::INFO::2014-01-27 10:30:00,445::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2404::INFO::2014-01-27 10:30:00,456::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2405::DEBUG::2014-01-27 10:30:00,461::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2405::INFO::2014-01-27 10:30:00,461::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2405::INFO::2014-01-27 10:30:00,466::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616374784', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2406::DEBUG::2014-01-27 10:30:01,559::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2406::DEBUG::2014-01-27 10:30:01,582::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:30:03,908::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:30:03,915::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000628988 s, 558 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:30:04,324::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:30:04,331::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000903486 s, 393 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:30:06,019::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:30:06,026::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000911142 s, 605 kB/s\n'; <rc> = 0
>Thread-2409::DEBUG::2014-01-27 10:30:06,622::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2409::DEBUG::2014-01-27 10:30:06,645::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2411::DEBUG::2014-01-27 10:30:10,488::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2411::INFO::2014-01-27 10:30:10,489::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2411::INFO::2014-01-27 10:30:10,500::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2412::DEBUG::2014-01-27 10:30:10,505::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2412::INFO::2014-01-27 10:30:10,505::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2412::INFO::2014-01-27 10:30:10,509::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616112640', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2413::DEBUG::2014-01-27 10:30:11,696::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2413::DEBUG::2014-01-27 10:30:11,718::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2414::INFO::2014-01-27 10:30:11,798::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2414::INFO::2014-01-27 10:30:11,798::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000911142', 'lastCheck': '5.8', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000903486', 'lastCheck': '7.5', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000628988', 'lastCheck': '7.9', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:30:13,927::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:30:13,934::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000667544 s, 526 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:30:14,347::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:30:14,354::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000899571 s, 395 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:30:16,046::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:30:16,053::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000939051 s, 587 kB/s\n'; <rc> = 0
>Thread-2417::DEBUG::2014-01-27 10:30:16,769::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2417::DEBUG::2014-01-27 10:30:16,792::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2419::DEBUG::2014-01-27 10:30:20,532::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2419::INFO::2014-01-27 10:30:20,533::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2419::INFO::2014-01-27 10:30:20,543::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2420::DEBUG::2014-01-27 10:30:20,547::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2420::INFO::2014-01-27 10:30:20,548::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2420::INFO::2014-01-27 10:30:20,552::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616112640', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2422::DEBUG::2014-01-27 10:30:21,836::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2422::DEBUG::2014-01-27 10:30:21,859::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2423::DEBUG::2014-01-27 10:30:23,546::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2423::ERROR::2014-01-27 10:30:23,560::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-39::DEBUG::2014-01-27 10:30:23,945::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:30:23,953::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000610903 s, 575 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:30:24,369::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:30:24,376::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000964449 s, 368 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:30:26,073::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:30:26,080::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00101624 s, 542 kB/s\n'; <rc> = 0
>Thread-2425::INFO::2014-01-27 10:30:26,868::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2425::INFO::2014-01-27 10:30:26,870::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00101624', 'lastCheck': '0.8', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000964449', 'lastCheck': '2.5', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000610903', 'lastCheck': '2.9', 'valid': True}}
>Thread-2427::DEBUG::2014-01-27 10:30:26,903::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2427::DEBUG::2014-01-27 10:30:26,925::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2429::DEBUG::2014-01-27 10:30:30,575::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2429::INFO::2014-01-27 10:30:30,576::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2429::INFO::2014-01-27 10:30:30,587::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2430::DEBUG::2014-01-27 10:30:30,592::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2430::INFO::2014-01-27 10:30:30,592::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2430::INFO::2014-01-27 10:30:30,596::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616112640', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2431::DEBUG::2014-01-27 10:30:31,974::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2431::DEBUG::2014-01-27 10:30:31,998::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:30:33,967::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:30:33,974::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000573612 s, 612 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:30:34,392::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:30:34,399::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00104879 s, 338 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:30:36,098::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:30:36,104::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000855751 s, 644 kB/s\n'; <rc> = 0
>Thread-2434::DEBUG::2014-01-27 10:30:37,047::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2434::DEBUG::2014-01-27 10:30:37,071::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2436::DEBUG::2014-01-27 10:30:40,619::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2436::INFO::2014-01-27 10:30:40,620::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2436::INFO::2014-01-27 10:30:40,630::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2437::DEBUG::2014-01-27 10:30:40,634::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2437::INFO::2014-01-27 10:30:40,635::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2437::INFO::2014-01-27 10:30:40,639::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616112640', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2438::INFO::2014-01-27 10:30:41,937::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2438::INFO::2014-01-27 10:30:41,937::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000855751', 'lastCheck': '5.8', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00104879', 'lastCheck': '7.5', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000573612', 'lastCheck': '8.0', 'valid': True}}
>Thread-2440::DEBUG::2014-01-27 10:30:42,119::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2440::DEBUG::2014-01-27 10:30:42,142::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:30:43,986::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:30:43,993::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.00075949 s, 462 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:30:44,415::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:30:44,421::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000939693 s, 378 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:30:46,123::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:30:46,130::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000955321 s, 577 kB/s\n'; <rc> = 0
>Thread-2442::DEBUG::2014-01-27 10:30:47,196::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2442::DEBUG::2014-01-27 10:30:47,219::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2444::DEBUG::2014-01-27 10:30:50,662::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2444::INFO::2014-01-27 10:30:50,662::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2444::INFO::2014-01-27 10:30:50,676::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2445::DEBUG::2014-01-27 10:30:50,680::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2445::INFO::2014-01-27 10:30:50,680::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2445::INFO::2014-01-27 10:30:50,684::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616112640', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2447::DEBUG::2014-01-27 10:30:52,261::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2447::DEBUG::2014-01-27 10:30:52,297::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:30:54,005::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:30:54,014::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000356666 s, 984 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:30:54,440::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:30:54,446::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000748037 s, 475 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:30:56,150::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:30:56,157::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000957792 s, 575 kB/s\n'; <rc> = 0
>Thread-2449::INFO::2014-01-27 10:30:57,024::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2449::INFO::2014-01-27 10:30:57,024::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000957792', 'lastCheck': '0.9', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000748037', 'lastCheck': '2.6', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000356666', 'lastCheck': '3.0', 'valid': True}}
>Thread-2451::DEBUG::2014-01-27 10:30:57,346::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2451::DEBUG::2014-01-27 10:30:57,370::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2453::DEBUG::2014-01-27 10:31:00,705::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2453::INFO::2014-01-27 10:31:00,705::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2453::INFO::2014-01-27 10:31:00,718::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2454::DEBUG::2014-01-27 10:31:00,722::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2454::INFO::2014-01-27 10:31:00,722::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2454::INFO::2014-01-27 10:31:00,727::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67616112640', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2455::DEBUG::2014-01-27 10:31:02,417::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2455::DEBUG::2014-01-27 10:31:02,439::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:31:04,028::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:31:04,035::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000679388 s, 517 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:31:04,462::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:31:04,469::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000886648 s, 400 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:31:06,175::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:31:06,182::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00100275 s, 549 kB/s\n'; <rc> = 0
>Thread-2458::DEBUG::2014-01-27 10:31:07,490::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2458::DEBUG::2014-01-27 10:31:07,514::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2460::DEBUG::2014-01-27 10:31:10,748::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2460::INFO::2014-01-27 10:31:10,749::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2460::INFO::2014-01-27 10:31:10,760::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2461::DEBUG::2014-01-27 10:31:10,765::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2461::INFO::2014-01-27 10:31:10,765::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2461::INFO::2014-01-27 10:31:10,770::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615981568', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2462::INFO::2014-01-27 10:31:12,094::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2462::INFO::2014-01-27 10:31:12,094::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00100275', 'lastCheck': '5.9', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000886648', 'lastCheck': '7.6', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000679388', 'lastCheck': '8.1', 'valid': True}}
>Thread-2464::DEBUG::2014-01-27 10:31:12,564::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2464::DEBUG::2014-01-27 10:31:12,586::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:31:14,047::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:31:14,054::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000497728 s, 705 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:31:14,485::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:31:14,491::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00081398 s, 436 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:31:16,201::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:31:16,208::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000836307 s, 659 kB/s\n'; <rc> = 0
>Thread-2466::DEBUG::2014-01-27 10:31:17,634::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2466::DEBUG::2014-01-27 10:31:17,657::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2468::DEBUG::2014-01-27 10:31:20,792::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2468::INFO::2014-01-27 10:31:20,793::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2468::INFO::2014-01-27 10:31:20,805::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2469::DEBUG::2014-01-27 10:31:20,810::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2469::INFO::2014-01-27 10:31:20,810::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2469::INFO::2014-01-27 10:31:20,815::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615981568', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2471::DEBUG::2014-01-27 10:31:22,706::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2471::DEBUG::2014-01-27 10:31:22,729::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2472::DEBUG::2014-01-27 10:31:23,577::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2472::ERROR::2014-01-27 10:31:23,592::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-39::DEBUG::2014-01-27 10:31:24,065::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:31:24,072::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000768366 s, 457 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:31:24,507::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:31:24,514::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00102538 s, 346 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:31:26,227::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:31:26,235::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000766129 s, 719 kB/s\n'; <rc> = 0
>Thread-2474::INFO::2014-01-27 10:31:27,168::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2474::INFO::2014-01-27 10:31:27,168::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000766129', 'lastCheck': '0.9', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00102538', 'lastCheck': '2.7', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000768366', 'lastCheck': '3.1', 'valid': True}}
>Thread-2476::DEBUG::2014-01-27 10:31:27,766::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2476::DEBUG::2014-01-27 10:31:27,789::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2478::DEBUG::2014-01-27 10:31:30,835::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2478::INFO::2014-01-27 10:31:30,836::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2478::INFO::2014-01-27 10:31:30,843::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2479::DEBUG::2014-01-27 10:31:30,847::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2479::INFO::2014-01-27 10:31:30,848::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2479::INFO::2014-01-27 10:31:30,853::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615981568', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2480::DEBUG::2014-01-27 10:31:32,834::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2480::DEBUG::2014-01-27 10:31:32,857::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:31:34,086::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:31:34,093::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000620649 s, 566 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:31:34,530::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:31:34,536::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000770868 s, 461 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:31:36,254::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:31:36,260::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00074993 s, 735 kB/s\n'; <rc> = 0
>Thread-2483::DEBUG::2014-01-27 10:31:37,895::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2483::DEBUG::2014-01-27 10:31:37,917::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2485::DEBUG::2014-01-27 10:31:40,875::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2485::INFO::2014-01-27 10:31:40,875::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2485::INFO::2014-01-27 10:31:40,887::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2486::DEBUG::2014-01-27 10:31:40,891::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2486::INFO::2014-01-27 10:31:40,891::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2486::INFO::2014-01-27 10:31:40,896::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615981568', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2487::INFO::2014-01-27 10:31:42,238::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2487::INFO::2014-01-27 10:31:42,238::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00074993', 'lastCheck': '6.0', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000770868', 'lastCheck': '7.7', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000620649', 'lastCheck': '8.1', 'valid': True}}
>Thread-2489::DEBUG::2014-01-27 10:31:42,970::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2489::DEBUG::2014-01-27 10:31:42,993::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:31:44,105::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:31:44,112::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000745762 s, 471 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:31:44,552::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:31:44,559::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00095909 s, 370 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:31:46,280::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:31:46,287::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000855613 s, 644 kB/s\n'; <rc> = 0
>Thread-2491::DEBUG::2014-01-27 10:31:48,043::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2491::DEBUG::2014-01-27 10:31:48,066::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2493::DEBUG::2014-01-27 10:31:50,918::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2493::INFO::2014-01-27 10:31:50,919::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2493::INFO::2014-01-27 10:31:50,931::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2494::DEBUG::2014-01-27 10:31:50,936::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2494::INFO::2014-01-27 10:31:50,936::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2494::INFO::2014-01-27 10:31:50,941::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615981568', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2496::DEBUG::2014-01-27 10:31:53,104::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2496::DEBUG::2014-01-27 10:31:53,126::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:31:54,124::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:31:54,131::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000643305 s, 546 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:31:54,575::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:31:54,582::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000759722 s, 467 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:31:56,306::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:31:56,313::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000919008 s, 600 kB/s\n'; <rc> = 0
>Thread-2498::INFO::2014-01-27 10:31:57,310::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2498::INFO::2014-01-27 10:31:57,310::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000919008', 'lastCheck': '1.0', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000759722', 'lastCheck': '2.7', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000643305', 'lastCheck': '3.2', 'valid': True}}
>Thread-2500::DEBUG::2014-01-27 10:31:58,171::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2500::DEBUG::2014-01-27 10:31:58,194::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2502::DEBUG::2014-01-27 10:32:00,962::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2502::INFO::2014-01-27 10:32:00,963::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2502::INFO::2014-01-27 10:32:00,970::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2503::DEBUG::2014-01-27 10:32:00,975::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2503::INFO::2014-01-27 10:32:00,975::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2503::INFO::2014-01-27 10:32:00,980::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615981568', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2504::DEBUG::2014-01-27 10:32:03,244::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2504::DEBUG::2014-01-27 10:32:03,267::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:32:04,145::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:32:04,152::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.00067158 s, 523 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:32:04,597::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:32:04,603::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.00076634 s, 463 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:32:06,332::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:32:06,339::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000860232 s, 641 kB/s\n'; <rc> = 0
>Thread-2507::DEBUG::2014-01-27 10:32:08,315::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2507::DEBUG::2014-01-27 10:32:08,338::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2509::DEBUG::2014-01-27 10:32:11,002::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2509::INFO::2014-01-27 10:32:11,003::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2509::INFO::2014-01-27 10:32:11,014::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2510::DEBUG::2014-01-27 10:32:11,019::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2510::INFO::2014-01-27 10:32:11,019::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2510::INFO::2014-01-27 10:32:11,024::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615850496', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2511::INFO::2014-01-27 10:32:12,381::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2511::INFO::2014-01-27 10:32:12,382::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000860232', 'lastCheck': '6.0', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00076634', 'lastCheck': '7.8', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.00067158', 'lastCheck': '8.2', 'valid': True}}
>Thread-2513::DEBUG::2014-01-27 10:32:13,384::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2513::DEBUG::2014-01-27 10:32:13,406::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:32:14,164::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:32:14,171::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000669059 s, 525 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:32:14,619::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:32:14,626::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000965918 s, 368 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:32:16,358::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:32:16,366::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000882978 s, 624 kB/s\n'; <rc> = 0
>Thread-2516::DEBUG::2014-01-27 10:32:18,444::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2516::DEBUG::2014-01-27 10:32:18,472::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2517::DEBUG::2014-01-27 10:32:21,044::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2517::INFO::2014-01-27 10:32:21,045::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2517::INFO::2014-01-27 10:32:21,057::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2518::DEBUG::2014-01-27 10:32:21,061::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2518::INFO::2014-01-27 10:32:21,062::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2518::INFO::2014-01-27 10:32:21,066::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615850496', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2520::DEBUG::2014-01-27 10:32:23,189::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('data', '', '') {}
>Thread-2520::DEBUG::2014-01-27 10:32:23,206::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data', 'pid': '1524', 'port': '49159'}], 'nfs': [{'status': 'ONLINE', 'hostname': '192.168.1.107', 'pid': '1560', 'port': '2049'}], 'shd': [], 'name': 'data'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2521::DEBUG::2014-01-27 10:32:23,215::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('fast_data', '', '') {}
>Thread-2521::DEBUG::2014-01-27 10:32:23,231::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data2', 'pid': '1516', 'port': '49160'}], 'nfs': [{'status': 'ONLINE', 'hostname': '192.168.1.107', 'pid': '1560', 'port': '2049'}], 'shd': [], 'name': 'fast_data'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2522::DEBUG::2014-01-27 10:32:23,241::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('ISO_DOMAIN', '', '') {}
>Thread-2522::DEBUG::2014-01-27 10:32:23,257::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data1/iso', 'pid': '1540', 'port': '49158'}], 'nfs': [{'status': 'ONLINE', 'hostname': '192.168.1.107', 'pid': '1560', 'port': '2049'}], 'shd': [], 'name': 'ISO_DOMAIN'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2523::DEBUG::2014-01-27 10:32:23,266::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumeStatus with ('test', '', '') {}
>Thread-2523::DEBUG::2014-01-27 10:32:23,282::BindingXMLRPC::977::vds::(wrapper) return volumeStatus with {'volumeStatus': {'bricks': [{'status': 'ONLINE', 'brick': 'cauldron.arclab:/data1/test', 'pid': '1530', 'port': '49161'}], 'nfs': [], 'shd': [], 'name': 'test'}, 'status': {'message': 'Done', 'code': 0}}
>Thread-2524::DEBUG::2014-01-27 10:32:23,515::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {} flowID [3833025d]
>Thread-2524::DEBUG::2014-01-27 10:32:23,536::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2525::DEBUG::2014-01-27 10:32:23,607::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2525::ERROR::2014-01-27 10:32:23,621::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-39::DEBUG::2014-01-27 10:32:24,187::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:32:24,195::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000600214 s, 585 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:32:24,643::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:32:24,650::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000999761 s, 355 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:32:26,384::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:32:26,391::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00096969 s, 568 kB/s\n'; <rc> = 0
>Thread-2527::INFO::2014-01-27 10:32:27,453::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2527::INFO::2014-01-27 10:32:27,455::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00096969', 'lastCheck': '1.1', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000999761', 'lastCheck': '2.8', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000600214', 'lastCheck': '3.3', 'valid': True}}
>Thread-2529::DEBUG::2014-01-27 10:32:28,570::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2529::DEBUG::2014-01-27 10:32:28,593::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2531::DEBUG::2014-01-27 10:32:31,089::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2531::INFO::2014-01-27 10:32:31,089::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2531::INFO::2014-01-27 10:32:31,102::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2532::DEBUG::2014-01-27 10:32:31,106::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2532::INFO::2014-01-27 10:32:31,107::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2532::INFO::2014-01-27 10:32:31,111::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615850496', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2534::DEBUG::2014-01-27 10:32:33,636::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2534::DEBUG::2014-01-27 10:32:33,659::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:32:34,198::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain abf4ff41-eb0a-4580-9ad0-bae53d56c6b0
>Thread-39::DEBUG::2014-01-27 10:32:34,204::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0
>Thread-39::DEBUG::2014-01-27 10:32:34,206::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-39::DEBUG::2014-01-27 10:32:34,214::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Iso', 'DESCRIPTION=ISO_DOMAIN', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=a13a367c-accd-451a-a2f3-4c83660f1719', 'REMOTE_PATH=cauldron.arclab:/ISO_DOMAIN', 'ROLE=Regular', 'SDUUID=abf4ff41-eb0a-4580-9ad0-bae53d56c6b0', 'TYPE=NFS', 'VERSION=0', '_SHA_CKSUM=f9e9b2005f92982aa61eb5738ed4051f4688ea08']
>Thread-39::DEBUG::2014-01-27 10:32:34,217::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-39::INFO::2014-01-27 10:32:34,217::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace abf4ff41-eb0a-4580-9ad0-bae53d56c6b0_imageNS already registered
>Thread-39::INFO::2014-01-27 10:32:34,217::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace abf4ff41-eb0a-4580-9ad0-bae53d56c6b0_volumeNS already registered
>Thread-39::DEBUG::2014-01-27 10:32:34,226::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:32:34,233::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000652681 s, 538 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:32:34,653::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d5a8d1ae-cfd4-483e-bdeb-9649a1134796
>Thread-36::DEBUG::2014-01-27 10:32:34,662::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796
>Thread-36::DEBUG::2014-01-27 10:32:34,663::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-36::DEBUG::2014-01-27 10:32:34,672::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=fast_data', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=0', 'POOL_UUID=a13a367c-accd-451a-a2f3-4c83660f1719', 'REMOTE_PATH=cauldron.arclab:fast_data', 'ROLE=Regular', 'SDUUID=d5a8d1ae-cfd4-483e-bdeb-9649a1134796', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=f86e05b564f9c2cb497ca896ffd782faa386be82']
>Thread-36::DEBUG::2014-01-27 10:32:34,676::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-36::INFO::2014-01-27 10:32:34,676::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace d5a8d1ae-cfd4-483e-bdeb-9649a1134796_imageNS already registered
>Thread-36::INFO::2014-01-27 10:32:34,676::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace d5a8d1ae-cfd4-483e-bdeb-9649a1134796_volumeNS already registered
>Thread-36::DEBUG::2014-01-27 10:32:34,686::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:32:34,693::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000868456 s, 409 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:32:36,398::domainMonitor::192::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain c074a6e3-87be-445e-890f-601c215ba320
>Thread-35::DEBUG::2014-01-27 10:32:36,407::fileSD::140::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320
>Thread-35::DEBUG::2014-01-27 10:32:36,408::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend
>Thread-35::DEBUG::2014-01-27 10:32:36,417::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=data_domain', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=arclab', 'POOL_DOMAINS=c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'POOL_SPM_ID=1', 'POOL_SPM_LVER=0', 'POOL_UUID=a13a367c-accd-451a-a2f3-4c83660f1719', 'REMOTE_PATH=cauldron.arclab:/data', 'ROLE=Master', 'SDUUID=c074a6e3-87be-445e-890f-601c215ba320', 'TYPE=GLUSTERFS', 'VERSION=3', '_SHA_CKSUM=0c3248047b8256dbf9bb7c8b0be9bb835eb1c779']
>Thread-35::DEBUG::2014-01-27 10:32:36,422::fileSD::575::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images []
>Thread-35::INFO::2014-01-27 10:32:36,422::sd::374::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c074a6e3-87be-445e-890f-601c215ba320_imageNS already registered
>Thread-35::INFO::2014-01-27 10:32:36,422::sd::382::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace c074a6e3-87be-445e-890f-601c215ba320_volumeNS already registered
>Thread-35::DEBUG::2014-01-27 10:32:36,432::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:32:36,439::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.00085222 s, 647 kB/s\n'; <rc> = 0
>Thread-2547::DEBUG::2014-01-27 10:32:38,701::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2547::DEBUG::2014-01-27 10:32:38,725::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2549::DEBUG::2014-01-27 10:32:41,133::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2549::INFO::2014-01-27 10:32:41,133::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2549::INFO::2014-01-27 10:32:41,144::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2550::DEBUG::2014-01-27 10:32:41,148::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2550::INFO::2014-01-27 10:32:41,149::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2550::INFO::2014-01-27 10:32:41,153::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615719424', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2551::INFO::2014-01-27 10:32:42,539::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2551::INFO::2014-01-27 10:32:42,539::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00085222', 'lastCheck': '6.1', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000868456', 'lastCheck': '7.8', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000652681', 'lastCheck': '8.3', 'valid': True}}
>Thread-2553::DEBUG::2014-01-27 10:32:43,774::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2553::DEBUG::2014-01-27 10:32:43,797::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:32:44,245::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:32:44,253::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000465571 s, 754 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:32:44,707::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:32:44,714::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000828744 s, 428 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:32:46,458::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:32:46,465::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000948822 s, 581 kB/s\n'; <rc> = 0
>Thread-2556::DEBUG::2014-01-27 10:32:48,845::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2556::DEBUG::2014-01-27 10:32:48,868::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2557::DEBUG::2014-01-27 10:32:51,173::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2557::INFO::2014-01-27 10:32:51,174::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2557::INFO::2014-01-27 10:32:51,185::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2558::DEBUG::2014-01-27 10:32:51,189::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2558::INFO::2014-01-27 10:32:51,190::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2558::INFO::2014-01-27 10:32:51,194::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615719424', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2560::DEBUG::2014-01-27 10:32:53,915::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2560::DEBUG::2014-01-27 10:32:53,938::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:32:54,265::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:32:54,272::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000503712 s, 697 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:32:54,730::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:32:54,737::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000894965 s, 397 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:32:56,485::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:32:56,492::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000944213 s, 584 kB/s\n'; <rc> = 0
>Thread-2562::INFO::2014-01-27 10:32:57,613::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2562::INFO::2014-01-27 10:32:57,613::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000944213', 'lastCheck': '1.1', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000894965', 'lastCheck': '2.9', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000503712', 'lastCheck': '3.3', 'valid': True}}
>Thread-2564::DEBUG::2014-01-27 10:32:58,981::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2564::DEBUG::2014-01-27 10:32:59,005::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2566::DEBUG::2014-01-27 10:33:01,215::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2566::INFO::2014-01-27 10:33:01,216::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2566::INFO::2014-01-27 10:33:01,228::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2567::DEBUG::2014-01-27 10:33:01,233::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2567::INFO::2014-01-27 10:33:01,233::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2567::INFO::2014-01-27 10:33:01,238::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615719424', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2569::DEBUG::2014-01-27 10:33:04,052::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2569::DEBUG::2014-01-27 10:33:04,075::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:33:04,285::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:33:04,292::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000664782 s, 528 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:33:04,753::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:33:04,760::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000886458 s, 400 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:33:06,511::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:33:06,518::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000773864 s, 712 kB/s\n'; <rc> = 0
>Thread-2571::DEBUG::2014-01-27 10:33:09,119::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2571::DEBUG::2014-01-27 10:33:09,142::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2573::DEBUG::2014-01-27 10:33:11,259::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2573::INFO::2014-01-27 10:33:11,260::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2573::INFO::2014-01-27 10:33:11,272::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2574::DEBUG::2014-01-27 10:33:11,277::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2574::INFO::2014-01-27 10:33:11,277::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2574::INFO::2014-01-27 10:33:11,281::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615719424', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2575::INFO::2014-01-27 10:33:12,684::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2575::INFO::2014-01-27 10:33:12,684::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000773864', 'lastCheck': '6.2', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000886458', 'lastCheck': '7.9', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000664782', 'lastCheck': '8.4', 'valid': True}}
>Thread-2577::DEBUG::2014-01-27 10:33:14,182::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2577::DEBUG::2014-01-27 10:33:14,205::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-39::DEBUG::2014-01-27 10:33:14,304::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:33:14,311::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000610077 s, 575 kB/s\n'; <rc> = 0
>Thread-36::DEBUG::2014-01-27 10:33:14,774::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:33:14,780::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000876415 s, 405 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:33:16,537::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:33:16,544::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000860484 s, 640 kB/s\n'; <rc> = 0
>Thread-2580::DEBUG::2014-01-27 10:33:19,255::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2580::DEBUG::2014-01-27 10:33:19,279::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2581::DEBUG::2014-01-27 10:33:21,304::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2581::INFO::2014-01-27 10:33:21,305::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2581::INFO::2014-01-27 10:33:21,315::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2582::DEBUG::2014-01-27 10:33:21,319::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2582::INFO::2014-01-27 10:33:21,319::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2582::INFO::2014-01-27 10:33:21,324::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615719424', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2584::DEBUG::2014-01-27 10:33:23,639::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call tasksList with () {}
>Thread-2584::ERROR::2014-01-27 10:33:23,653::BindingXMLRPC::986::vds::(wrapper) vdsm exception occured
>Traceback (most recent call last):
>  File "/usr/share/vdsm/BindingXMLRPC.py", line 973, in wrapper
>    res = f(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 54, in wrapper
>    rv = func(*args, **kwargs)
>  File "/usr/share/vdsm/gluster/api.py", line 306, in tasksList
>    status = self.svdsmProxy.glusterTasksList(taskIds)
>  File "/usr/share/vdsm/supervdsm.py", line 50, in __call__
>    return callMethod()
>  File "/usr/share/vdsm/supervdsm.py", line 48, in <lambda>
>    **kwargs)
>  File "<string>", line 2, in glusterTasksList
>  File "/usr/lib64/python2.6/multiprocessing/managers.py", line 740, in _callmethod
>    raise convert_to_error(kind, result)
>GlusterCmdExecFailedException: Command execution failed
>error: tasks is not a valid status option
>Usage: volume status [all | <VOLNAME> [nfs|shd|<BRICK>]] [detail|clients|mem|inode|fd|callpool]
>return code: 1
>Thread-2585::DEBUG::2014-01-27 10:33:24,312::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-39::DEBUG::2014-01-27 10:33:24,323::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:33:24,331::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000446422 s, 786 kB/s\n'; <rc> = 0
>Thread-2585::DEBUG::2014-01-27 10:33:24,334::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-36::DEBUG::2014-01-27 10:33:24,794::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:33:24,801::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000522516 s, 679 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:33:26,563::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:33:26,569::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000895079 s, 616 kB/s\n'; <rc> = 0
>Thread-2587::INFO::2014-01-27 10:33:27,754::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2587::INFO::2014-01-27 10:33:27,754::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000895079', 'lastCheck': '1.2', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000522516', 'lastCheck': '3.0', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000446422', 'lastCheck': '3.4', 'valid': True}}
>Thread-2589::DEBUG::2014-01-27 10:33:29,368::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2589::DEBUG::2014-01-27 10:33:29,391::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2591::DEBUG::2014-01-27 10:33:31,346::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2591::INFO::2014-01-27 10:33:31,347::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2591::INFO::2014-01-27 10:33:31,358::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2592::DEBUG::2014-01-27 10:33:31,362::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2592::INFO::2014-01-27 10:33:31,363::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2592::INFO::2014-01-27 10:33:31,368::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615719424', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:33:34,344::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:33:34,351::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000676579 s, 519 kB/s\n'; <rc> = 0
>Thread-2594::DEBUG::2014-01-27 10:33:34,442::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2594::DEBUG::2014-01-27 10:33:34,465::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-36::DEBUG::2014-01-27 10:33:34,816::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:33:34,823::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000928385 s, 382 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:33:36,589::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:33:36,596::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000926063 s, 595 kB/s\n'; <rc> = 0
>Thread-2596::DEBUG::2014-01-27 10:33:39,502::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2596::DEBUG::2014-01-27 10:33:39,525::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2598::DEBUG::2014-01-27 10:33:41,389::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2598::INFO::2014-01-27 10:33:41,390::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2598::INFO::2014-01-27 10:33:41,402::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2599::DEBUG::2014-01-27 10:33:41,406::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2599::INFO::2014-01-27 10:33:41,406::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2599::INFO::2014-01-27 10:33:41,411::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615588352', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-2600::INFO::2014-01-27 10:33:42,825::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2600::INFO::2014-01-27 10:33:42,826::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000926063', 'lastCheck': '6.2', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000928385', 'lastCheck': '8.0', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000676579', 'lastCheck': '8.5', 'valid': True}}
>Thread-39::DEBUG::2014-01-27 10:33:44,362::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:33:44,370::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000693577 s, 506 kB/s\n'; <rc> = 0
>Thread-2602::DEBUG::2014-01-27 10:33:44,561::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2602::DEBUG::2014-01-27 10:33:44,584::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-36::DEBUG::2014-01-27 10:33:44,836::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:33:44,844::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000968048 s, 367 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:33:46,616::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:33:46,623::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000894762 s, 616 kB/s\n'; <rc> = 0
>Thread-2605::DEBUG::2014-01-27 10:33:49,633::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2605::DEBUG::2014-01-27 10:33:49,656::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2606::DEBUG::2014-01-27 10:33:51,432::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2606::INFO::2014-01-27 10:33:51,433::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2606::INFO::2014-01-27 10:33:51,444::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2607::DEBUG::2014-01-27 10:33:51,449::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2607::INFO::2014-01-27 10:33:51,449::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2607::INFO::2014-01-27 10:33:51,454::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615588352', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:33:54,381::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:33:54,389::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.000739191 s, 475 kB/s\n'; <rc> = 0
>Thread-2609::DEBUG::2014-01-27 10:33:54,703::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2609::DEBUG::2014-01-27 10:33:54,726::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-36::DEBUG::2014-01-27 10:33:54,859::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:33:54,866::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000944666 s, 376 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:33:56,642::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:33:56,649::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000895677 s, 615 kB/s\n'; <rc> = 0
>Thread-2611::INFO::2014-01-27 10:33:57,897::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None)
>Thread-2611::INFO::2014-01-27 10:33:57,898::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'c074a6e3-87be-445e-890f-601c215ba320': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000895677', 'lastCheck': '1.2', 'valid': True}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000944666', 'lastCheck': '3.0', 'valid': True}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'code': 0, 'version': 0, 'acquired': True, 'delay': '0.000739191', 'lastCheck': '3.5', 'valid': True}}
>Thread-2613::DEBUG::2014-01-27 10:33:59,771::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2613::DEBUG::2014-01-27 10:33:59,794::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2615::DEBUG::2014-01-27 10:34:01,475::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2615::INFO::2014-01-27 10:34:01,476::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2615::INFO::2014-01-27 10:34:01,486::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}
>Thread-2616::DEBUG::2014-01-27 10:34:01,491::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2616::INFO::2014-01-27 10:34:01,491::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2616::INFO::2014-01-27 10:34:01,496::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'arclab', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'pool_status': 'connected', 'lver': 0, 'domains': 'c074a6e3-87be-445e-890f-601c215ba320:Active,d5a8d1ae-cfd4-483e-bdeb-9649a1134796:Active,abf4ff41-eb0a-4580-9ad0-bae53d56c6b0:Active', 'master_uuid': 'c074a6e3-87be-445e-890f-601c215ba320', 'version': '3', 'spm_id': 1, 'type': 'GLUSTERFS', 'master_ver': 1}, 'dominfo': {'c074a6e3-87be-445e-890f-601c215ba320': {'status': 'Active', 'diskfree': '425801940992', 'isoprefix': '', 'alerts': [], 'disktotal': '429152665600', 'version': 3}, 'd5a8d1ae-cfd4-483e-bdeb-9649a1134796': {'status': 'Active', 'diskfree': '67615588352', 'isoprefix': '', 'alerts': [], 'disktotal': '111443574784', 'version': 3}, 'abf4ff41-eb0a-4580-9ad0-bae53d56c6b0': {'status': 'Active', 'diskfree': '559528476672', 'isoprefix': '/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '570291519488', 'version': 0}}}
>Thread-39::DEBUG::2014-01-27 10:34:04,402::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/cauldron.arclab:_ISO__DOMAIN/abf4ff41-eb0a-4580-9ad0-bae53d56c6b0/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-39::DEBUG::2014-01-27 10:34:04,409::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n351 bytes (351 B) copied, 0.00063476 s, 553 kB/s\n'; <rc> = 0
>Thread-2618::DEBUG::2014-01-27 10:34:04,832::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2618::DEBUG::2014-01-27 10:34:04,855::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-36::DEBUG::2014-01-27 10:34:04,884::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:fast__data/d5a8d1ae-cfd4-483e-bdeb-9649a1134796/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-36::DEBUG::2014-01-27 10:34:04,891::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n355 bytes (355 B) copied, 0.000928498 s, 382 kB/s\n'; <rc> = 0
>Thread-35::DEBUG::2014-01-27 10:34:06,668::fileSD::225::Storage.Misc.excCmd::(getReadDelay) '/bin/dd iflag=direct if=/rhev/data-center/mnt/glusterSD/cauldron.arclab:_data/c074a6e3-87be-445e-890f-601c215ba320/dom_md/metadata bs=4096 count=1' (cwd None)
>Thread-35::DEBUG::2014-01-27 10:34:06,675::fileSD::225::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n551 bytes (551 B) copied, 0.000916182 s, 601 kB/s\n'; <rc> = 0
>Thread-2620::DEBUG::2014-01-27 10:34:09,892::BindingXMLRPC::970::vds::(wrapper) client [192.168.1.107]::call volumesList with () {}
>Thread-2620::DEBUG::2014-01-27 10:34:09,915::BindingXMLRPC::977::vds::(wrapper) return volumesList with {'status': {'message': 'Done', 'code': 0}, 'volumes': {'fast_data': {'transportType': ['TCP'], 'uuid': '09ae99f1-01f9-47ff-baf1-4a4999c6aac0', 'bricks': ['cauldron.arclab:/data2'], 'volumeName': 'fast_data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'test': {'transportType': ['TCP'], 'uuid': 'e2821e53-f29a-42b5-ad31-50d8066b2318', 'bricks': ['cauldron.arclab:/data1/test'], 'volumeName': 'test', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'user.cifs': 'enable', 'server.allow-insecure': 'on', 'nfs.disable': 'on', 'auth.allow': '*'}}, 'ISO_DOMAIN': {'transportType': ['TCP'], 'uuid': '3e05e788-4fbc-438d-98c0-fddfe5ab9551', 'bricks': ['cauldron.arclab:/data1/iso'], 'volumeName': 'ISO_DOMAIN', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}, 'data': {'transportType': ['TCP'], 'uuid': '26a05930-9207-4e92-9cf9-bb292eff940b', 'bricks': ['cauldron.arclab:/data'], 'volumeName': 'data', 'volumeType': 'DISTRIBUTE', 'replicaCount': '1', 'brickCount': '1', 'distCount': '1', 'volumeStatus': 'ONLINE', 'stripeCount': '1', 'bricksInfo': [], 'options': {'cluster.server-quorum-type': 'server', 'cluster.eager-lock': 'enable', 'performance.stat-prefetch': 'off', 'auth.allow': '*', 'cluster.quorum-type': 'auto', 'performance.quick-read': 'off', 'network.remote-dio': 'enable', 'nfs.disable': 'off', 'performance.io-cache': 'off', 'server.allow-insecure': 'on', 'storage.owner-uid': '36', 'user.cifs': 'enable', 'performance.read-ahead': 'off', 'storage.owner-gid': '36'}}}}
>Thread-2622::DEBUG::2014-01-27 10:34:11,518::BindingXMLRPC::159::vds::(wrapper) client [192.168.1.107]
>Thread-2622::INFO::2014-01-27 10:34:11,518::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='a13a367c-accd-451a-a2f3-4c83660f1719', options=None)
>Thread-2622::INFO::2014-01-27 10:34:11,530::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}}

Actions: View

Attachments on bug 1058253: 856008 | 856012 | 856014