Bug 1574990

Summary:

Unable to delete heketi volumes

Product:

[Red Hat Storage] Red Hat Gluster Storage

Reporter:

vinutha <vinug>

Component:

heketi

Assignee:

John Mulligan <jmulligan>

Status:

CLOSED WORKSFORME

QA Contact:

vinutha <vinug>

Severity:

medium

Docs Contact:

Priority:

unspecified

Version:

rhgs-3.3

CC:

hchiramm, jmulligan, kramdoss, madam, pprakash, rhs-bugs, rtalur, sankarshan, storage-qa-internal, vinug

Target Milestone:

---

Keywords:

Triaged

Target Release:

---

Hardware:

Unspecified

OS:

Unspecified

Whiteboard:

Fixed In Version:

Doc Type:

If docs needed, set a value

Doc Text:

Story Points:

---

Clone Of:

Environment:

Last Closed:

2018-07-02 22:31:26 UTC

Type:

Bug

Regression:

---

Mount Type:

---

Documentation:

---

CRM:

Verified Versions:

Category:

---

oVirt Team:

---

RHEL 7.3 requirements from Atomic Host:

Cloudforms Team:

---

Target Upstream Version:

Embargoed:

Attachments:

Description	Flags
Heketi logs	none
sosreports	none

Description vinutha 2018-05-04 13:19:26 UTC

Description of problem:

Unable to delete heketi volumes. 

Error message : Error: Unable to determine number of logical volumes in thin pool tp_4ebe9e2154f457321b9309e088bda1f7 on host dhcp47-164.lab.eng.blr.redhat.com


Version-Release number of selected component (if applicable):
heketi-client-6.0

How reproducible:
Always reproducible on my setup

Steps to Reproduce:
1.Configure OCP 3.9 and CNS on a 3 node cluster using ansible.
2.Created multiple mongodb pods 
3.Trying to delete any of the heketi volume fails  

Actual results:

heketi-cli volume delete 901d43d11f19526521415f6d44fa29b7

Error: Unable to determine number of logical volumes in thin pool tp_4ebe9e2154f457321b9309e088bda1f7 on host dhcp47-164.lab.eng.blr.redhat.com


Expected results:

Volume deletion should be successful


Additional info:

# heketi-cli topology info

Cluster Id: 90c2d17d3c38caf7babe55a21811b618

    File:  true
    Block: true

    Volumes:

	Name: vol_0c13a6566ba35bee8ff2746ef11f2906
	Size: 2
	Id: 0c13a6566ba35bee8ff2746ef11f2906
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_0c13a6566ba35bee8ff2746ef11f2906
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 0a612ebcba5958e816bd5acaeafdb083
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_0a612ebcba5958e816bd5acaeafdb083/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: 4c26c5cf7fb549db41921fd632a585f6
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_4c26c5cf7fb549db41921fd632a585f6/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: bdaab9427f7db778268d272438f5e9bc
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_bdaab9427f7db778268d272438f5e9bc/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: vol_147607a76c6058ac92b8d4d43545755b
	Size: 2
	Id: 147607a76c6058ac92b8d4d43545755b
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_147607a76c6058ac92b8d4d43545755b
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 0b8cdf66a493c68b2b7a17889d71263b
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_0b8cdf66a493c68b2b7a17889d71263b/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: 7e63c67287bd76187c5b930c5331ec19
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_7e63c67287bd76187c5b930c5331ec19/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: c9c2a3f6e513d8e7d56a4af7c3eb18e0
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_c9c2a3f6e513d8e7d56a4af7c3eb18e0/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e


	Name: vol_1c511c9fd9391836c4d3ae6dae534dc6
	Size: 1
	Id: 1c511c9fd9391836c4d3ae6dae534dc6
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_1c511c9fd9391836c4d3ae6dae534dc6
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 12b5fe102300188a4b2046f05adfb3c1
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_12b5fe102300188a4b2046f05adfb3c1/brick
			Size (GiB): 1
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: 6629c827e8d0ddb10467af6ca9604f86
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_6629c827e8d0ddb10467af6ca9604f86/brick
			Size (GiB): 1
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: a883487f215f7a25dceb981eb613053c
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_a883487f215f7a25dceb981eb613053c/brick
			Size (GiB): 1
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: vol_21cdc9c08c9d6f210d8e7d198fe126b2
	Size: 4
	Id: 21cdc9c08c9d6f210d8e7d198fe126b2
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_21cdc9c08c9d6f210d8e7d198fe126b2
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 5e4497b0c830dbc42e2a8f4f98f50800
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_5e4497b0c830dbc42e2a8f4f98f50800/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: 72bdb3359d9fbf352150b6e4d3d8c054
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_72bdb3359d9fbf352150b6e4d3d8c054/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: fe42e060b0a1de8156345178fbfd12fa
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_fe42e060b0a1de8156345178fbfd12fa/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e


	Name: vol_2a88c90189e7ef87be9faf9234817195
	Size: 2
	Id: 2a88c90189e7ef87be9faf9234817195
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_2a88c90189e7ef87be9faf9234817195
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 7eca9c43426301cd7cd3132e48672b6b
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_7eca9c43426301cd7cd3132e48672b6b/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: d55b7d860e9bffe18107066fe7f92918
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_d55b7d860e9bffe18107066fe7f92918/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6

			Id: f6abb038dcf53634f3832bb7d74f7491
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_f6abb038dcf53634f3832bb7d74f7491/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e


	Name: vol_3fa8090f73571dff97d4b42159bc1405
	Size: 4
	Id: 3fa8090f73571dff97d4b42159bc1405
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_3fa8090f73571dff97d4b42159bc1405
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 35bae504176e60b3e6a99fb69f10dca9
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_35bae504176e60b3e6a99fb69f10dca9/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a

			Id: b5059c0b284ea5a32cc3bab81f79acfe
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_b5059c0b284ea5a32cc3bab81f79acfe/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: ed29fb678b8e4d6d466278848eb56606
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_ed29fb678b8e4d6d466278848eb56606/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: vol_41795374286c66504ef2f391831ea302
	Size: 4
	Id: 41795374286c66504ef2f391831ea302
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_41795374286c66504ef2f391831ea302
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: a0e76c408c11d4d2a2086337a1153490
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_a0e76c408c11d4d2a2086337a1153490/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6

			Id: c6e1b117f47ca8b8112a116d7d7e653d
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_c6e1b117f47ca8b8112a116d7d7e653d/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: e6bf9ea4478f3dfdabf34f85f422361e
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_e6bf9ea4478f3dfdabf34f85f422361e/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929


	Name: vol_43cbb007b10a2ee039611fc2a199bcc5
	Size: 4
	Id: 43cbb007b10a2ee039611fc2a199bcc5
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_43cbb007b10a2ee039611fc2a199bcc5
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 0d5af994b3292ae0c95342c99043a3d0
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_0d5af994b3292ae0c95342c99043a3d0/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a

			Id: 9e6922aed343c011e64f1b158c56ea7a
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_9e6922aed343c011e64f1b158c56ea7a/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: e0fa69cf282c63e626ec4dcd01111a0b
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_e0fa69cf282c63e626ec4dcd01111a0b/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e


	Name: vol_62f0877270b3ba5a6748c1048a2747af
	Size: 4
	Id: 62f0877270b3ba5a6748c1048a2747af
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_62f0877270b3ba5a6748c1048a2747af
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 2b92b650ff97f762372caa04fdeb6801
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_2b92b650ff97f762372caa04fdeb6801/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: 869d92d550eadd9f5b4fa2516684a159
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_869d92d550eadd9f5b4fa2516684a159/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: d69aa58a8a07dc46e9eee522b64238f3
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_d69aa58a8a07dc46e9eee522b64238f3/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: vol_732dd1bf04d1e8f0f02f70b95a6757cd
	Size: 2
	Id: 732dd1bf04d1e8f0f02f70b95a6757cd
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_732dd1bf04d1e8f0f02f70b95a6757cd
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 8fa522925433d4ce685ada1124c3ccbf
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_8fa522925433d4ce685ada1124c3ccbf/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: a6caedf056229c4e48c3be7065415ee0
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_a6caedf056229c4e48c3be7065415ee0/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: d86e85e97af107a09f4e4a0c67c61bbe
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_d86e85e97af107a09f4e4a0c67c61bbe/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: heketidbstorage
	Size: 2
	Id: 900675a4e7d6dde36db3b6c7956038df
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:heketidbstorage
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 6065a4923e537303424e997dd302a2de
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_6065a4923e537303424e997dd302a2de/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: ab8dfede46f90a97f7d21bdd83d86db8
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_ab8dfede46f90a97f7d21bdd83d86db8/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6

			Id: d690242bbaa0e0b1a1fc6b4139377823
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_d690242bbaa0e0b1a1fc6b4139377823/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a


	Name: vol_901d43d11f19526521415f6d44fa29b7
	Size: 2
	Id: 901d43d11f19526521415f6d44fa29b7
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_901d43d11f19526521415f6d44fa29b7
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 2e551b8d470fda3a439c1b312209a157
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_2e551b8d470fda3a439c1b312209a157/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: 390f8b20654ef60734dea56ef0e8122e
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_390f8b20654ef60734dea56ef0e8122e/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: 4ebe9e2154f457321b9309e088bda1f7
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_4ebe9e2154f457321b9309e088bda1f7/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: vol_94a03c98e16af034eae3289ad06cd1fd
	Size: 2
	Id: 94a03c98e16af034eae3289ad06cd1fd
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_94a03c98e16af034eae3289ad06cd1fd
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 326e947941610fe85c537cd3b244f787
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_326e947941610fe85c537cd3b244f787/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: 46f82c9a923d8812eb88f03802b54d6a
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_46f82c9a923d8812eb88f03802b54d6a/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e

			Id: 4f36fd69b612bd591546352dd238aa95
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_4f36fd69b612bd591546352dd238aa95/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929


	Name: vol_95a9aa52c05d64ccc72865d353c0736c
	Size: 4
	Id: 95a9aa52c05d64ccc72865d353c0736c
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_95a9aa52c05d64ccc72865d353c0736c
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 13aa805f1febbfe74e485c0501d2d481
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_13aa805f1febbfe74e485c0501d2d481/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: b93d132f8482dcdbc531d6705a824169
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_b93d132f8482dcdbc531d6705a824169/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6

			Id: e46ab30eb297a8cb1e9a48d55504fd4d
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_e46ab30eb297a8cb1e9a48d55504fd4d/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a


	Name: vol_9b1df38af74f6a6d38d880aecc23cd42
	Size: 4
	Id: 9b1df38af74f6a6d38d880aecc23cd42
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_9b1df38af74f6a6d38d880aecc23cd42
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 26cf87a9bbf348c599101b5df2f76fed
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_26cf87a9bbf348c599101b5df2f76fed/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a

			Id: 851d1a47c7c7da4cbb1597511649b216
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_851d1a47c7c7da4cbb1597511649b216/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6

			Id: f5f484be58329b901293e750e9a41613
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_f5f484be58329b901293e750e9a41613/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931


	Name: vol_a283b8cdd622139eafeaa4a794ef2d69
	Size: 2
	Id: a283b8cdd622139eafeaa4a794ef2d69
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_a283b8cdd622139eafeaa4a794ef2d69
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 582a4d6001441f002df75bbe5346cca3
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_582a4d6001441f002df75bbe5346cca3/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e

			Id: 77ea84c820c7a644ab064536fa8e2f41
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_77ea84c820c7a644ab064536fa8e2f41/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a

			Id: 90f4d0eed811840a122de2fb72225f26
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_90f4d0eed811840a122de2fb72225f26/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929


	Name: vol_acf580de36b3ea8f19e80f9b8f4593ea
	Size: 4
	Id: acf580de36b3ea8f19e80f9b8f4593ea
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_acf580de36b3ea8f19e80f9b8f4593ea
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 5774b634698fac743f6967a2c204cbd6
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_5774b634698fac743f6967a2c204cbd6/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a

			Id: 5af9dd87d54b89d321379e4c070493fd
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_5af9dd87d54b89d321379e4c070493fd/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: e73fd4e2ea30078fc3c0c8ff57f3a48d
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_e73fd4e2ea30078fc3c0c8ff57f3a48d/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e


	Name: vol_b7c408f50a709c8faab7d4a5af57422a
	Size: 2
	Id: b7c408f50a709c8faab7d4a5af57422a
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_b7c408f50a709c8faab7d4a5af57422a
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 2e32c01372864c3e428057f7d23c9442
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_2e32c01372864c3e428057f7d23c9442/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: 428a7ea23e1927a95f3969f537c344ea
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_428a7ea23e1927a95f3969f537c344ea/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e

			Id: 67d6256b51fc41647a6cf4abc026033e
			Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_67d6256b51fc41647a6cf4abc026033e/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: df5066ca750f1e87aca0dbe629a478c6


	Name: vol_bb72566f127d040c544dc999921e2511
	Size: 2
	Id: bb72566f127d040c544dc999921e2511
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_bb72566f127d040c544dc999921e2511
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 191536cc9132d24bb967a5003ba0ed03
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_191536cc9132d24bb967a5003ba0ed03/brick
			Size (GiB): 2
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e

			Id: 5b0d17c3783fa22f00bf82f7069e295d
			Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_5b0d17c3783fa22f00bf82f7069e295d/brick
			Size (GiB): 2
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 1834ba558b62b4576dc6782c7cc89929

			Id: ba1bb7f9eac31e866c2555c816b54775
			Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_ba1bb7f9eac31e866c2555c816b54775/brick
			Size (GiB): 2
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: 18313d235763e6ed73931af8bd7c324e


	Name: vol_d961c7c239fd1910599005832255a495
	Size: 4
	Id: d961c7c239fd1910599005832255a495
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_d961c7c239fd1910599005832255a495
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: d11574b7e27f8b186ad664b7b3042f53
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_d11574b7e27f8b186ad664b7b3042f53/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: dad8f8d4c4229b180fc30c9690084948
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_dad8f8d4c4229b180fc30c9690084948/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e

			Id: fa88ab0bfb470cb54a2d2af51c99bc99
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_fa88ab0bfb470cb54a2d2af51c99bc99/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a


	Name: vol_ef543c32051f13e5a650b0ec47fb3c8d
	Size: 4
	Id: ef543c32051f13e5a650b0ec47fb3c8d
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Mount: 10.70.46.126:vol_ef543c32051f13e5a650b0ec47fb3c8d
	Mount Options: backup-volfile-servers=10.70.46.34,10.70.47.164
	Durability Type: replicate
	Replica: 3
	Snapshot: Disabled

		Bricks:
			Id: 0920f4c77505071707008801112ef365
			Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_0920f4c77505071707008801112ef365/brick
			Size (GiB): 4
			Node: 98dcfe81b6899e6b159a617e5c31353f
			Device: 54c725f09071d8ef33e17c1279b24931

			Id: 99bca3bd2c3bc84d921bf846c838df04
			Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_99bca3bd2c3bc84d921bf846c838df04/brick
			Size (GiB): 4
			Node: c7849230793eb0ae5bec0e0826e92347
			Device: aefc534d597a52cc661048b7b8bc300e

			Id: b3e39077fa31f34f6a62445f9ef5d381
			Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_b3e39077fa31f34f6a62445f9ef5d381/brick
			Size (GiB): 4
			Node: 2c10a8def2fe66f9817cdb44769bd17a
			Device: ee4815417b7d559207c5ebffddb3fe5a


    Nodes:

	Node Id: 2c10a8def2fe66f9817cdb44769bd17a
	State: online
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Zone: 1
	Management Hostnames: dhcp46-126.lab.eng.blr.redhat.com
	Storage Hostnames: 10.70.46.126
	Devices:
		Id:18313d235763e6ed73931af8bd7c324e   Name:/dev/sdd            State:online    Size (GiB):199     Used (GiB):29      Free (GiB):170     
			Bricks:
				Id:0b8cdf66a493c68b2b7a17889d71263b   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_0b8cdf66a493c68b2b7a17889d71263b/brick
				Id:326e947941610fe85c537cd3b244f787   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_326e947941610fe85c537cd3b244f787/brick
				Id:390f8b20654ef60734dea56ef0e8122e   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_390f8b20654ef60734dea56ef0e8122e/brick
				Id:428a7ea23e1927a95f3969f537c344ea   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_428a7ea23e1927a95f3969f537c344ea/brick
				Id:4c26c5cf7fb549db41921fd632a585f6   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_4c26c5cf7fb549db41921fd632a585f6/brick
				Id:5e4497b0c830dbc42e2a8f4f98f50800   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_5e4497b0c830dbc42e2a8f4f98f50800/brick
				Id:6629c827e8d0ddb10467af6ca9604f86   Size (GiB):1       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_6629c827e8d0ddb10467af6ca9604f86/brick
				Id:869d92d550eadd9f5b4fa2516684a159   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_869d92d550eadd9f5b4fa2516684a159/brick
				Id:a6caedf056229c4e48c3be7065415ee0   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_a6caedf056229c4e48c3be7065415ee0/brick
				Id:ba1bb7f9eac31e866c2555c816b54775   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_ba1bb7f9eac31e866c2555c816b54775/brick
				Id:c6e1b117f47ca8b8112a116d7d7e653d   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_c6e1b117f47ca8b8112a116d7d7e653d/brick
				Id:f6abb038dcf53634f3832bb7d74f7491   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_18313d235763e6ed73931af8bd7c324e/brick_f6abb038dcf53634f3832bb7d74f7491/brick
		Id:ee4815417b7d559207c5ebffddb3fe5a   Name:/dev/sde            State:online    Size (GiB):199     Used (GiB):34      Free (GiB):165     
			Bricks:
				Id:0d5af994b3292ae0c95342c99043a3d0   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_0d5af994b3292ae0c95342c99043a3d0/brick
				Id:26cf87a9bbf348c599101b5df2f76fed   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_26cf87a9bbf348c599101b5df2f76fed/brick
				Id:35bae504176e60b3e6a99fb69f10dca9   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_35bae504176e60b3e6a99fb69f10dca9/brick
				Id:5774b634698fac743f6967a2c204cbd6   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_5774b634698fac743f6967a2c204cbd6/brick
				Id:77ea84c820c7a644ab064536fa8e2f41   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_77ea84c820c7a644ab064536fa8e2f41/brick
				Id:9619140699ad829ec22cfe752e6a6e23   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_9619140699ad829ec22cfe752e6a6e23/brick
				Id:b3e39077fa31f34f6a62445f9ef5d381   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_b3e39077fa31f34f6a62445f9ef5d381/brick
				Id:d690242bbaa0e0b1a1fc6b4139377823   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_d690242bbaa0e0b1a1fc6b4139377823/brick
				Id:e46ab30eb297a8cb1e9a48d55504fd4d   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_e46ab30eb297a8cb1e9a48d55504fd4d/brick
				Id:fa88ab0bfb470cb54a2d2af51c99bc99   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_ee4815417b7d559207c5ebffddb3fe5a/brick_fa88ab0bfb470cb54a2d2af51c99bc99/brick

	Node Id: 98dcfe81b6899e6b159a617e5c31353f
	State: online
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Zone: 1
	Management Hostnames: dhcp46-34.lab.eng.blr.redhat.com
	Storage Hostnames: 10.70.46.34
	Devices:
		Id:1834ba558b62b4576dc6782c7cc89929   Name:/dev/sdd            State:online    Size (GiB):199     Used (GiB):32      Free (GiB):167     
			Bricks:
				Id:0a612ebcba5958e816bd5acaeafdb083   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_0a612ebcba5958e816bd5acaeafdb083/brick
				Id:2e551b8d470fda3a439c1b312209a157   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_2e551b8d470fda3a439c1b312209a157/brick
				Id:4f36fd69b612bd591546352dd238aa95   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_4f36fd69b612bd591546352dd238aa95/brick
				Id:5af9dd87d54b89d321379e4c070493fd   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_5af9dd87d54b89d321379e4c070493fd/brick
				Id:5b0d17c3783fa22f00bf82f7069e295d   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_5b0d17c3783fa22f00bf82f7069e295d/brick
				Id:72bdb3359d9fbf352150b6e4d3d8c054   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_72bdb3359d9fbf352150b6e4d3d8c054/brick
				Id:7e63c67287bd76187c5b930c5331ec19   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_7e63c67287bd76187c5b930c5331ec19/brick
				Id:7eca9c43426301cd7cd3132e48672b6b   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_7eca9c43426301cd7cd3132e48672b6b/brick
				Id:8fa522925433d4ce685ada1124c3ccbf   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_8fa522925433d4ce685ada1124c3ccbf/brick
				Id:90f4d0eed811840a122de2fb72225f26   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_90f4d0eed811840a122de2fb72225f26/brick
				Id:9e6922aed343c011e64f1b158c56ea7a   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_9e6922aed343c011e64f1b158c56ea7a/brick
				Id:e6bf9ea4478f3dfdabf34f85f422361e   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_1834ba558b62b4576dc6782c7cc89929/brick_e6bf9ea4478f3dfdabf34f85f422361e/brick
		Id:54c725f09071d8ef33e17c1279b24931   Name:/dev/sde            State:online    Size (GiB):199     Used (GiB):31      Free (GiB):168     
			Bricks:
				Id:0920f4c77505071707008801112ef365   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_0920f4c77505071707008801112ef365/brick
				Id:12b5fe102300188a4b2046f05adfb3c1   Size (GiB):1       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_12b5fe102300188a4b2046f05adfb3c1/brick
				Id:13aa805f1febbfe74e485c0501d2d481   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_13aa805f1febbfe74e485c0501d2d481/brick
				Id:2b92b650ff97f762372caa04fdeb6801   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_2b92b650ff97f762372caa04fdeb6801/brick
				Id:2e32c01372864c3e428057f7d23c9442   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_2e32c01372864c3e428057f7d23c9442/brick
				Id:6065a4923e537303424e997dd302a2de   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_6065a4923e537303424e997dd302a2de/brick
				Id:b2262d32493b4427012850aba872dc77   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_b2262d32493b4427012850aba872dc77/brick
				Id:b5059c0b284ea5a32cc3bab81f79acfe   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_b5059c0b284ea5a32cc3bab81f79acfe/brick
				Id:d11574b7e27f8b186ad664b7b3042f53   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_d11574b7e27f8b186ad664b7b3042f53/brick
				Id:f5f484be58329b901293e750e9a41613   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_54c725f09071d8ef33e17c1279b24931/brick_f5f484be58329b901293e750e9a41613/brick

	Node Id: c7849230793eb0ae5bec0e0826e92347
	State: online
	Cluster Id: 90c2d17d3c38caf7babe55a21811b618
	Zone: 1
	Management Hostnames: dhcp47-164.lab.eng.blr.redhat.com
	Storage Hostnames: 10.70.47.164
	Devices:
		Id:aefc534d597a52cc661048b7b8bc300e   Name:/dev/sde            State:online    Size (GiB):199     Used (GiB):30      Free (GiB):169     
			Bricks:
				Id:191536cc9132d24bb967a5003ba0ed03   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_191536cc9132d24bb967a5003ba0ed03/brick
				Id:46f82c9a923d8812eb88f03802b54d6a   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_46f82c9a923d8812eb88f03802b54d6a/brick
				Id:582a4d6001441f002df75bbe5346cca3   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_582a4d6001441f002df75bbe5346cca3/brick
				Id:65dac6dcd77cf95d0fc083bcdfc4a45f   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_65dac6dcd77cf95d0fc083bcdfc4a45f/brick
				Id:99bca3bd2c3bc84d921bf846c838df04   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_99bca3bd2c3bc84d921bf846c838df04/brick
				Id:c9c2a3f6e513d8e7d56a4af7c3eb18e0   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_c9c2a3f6e513d8e7d56a4af7c3eb18e0/brick
				Id:dad8f8d4c4229b180fc30c9690084948   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_dad8f8d4c4229b180fc30c9690084948/brick
				Id:e0fa69cf282c63e626ec4dcd01111a0b   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_e0fa69cf282c63e626ec4dcd01111a0b/brick
				Id:e73fd4e2ea30078fc3c0c8ff57f3a48d   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_e73fd4e2ea30078fc3c0c8ff57f3a48d/brick
				Id:fe42e060b0a1de8156345178fbfd12fa   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_aefc534d597a52cc661048b7b8bc300e/brick_fe42e060b0a1de8156345178fbfd12fa/brick
		Id:df5066ca750f1e87aca0dbe629a478c6   Name:/dev/sdd            State:online    Size (GiB):199     Used (GiB):33      Free (GiB):166     
			Bricks:
				Id:4ebe9e2154f457321b9309e088bda1f7   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_4ebe9e2154f457321b9309e088bda1f7/brick
				Id:67d6256b51fc41647a6cf4abc026033e   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_67d6256b51fc41647a6cf4abc026033e/brick
				Id:851d1a47c7c7da4cbb1597511649b216   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_851d1a47c7c7da4cbb1597511649b216/brick
				Id:a0e76c408c11d4d2a2086337a1153490   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_a0e76c408c11d4d2a2086337a1153490/brick
				Id:a883487f215f7a25dceb981eb613053c   Size (GiB):1       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_a883487f215f7a25dceb981eb613053c/brick
				Id:ab8dfede46f90a97f7d21bdd83d86db8   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_ab8dfede46f90a97f7d21bdd83d86db8/brick
				Id:b93d132f8482dcdbc531d6705a824169   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_b93d132f8482dcdbc531d6705a824169/brick
				Id:bdaab9427f7db778268d272438f5e9bc   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_bdaab9427f7db778268d272438f5e9bc/brick
				Id:d55b7d860e9bffe18107066fe7f92918   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_d55b7d860e9bffe18107066fe7f92918/brick
				Id:d69aa58a8a07dc46e9eee522b64238f3   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_d69aa58a8a07dc46e9eee522b64238f3/brick
				Id:d86e85e97af107a09f4e4a0c67c61bbe   Size (GiB):2       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_d86e85e97af107a09f4e4a0c67c61bbe/brick
				Id:ed29fb678b8e4d6d466278848eb56606   Size (GiB):4       Path: /var/lib/heketi/mounts/vg_df5066ca750f1e87aca0dbe629a478c6/brick_ed29fb678b8e4d6d466278848eb56606/brick

Comment 2 vinutha 2018-05-06 00:06:39 UTC

Created attachment 1432163 [details]
Heketi logs

Comment 3 vinutha 2018-05-06 01:49:28 UTC

Created attachment 1432200 [details]
sosreports

Comment 4 John Mulligan 2018-05-09 13:50:54 UTC

The heketi logs don't seem to cover the error condition reported. The error emitted 'Unable to determine number of logical volumes in thin pool' is only reported when the command "lvs --options=lv_name,thin_count --separator=:" fails. However, I can not find any lvs commands issued in the attached logs.
In addition the sosrepots attachment appears to refer to a local path.

Can you reproduce and attach a heketi log from immediately after the failure? Thank you.

Comment 6 vinutha 2018-05-10 04:34:59 UTC

Comment on attachment 1432200 [details]
sosreports

SOSreport is placed at 

qe.eng.blr.redhat.com - /var/www/html/sosreports/bug.1574990