Bug 1637688
| Summary: | OCS 3.10 Multipath Adds ALUA which may not always be supported | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Matthew Robson <mrobson> |
| Component: | gluster-block | Assignee: | Prasanna Kumar Kalever <prasanna.kalever> |
| Status: | CLOSED ERRATA | QA Contact: | RamaKasturi <knarra> |
| Severity: | urgent | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | cns-3.10 | CC: | akhakhar, bgoyal, bkunal, grodrigu, hchiramm, jarrpa, knarra, kramdoss, madam, nberry, pkarampu, pprakash, prasanna.kalever, rhs-bugs, rtalur, sankarshan, sarora, vbellur, vinug, xiubli |
| Target Milestone: | --- | Keywords: | ZStream |
| Target Release: | OCS 3.11.1 | ||
| Hardware: | All | ||
| OS: | All | ||
| Whiteboard: | |||
| Fixed In Version: | gluster-block-0.2.1-28.el7rhgs | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-02-07 03:38:29 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1638002, 1638988 | ||
| Bug Blocks: | 1641915, 1644156 | ||
Verified the following in the build gluster-block-0.2.1-29.el7rhgs.x86_64.
created block volumes in cns3.9 where gluster-block version is gluster-block-0.2.1-14.1.el7rhgs.x86_64
sh-4.2# targetcli ls
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
| o- user:glfs .............................................................................................. [Storage Objects: 5]
| o- vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/00dab599-d5c0-4dc7-bf6f-3241ac3558c9 (5.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/df906384-cef4-4d2d-abc2-7ab74543afc3 (6.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/bc0e2585-9b6a-4179-b06b-d30ffe6ea095 (7.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9 (4.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/94ab3b93-8dd3-4a38-8809-49dc6c895f42 (8.0GiB) activated]
glusterfs-storage-q5v4g
+++++++++++++++++++++++
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
| o- user:glfs .............................................................................................. [Storage Objects: 5]
| o- vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.46.23/block-store/00dab599-d5c0-4dc7-bf6f-3241ac3558c9 (5.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.46.23/block-store/df906384-cef4-4d2d-abc2-7ab74543afc3 (6.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.46.23/block-store/bc0e2585-9b6a-4179-b06b-d30ffe6ea095 (7.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.46.23/block-store/a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9 (4.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.46.23/block-store/94ab3b93-8dd3-4a38-8809-49dc6c895f42 (8.0GiB) activated]
| o- alua ................................................................................................... [ALUA Groups: 1]
| o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
glusterfs-storage-qjfqr
+++++++++++++++++++++++
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
| o- user:glfs .............................................................................................. [Storage Objects: 5]
| o- vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.80/block-store/00dab599-d5c0-4dc7-bf6f-3241ac3558c9 (5.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.80/block-store/df906384-cef4-4d2d-abc2-7ab74543afc3 (6.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.80/block-store/bc0e2585-9b6a-4179-b06b-d30ffe6ea095 (7.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.80/block-store/a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9 (4.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.80/block-store/94ab3b93-8dd3-4a38-8809-49dc6c895f42 (8.0GiB) activated]
| o- alua ................................................................................................... [ALUA Groups: 1]
| o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
[root@dhcp46-160 ~]# oc rsh glusterfs-storage-t788q
sh-4.2# cat /etc/target/saveconfig.json |grep iqn
"wwn":"iqn.2016-12.org.gluster-block:00dab599-d5c0-4dc7-bf6f-3241ac3558c9"
"wwn":"iqn.2016-12.org.gluster-block:df906384-cef4-4d2d-abc2-7ab74543afc3"
"wwn":"iqn.2016-12.org.gluster-block:bc0e2585-9b6a-4179-b06b-d30ffe6ea095"
"wwn":"iqn.2016-12.org.gluster-block:a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9"
"wwn":"iqn.2016-12.org.gluster-block:94ab3b93-8dd3-4a38-8809-49dc6c895f42"
After upgrade :
=======================
[root@dhcp46-160 ~]# oc rsh glusterfs-storage-qsrz5
sh-4.2# targetcli ls
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
| o- user:glfs .............................................................................................. [Storage Objects: 5]
| o- vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/00dab599-d5c0-4dc7-bf6f-3241ac3558c9 (5.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 3]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| | o- glfs_tg_pt_gp_ano .................................................................. [ALUA state: Active/non-optimized]
| | o- glfs_tg_pt_gp_ao ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/df906384-cef4-4d2d-abc2-7ab74543afc3 (6.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 3]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| | o- glfs_tg_pt_gp_ano .................................................................. [ALUA state: Active/non-optimized]
| | o- glfs_tg_pt_gp_ao ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/bc0e2585-9b6a-4179-b06b-d30ffe6ea095 (7.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 3]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| | o- glfs_tg_pt_gp_ano .................................................................. [ALUA state: Active/non-optimized]
| | o- glfs_tg_pt_gp_ao ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9 (4.0GiB) activated]
| | o- alua ................................................................................................... [ALUA Groups: 3]
| | o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| | o- glfs_tg_pt_gp_ano .................................................................. [ALUA state: Active/non-optimized]
| | o- glfs_tg_pt_gp_ao ....................................................................... [ALUA state: Active/optimized]
| o- vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/94ab3b93-8dd3-4a38-8809-49dc6c895f42 (8.0GiB) activated]
| o- alua ................................................................................................... [ALUA Groups: 3]
| o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- glfs_tg_pt_gp_ano .................................................................. [ALUA state: Active/non-optimized]
| o- glfs_tg_pt_gp_ao ....................................................................... [ALUA state: Active/optimized]
o- iscsi ............................................................................................................ [Targets: 5]
| o- iqn.2016-12.org.gluster-block:00dab599-d5c0-4dc7-bf6f-3241ac3558c9 ................................................ [TPGs: 3]
| | o- tpg1 ..................................................................................... [gen-acls, tpg-auth, 1-way auth]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ...................... [user/vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ao)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.37:3260 ................................................................................................. [OK]
| | o- tpg2 ........................................................................................................... [disabled]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.80:3260 ................................................................................................. [OK]
| | o- tpg3 ........................................................................................................... [disabled]
| | o- acls .......................................................................................................... [ACLs: 0]
| | o- luns .......................................................................................................... [LUNs: 1]
| | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim1_0d165c58-eda7-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | o- portals .................................................................................................... [Portals: 1]
| | o- 10.70.46.23:3260 ................................................................................................. [OK]
| o- iqn.2016-12.org.gluster-block:94ab3b93-8dd3-4a38-8809-49dc6c895f42 ................................................ [TPGs: 3]
| | o- tpg1 ..................................................................................... [gen-acls, tpg-auth, 1-way auth]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ...................... [user/vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ao)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.37:3260 ................................................................................................. [OK]
| | o- tpg2 ........................................................................................................... [disabled]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.46.23:3260 ................................................................................................. [OK]
| | o- tpg3 ........................................................................................................... [disabled]
| | o- acls .......................................................................................................... [ACLs: 0]
| | o- luns .......................................................................................................... [LUNs: 1]
| | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim5_5eaadd88-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | o- portals .................................................................................................... [Portals: 1]
| | o- 10.70.47.80:3260 ................................................................................................. [OK]
| o- iqn.2016-12.org.gluster-block:a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9 ................................................ [TPGs: 3]
| | o- tpg1 ........................................................................................................... [disabled]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ...................... [user/vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ao)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.80:3260 ................................................................................................. [OK]
| | o- tpg2 ..................................................................................... [gen-acls, tpg-auth, 1-way auth]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.37:3260 ................................................................................................. [OK]
| | o- tpg3 ........................................................................................................... [disabled]
| | o- acls .......................................................................................................... [ACLs: 0]
| | o- luns .......................................................................................................... [LUNs: 1]
| | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim4_54687b22-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | o- portals .................................................................................................... [Portals: 1]
| | o- 10.70.46.23:3260 ................................................................................................. [OK]
| o- iqn.2016-12.org.gluster-block:bc0e2585-9b6a-4179-b06b-d30ffe6ea095 ................................................ [TPGs: 3]
| | o- tpg1 ..................................................................................... [gen-acls, tpg-auth, 1-way auth]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.37:3260 ................................................................................................. [OK]
| | o- tpg2 ........................................................................................................... [disabled]
| | | o- acls .......................................................................................................... [ACLs: 0]
| | | o- luns .......................................................................................................... [LUNs: 1]
| | | | o- lun0 ...................... [user/vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ao)]
| | | o- portals .................................................................................................... [Portals: 1]
| | | o- 10.70.47.80:3260 ................................................................................................. [OK]
| | o- tpg3 ........................................................................................................... [disabled]
| | o- acls .......................................................................................................... [ACLs: 0]
| | o- luns .......................................................................................................... [LUNs: 1]
| | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim3_4c1e4cbf-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | o- portals .................................................................................................... [Portals: 1]
| | o- 10.70.46.23:3260 ................................................................................................. [OK]
| o- iqn.2016-12.org.gluster-block:df906384-cef4-4d2d-abc2-7ab74543afc3 ................................................ [TPGs: 3]
| o- tpg1 ..................................................................................... [gen-acls, tpg-auth, 1-way auth]
| | o- acls .......................................................................................................... [ACLs: 0]
| | o- luns .......................................................................................................... [LUNs: 1]
| | | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| | o- portals .................................................................................................... [Portals: 1]
| | o- 10.70.47.37:3260 ................................................................................................. [OK]
| o- tpg2 ........................................................................................................... [disabled]
| | o- acls .......................................................................................................... [ACLs: 0]
| | o- luns .......................................................................................................... [LUNs: 1]
| | | o- lun0 ...................... [user/vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ao)]
| | o- portals .................................................................................................... [Portals: 1]
| | o- 10.70.46.23:3260 ................................................................................................. [OK]
| o- tpg3 ........................................................................................................... [disabled]
| o- acls .......................................................................................................... [ACLs: 0]
| o- luns .......................................................................................................... [LUNs: 1]
| | o- lun0 ..................... [user/vol-39_glusterfs_blockclaim2_3cf680ce-eda8-11e8-9f94-0a580a820002 (glfs_tg_pt_gp_ano)]
| o- portals .................................................................................................... [Portals: 1]
| o- 10.70.47.80:3260 ................................................................................................. [OK]
o- loopback ......................................................................................................... [Targets: 0]
created new block volume with new sc :
==========================================
o- vol-39_glusterfs_blockclaim7_89ce2133-ef3e-11e8-8cef-0a580a80004c [vol_fb52bfcbc97ebf63b5e540f7da250dd9.47.37/block-store/c662d970-3c25-4606-9ce8-99406f93db1c (9.0GiB) activated]
| o- alua ................................................................................................... [ALUA Groups: 3]
| o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
| o- glfs_tg_pt_gp_ano .................................................................. [ALUA state: Active/non-optimized]
| o- glfs_tg_pt_gp_ao ....................................................................... [ALUA state: Active/optimized]
PRIO_PATH set for new block volume created :
==================================================
[root@dhcp46-160 block-meta]# cat vol-39_glusterfs_blockclaim7_89ce2133-ef3e-11e8-8cef-0a580a80004c
VOLUME: vol_fb52bfcbc97ebf63b5e540f7da250dd9
GBID: c662d970-3c25-4606-9ce8-99406f93db1c
HA: 3
ENTRYCREATE: INPROGRESS
PRIOPATH: 10.70.46.222
SIZE: 9663676416
RINGBUFFER: 0
ENTRYCREATE: SUCCESS
PASSWORD: 4908c303-978d-48dc-b4bb-1eb12f4dbdb6
10.70.47.37: CONFIGINPROGRESS
10.70.46.222: CONFIGINPROGRESS
10.70.46.23: CONFIGINPROGRESS
10.70.46.222: CONFIGSUCCESS
10.70.46.222: AUTHENFORCED
10.70.47.37: CONFIGSUCCESS
10.70.47.37: AUTHENFORCED
10.70.46.23: CONFIGSUCCESS
10.70.46.23: AUTHENFORCED
sh-4.2# cat /etc/target/saveconfig.json |grep iqn
"wwn": "iqn.2016-12.org.gluster-block:00dab599-d5c0-4dc7-bf6f-3241ac3558c9"
"wwn": "iqn.2016-12.org.gluster-block:df906384-cef4-4d2d-abc2-7ab74543afc3"
"wwn": "iqn.2016-12.org.gluster-block:bc0e2585-9b6a-4179-b06b-d30ffe6ea095"
"wwn": "iqn.2016-12.org.gluster-block:a97f81e6-6eb0-48d2-8c3f-820a2e2c98c9"
"wwn": "iqn.2016-12.org.gluster-block:94ab3b93-8dd3-4a38-8809-49dc6c895f42"
"wwn": "iqn.2016-12.org.gluster-block:c662d970-3c25-4606-9ce8-99406f93db1c"
[root@dhcp46-160 ~]# cat /var/log/dmesg | grep -i alua
[ 1.266705] alua: device handler registered
[root@dhcp46-160 ~]# cat /var/log/dmesg | grep 'alua not supported'
[root@dhcp46-160 ~]#
[root@dhcp46-160 log]# cat dmesg |grep 'failed in domap for addition of new path'
[root@dhcp46-160 log]#
sh-4.2# systemctl cat tcmu-runner
# /usr/lib/systemd/system/tcmu-runner.service
[Unit]
Description=LIO Userspace-passthrough daemon
After=glusterd.service
[Service]
LimitNOFILE=1000000
Type=dbus
BusName=org.kernel.TCMUService1
KillMode=process
EnvironmentFile=-/etc/sysconfig/tcmu-runner-params
ExecStartPre=-/usr/libexec/gluster-block/upgrade_activities.sh
EnvironmentFile=-/etc/sysconfig/tcmu-runner-params
ExecStart=/usr/bin/tcmu-runner --tcmu-log-dir $TCMU_LOGDIR
TimeoutStartSec=600
sh-4.2# ip a|grep 10.70
inet 10.70.46.23/22 brd 10.70.47.255 scope global noprefixroute dynamic ens192
sh-4.2# ls -l /etc/target/
total 80
drwxr-xr-x. 2 root root 252 Nov 23 16:40 backup
-rw-------. 1 root root 27812 Nov 23 16:40 saveconfig.json
-rw-------. 1 root root 50703 Nov 22 06:33 saveconfig.json-bak-2018-11-23-10:04:20
sh-4.2# ls -l /var/lib/gluster-block/gb_upgrade.status
-rw-r--r--. 1 root root 0 Nov 23 10:04 /var/lib/gluster-block/gb_upgrade.status
sh-4.2# date
Fri Nov 23 17:06:15 UTC 2018
sh-4.2# systemctl restart tcmu-runner
sh-4.2# tail -f /var/log/glusterfs/gluster-block/gluster-block-bricks-start.log
[2018-11-23 06:08:51] INFO: All bricks for Block Hosting Volumes came online in 17 seconds
[2018-11-23 10:04:19] INFO: All bricks for Block Hosting Volumes came online in 54 seconds
[2018-11-23 10:24:29] INFO: All bricks for Block Hosting Volumes came online in 31 seconds
[2018-11-23 01:10:16] INFO: All bricks for Block Hosting Volumes came online in 18 seconds
[2018-11-23 05:06:40] INFO: All bricks for Block Hosting Volumes came online in 0 seconds
sh-4.2# tail -f /var/log/glusterfs/gluster-block/gluster-block-upgrade-activities.log
[2018-11-23 06:08:51] INFO: started upgrade activities
[2018-11-23 06:08:51] ERROR: Gluster-blockd is not running
[2018-11-23 10:04:19] INFO: started upgrade activities
[2018-11-23 10:04:20] INFO: successfully completed the upgrade activities
[2018-11-23 10:24:29] INFO: skipping upgrade activities
[2018-11-23 01:10:16] INFO: skipping upgrade activities
[2018-11-23 05:06:40] INFO: skipping upgrade activities
Have not specifically verified the upgrade_activities.sh script because this has already been verified in the other bug https://bugzilla.redhat.com/show_bug.cgi?id=1638988#c7 and the fixed in version is same for BUG 1637688 and 1638988
Hello Neha, I am planning to develop below cases for this bug and let me know if the test case list looks good or if you would like me to include anything else as part of this. Verify if targetcli ls displays alua supported for old block volumes once upgraded from cns3.9 to cns3.11 Verify creating new block volumes on the upgraded setup has the functionality of alua included Verify restarting tcmu-runner does not trigger upgrade_activities.sh Verify pod restart does not trigger upgrade_activities.sh Validate upgrade_activities.sh script Thanks kasturi (In reply to RamaKasturi from comment #28) > Hello Neha, > > I am planning to develop below cases for this bug and let me know if the > test case list looks good or if you would like me to include anything else > as part of this. > > Verify if targetcli ls displays alua supported for old block volumes once > upgraded from cns3.9 to cns3.11 > > Verify creating new block volumes on the upgraded setup has the > functionality of alua included > > Verify restarting tcmu-runner does not trigger upgrade_activities.sh > > Verify pod restart does not trigger upgrade_activities.sh > > Validate upgrade_activities.sh script > > Thanks > kasturi Hi Kasturi, these tests look good. But maybe we can confirm with Prasanna if any other coverage is required. Also, for test#1 below: Verify if targetcli ls displays alua supported for old block volumes once > upgraded from cns3.9 to cns3.11 we would need to edit the multipath.conf file to include "alua" once all pods are upgraded. Regards, neha Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:0285 |
Description of problem: OCS 3.10 changes multipath conf to use ALUA, but doesn't call out the change or ensure ensure customers backend storage supports it. This may lead to failures in the upgrade. Existing gluster-block devices created in CNS 3.9 failed with this change: Oct 03 15:31:10 ociopf-t-322.dmz multipathd[108699]: sdf: alua not supported Oct 03 15:31:10 ociopf-t-322.dmz kernel: sd 46:0:0:0: alua: not supported Oct 03 15:31:10 ociopf-t-322.dmz kernel: sd 46:0:0:0: alua: not attached Oct 03 15:31:10 ociopf-t-322.dmz multipathd[108699]: sdf: alua not supported Oct 03 15:31:12 ociopf-t-322.dmz multipathd[108699]: sdg: alua not supported Oct 03 15:31:12 ociopf-t-322.dmz multipathd[108699]: sdf: alua not supported Oct 03 15:31:12 ociopf-t-322.dmz multipathd[108699]: sdg: alua not supported Oct 03 15:31:12 ociopf-t-322.dmz kernel: sd 46:0:0:0: alua: not supported Oct 03 15:31:12 ociopf-t-322.dmz kernel: sd 46:0:0:0: alua: not attached Oct 03 15:31:13 ociopf-t-322.dmz multipathd[108699]: sdh: alua not supported Oct 03 15:31:13 ociopf-t-322.dmz multipathd[108699]: sdf: alua not supported Oct 03 15:31:13 ociopf-t-322.dmz multipathd[108699]: sdg: alua not supported Oct 03 15:31:13 ociopf-t-322.dmz multipathd[108699]: sdh: alua not supported Oct 03 15:31:13 ociopf-t-322.dmz kernel: sd 46:0:0:0: alua: not supported Oct 03 15:31:13 ociopf-t-322.dmz kernel: sd 46:0:0:0: alua: not attached Gluster Block Physical Volumes: [root@server ~]# pvs | grep -E "vg_4e8106ce4a9c5278a4f231231ac3d915|vg_0aa5d6dad3331d73c048bc1d2dd575a0" /dev/sdd vg_4e8106ce4a9c5278a4f231231ac3d915 lvm2 a-- 99.87g 14.25g /dev/sdg vg_0aa5d6dad3331d73c048bc1d2dd575a0 lvm2 a-- 249.87g <132.11g Multipath Devices: [root@ociopf-t-322 ~]# multipath -ll mpathh (36001405df12f77c20d14a089583d5536) dm-48 LIO-ORG ,TCMU device size=2.0G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=1 status=active | `- 54:0:0:0 sdg 8:96 active ready running |-+- policy='round-robin 0' prio=1 status=enabled | `- 57:0:0:0 sdj 8:144 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 58:0:0:0 sdk 8:160 active ready running mpathg (3600140505d45aa0050948a6abe73e172) dm-49 LIO-ORG ,TCMU device size=1.0G features='1 queue_if_no_path' hwhandler='0' wp=rw |-+- policy='round-robin 0' prio=1 status=active | `- 52:0:0:0 sdf 8:80 active ready running |-+- policy='round-robin 0' prio=1 status=enabled | `- 55:0:0:0 sdh 8:112 active ready running `-+- policy='round-robin 0' prio=1 status=enabled `- 56:0:0:0 sdi 8:128 active ready running sdg is from mpathh which is a new gluster block devices created post upgrade: [root@server ~]# sg_rtpg -d /dev/sdg Report target port groups: target port group id : 0x2 , Pref=0 target port group asymmetric access state : 0x01 (active/non optimized) T_SUP : 1, O_SUP : 1, LBD_SUP : 0, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1 status code : 0x02 (target port asym. state changed by implicit lu behaviour) vendor unique status : 0x00 target port count : 02 Relative target port ids: 0x02 0x03 target port group id : 0x1 , Pref=0 target port group asymmetric access state : 0x00 (active/optimized) T_SUP : 1, O_SUP : 1, LBD_SUP : 0, U_SUP : 1, S_SUP : 1, AN_SUP : 1, AO_SUP : 1 status code : 0x00 (no status available) vendor unique status : 0x00 target port count : 01 Relative target port ids: 0x01 This is a gluster block device which existed pre-upgrade. It does not seem to have support: [root@server ~]# sg_rtpg -d /dev/sdh Report Target Port Groups command not supported sh-4.2# grep alua_tg_pt_gp_name /etc/target/saveconfig.json "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": null, "alua_tg_pt_gp_name": "glfs_tg_pt_gp_ano", "alua_tg_pt_gp_name": "glfs_tg_pt_gp_ano", "alua_tg_pt_gp_name": "glfs_tg_pt_gp_ao", Version-Release number of selected component (if applicable): OCS 3.10 How reproducible: Always Steps to Reproduce: 1. Upgrade from CNS 3.9 with existing block volumes 2. Change multipath conf per 3.10 3. If storage device doesn't support ALUA, it will fail 4. Actual results: Can not apply updated multipath config Expected results: Should apply without issue and note the caveats. Additional info: