Bug 1902468
Summary: | Support 50 concurrent add_lockspace operations | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 8 | Reporter: | Nir Soffer <nsoffer> | |
Component: | sanlock | Assignee: | David Teigland <teigland> | |
Status: | CLOSED ERRATA | QA Contact: | cluster-qe <cluster-qe> | |
Severity: | high | Docs Contact: | ||
Priority: | high | |||
Version: | 8.3 | CC: | aefrat, agk, cluster-maint, cmarthal, jbrassow, mtessun, rhandlin, teigland, vjuranek | |
Target Milestone: | rc | Keywords: | Triaged, ZStream | |
Target Release: | 8.4 | |||
Hardware: | Unspecified | |||
OS: | Unspecified | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | If docs needed, set a value | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | ||||
: | 1912923 (view as bug list) | Environment: | ||
Last Closed: | 2021-05-18 15:09:01 UTC | Type: | Bug | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | ||||
Bug Blocks: | 1845909, 1903358, 1912923 |
Description
Nir Soffer
2020-11-29 11:37:56 UTC
I posted patch to allow configuring max_worker_threads via /etc/sanlock.conf: https://lists.fedorahosted.org/archives/list/sanlock-devel@lists.fedorahosted.org/thread/X3B2KMSNIRO7SOCRMTTV7CN7AACYLC3G/ This is good enough for RHV to tune this value. David, can we this change in 8.3.z? pushed to git https://pagure.io/sanlock/c/e722c69cef883af1f7d60cbb2708f0092dbf2ab0 Putting in zstream is ok with me, and would just need the necessary bug with approvals to be created which I'm not able to do. Since RHV QE will be testing this would they provide the qa_ack? (In reply to David Teigland from comment #4) > Since RHV QE will be testing this would they provide the qa_ack? Avihay, can you test this bug? The next RHV build will use this feature (see https://gerrit.ovirt.org/c/vdsm/+/112498) So testing should be pretty easy. David, can we get a scratch build for testing? It should include these patches: 32a73a5 sanlock: Report actual max_worker_threads e722c69 config: Add max_worker_threads It would be nice to have also this patch: 0ff9c1a sanlock: Shrink thread pool when there is no work scratch build from top of current master branch https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=33533874 I tested this change with vdsm 4.40.39, with 40 storage domains, and it works fine. Tested configuration: $ cat /etc/vdsm/vdsm.conf.d/99-foobar.conf # Configuration for foobar storage. [sanlock] # Set renewal timeout to 160 seconds # (8 * io_timeout == 160). io_timeout = 20 $ cat /etc/sanlock/sanlock.conf # Configuration for vdsm our_host_name = 095bce26-9d5b-4ae2-b3bf-dddfc9cdf541 max_worker_threads = 50 $ cat /etc/multipath/conf.d/foobar.conf # Configuration for foobar storage. overrides { # Queue I/O for 160 seconds when all paths fail # (no_path_retry * polling_interval == 160). no_path_retry 32 } # sanlock client status -D daemon 095bce26-9d5b-4ae2-b3bf-dddfc9cdf541 our_host_name=095bce26-9d5b-4ae2-b3bf-dddfc9cdf541 use_watchdog=1 high_priority=0 mlock_level=1 quiet_fail=1 debug_renew=0 debug_clients=0 debug_cmds=0xfffffffffe06ffff renewal_history_size=180 gid=179 uid=179 sh_retries=8 max_sectors_kb_ignore=0 max_sectors_kb_align=0 max_sectors_kb_num=1024 max_worker_threads=50 write_init_io_timeout=60 use_aio=1 kill_grace_seconds=40 helper_pid=964 helper_kill_fd=7 helper_full_count=0 helper_last_status=102814 monotime=102829 version_str=3.8.2 version_num=3.8.2 version_hex=03080200 smproto_hex=00000001 ... When adding lockspaces, we can see that we add all lockspaces concurrently: # sanlock client status daemon 095bce26-9d5b-4ae2-b3bf-dddfc9cdf541 p -1 helper p -1 listener p -1 status p 3147 s 1f4e2acd-6224-4345-ae06-1db97c403a0c:1:/rhev/data-center/mnt/storage\:_export_11/1f4e2acd-6224-4345-ae06-1db97c403a0c/dom_md/ids:0 ADD s b2ff7cd6-64a6-453d-b71e-22fa4b11f16e:1:/rhev/data-center/mnt/storage\:_export_00/b2ff7cd6-64a6-453d-b71e-22fa4b11f16e/dom_md/ids:0 ADD s e0549942-c08c-478b-9c8d-c116e98b59d4:1:/rhev/data-center/mnt/storage\:_export_09/e0549942-c08c-478b-9c8d-c116e98b59d4/dom_md/ids:0 ADD s 27afe55d-68d6-49c1-a085-83425b4bdd67:1:/rhev/data-center/mnt/storage\:_export_14/27afe55d-68d6-49c1-a085-83425b4bdd67/dom_md/ids:0 ADD s 1297b300-a126-45e8-8a7f-c69edcf057f1:1:/rhev/data-center/mnt/storage\:_export_08/1297b300-a126-45e8-8a7f-c69edcf057f1/dom_md/ids:0 ADD s cd205f88-9b6a-46c6-9993-8569a5dfb8f1:1:/rhev/data-center/mnt/storage\:_export_15/cd205f88-9b6a-46c6-9993-8569a5dfb8f1/dom_md/ids:0 ADD s 0d38fccd-5585-40f7-9a73-d215b23bf7ed:1:/rhev/data-center/mnt/storage\:_export_18/0d38fccd-5585-40f7-9a73-d215b23bf7ed/dom_md/ids:0 ADD s 09525896-e9c1-4d24-8e4d-dab18346a274:1:/rhev/data-center/mnt/storage\:_export_07/09525896-e9c1-4d24-8e4d-dab18346a274/dom_md/ids:0 ADD s a87b183d-df2d-4de3-b370-507c6261c328:1:/rhev/data-center/mnt/storage\:_export_02/a87b183d-df2d-4de3-b370-507c6261c328/dom_md/ids:0 ADD s 839db77f-fde3-4e13-bfb6-56be604631ed:1:/rhev/data-center/mnt/storage\:_export_01/839db77f-fde3-4e13-bfb6-56be604631ed/dom_md/ids:0 ADD s 7e25e16e-2cdf-4063-a638-3d61c0427ac6:1:/rhev/data-center/mnt/storage\:_export_19/7e25e16e-2cdf-4063-a638-3d61c0427ac6/dom_md/ids:0 ADD s 8f870f86-3e4e-44a3-9bc6-c0af6f9d7944:1:/rhev/data-center/mnt/storage\:_export_05/8f870f86-3e4e-44a3-9bc6-c0af6f9d7944/dom_md/ids:0 ADD s 2b5b232f-4a59-4ace-9e18-3841c63505de:1:/rhev/data-center/mnt/storage\:_export_16/2b5b232f-4a59-4ace-9e18-3841c63505de/dom_md/ids:0 ADD s 2b5ccfe5-d05e-43d8-ae60-d36e3f7dd2e3:1:/rhev/data-center/mnt/storage\:_export_04/2b5ccfe5-d05e-43d8-ae60-d36e3f7dd2e3/dom_md/ids:0 ADD s d046c651-ac7c-49ca-9541-31e47b7c38ef:1:/rhev/data-center/mnt/storage\:_export_06/d046c651-ac7c-49ca-9541-31e47b7c38ef/dom_md/ids:0 ADD s 980601ff-1e81-4e10-bf79-a29d2d5b0e9f:1:/rhev/data-center/mnt/storage\:_export_10/980601ff-1e81-4e10-bf79-a29d2d5b0e9f/dom_md/ids:0 ADD s 82d93a3d-3d88-42ca-a328-f295dfc48ef2:1:/rhev/data-center/mnt/storage\:_export_13/82d93a3d-3d88-42ca-a328-f295dfc48ef2/dom_md/ids:0 ADD s c345fc52-f82e-4904-9ad4-1cc84b0fc639:1:/rhev/data-center/mnt/storage\:_export_03/c345fc52-f82e-4904-9ad4-1cc84b0fc639/dom_md/ids:0 ADD s 0e49277b-b13e-49d6-bb56-e3da6d0cc397:1:/rhev/data-center/mnt/storage\:_export_12/0e49277b-b13e-49d6-bb56-e3da6d0cc397/dom_md/ids:0 ADD s 5bb456c1-bcef-4fb5-adf2-01de35f46f6d:1:/rhev/data-center/mnt/storage\:_export_17/5bb456c1-bcef-4fb5-adf2-01de35f46f6d/dom_md/ids:0 ADD s c202aeb7-771a-4f8c-b94b-39b805695b35:1:/dev/c202aeb7-771a-4f8c-b94b-39b805695b35/ids:0 ADD s e673e145-d644-44ac-a416-a8fc14b029ca:1:/dev/e673e145-d644-44ac-a416-a8fc14b029ca/ids:0 ADD s 06e43bfc-2ffe-419e-843b-59a5d23165e1:1:/dev/06e43bfc-2ffe-419e-843b-59a5d23165e1/ids:0 ADD s 2e7daa04-2cd4-4e93-bab8-866227a8a37b:1:/dev/2e7daa04-2cd4-4e93-bab8-866227a8a37b/ids:0 ADD s 31e3675e-302a-4a22-ad9a-402c0c1e9765:1:/dev/31e3675e-302a-4a22-ad9a-402c0c1e9765/ids:0 ADD s f53d7919-d275-4ad4-ba57-f280000de397:1:/dev/f53d7919-d275-4ad4-ba57-f280000de397/ids:0 ADD s ebf4cf14-0a33-4c3d-bd83-9f161760fecb:1:/dev/ebf4cf14-0a33-4c3d-bd83-9f161760fecb/ids:0 ADD s ae5fe4a4-cd9e-421d-b501-93ea1c23bc73:1:/dev/ae5fe4a4-cd9e-421d-b501-93ea1c23bc73/ids:0 ADD s 2f8c68ef-d5c1-4f34-9a6a-f7cb2f0aef0a:1:/dev/2f8c68ef-d5c1-4f34-9a6a-f7cb2f0aef0a/ids:0 ADD s 7cf51212-8247-41b0-ab38-da5954059d7a:1:/dev/7cf51212-8247-41b0-ab38-da5954059d7a/ids:0 ADD s 829b9bed-1b74-4f74-9d18-1006c87ef620:1:/dev/829b9bed-1b74-4f74-9d18-1006c87ef620/ids:0 ADD s 8ef0cc64-77c5-49e0-a0f5-03cee9bbee20:1:/dev/8ef0cc64-77c5-49e0-a0f5-03cee9bbee20/ids:0 ADD s f5597bd4-5c4f-4c4c-bd98-e8a33dace830:1:/dev/f5597bd4-5c4f-4c4c-bd98-e8a33dace830/ids:0 ADD s 84dc4e3c-00fd-4263-84e8-fc246eeee6e9:1:/dev/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/ids:0 ADD s 103047b0-7650-4d31-89ee-6a39da955804:1:/dev/103047b0-7650-4d31-89ee-6a39da955804/ids:0 ADD s 301a38ee-f20b-4eaa-9781-722434d8c8a0:1:/dev/301a38ee-f20b-4eaa-9781-722434d8c8a0/ids:0 ADD s 71702e1e-82d2-41d2-93df-678f2415da2a:1:/dev/71702e1e-82d2-41d2-93df-678f2415da2a/ids:0 ADD s 6749f0e2-848c-4b54-b206-51ae90109efc:1:/dev/6749f0e2-848c-4b54-b206-51ae90109efc/ids:0 ADD s 08c7e615-9fa0-4abd-8b0a-4e7f77bf3bd1:1:/dev/08c7e615-9fa0-4abd-8b0a-4e7f77bf3bd1/ids:0 ADD s 747ea907-c24c-43ff-a389-660c729703b0:1:/dev/747ea907-c24c-43ff-a389-660c729703b0/ids:0 ADD Status after all lockspaces were added: # sanlock client status daemon 095bce26-9d5b-4ae2-b3bf-dddfc9cdf541 p -1 helper p -1 listener p 79833 backup-raw p 3147 p -1 status s 82d93a3d-3d88-42ca-a328-f295dfc48ef2:1:/rhev/data-center/mnt/storage\:_export_13/82d93a3d-3d88-42ca-a328-f295dfc48ef2/dom_md/ids:0 s 7e25e16e-2cdf-4063-a638-3d61c0427ac6:1:/rhev/data-center/mnt/storage\:_export_19/7e25e16e-2cdf-4063-a638-3d61c0427ac6/dom_md/ids:0 s 0d38fccd-5585-40f7-9a73-d215b23bf7ed:1:/rhev/data-center/mnt/storage\:_export_18/0d38fccd-5585-40f7-9a73-d215b23bf7ed/dom_md/ids:0 s 0e49277b-b13e-49d6-bb56-e3da6d0cc397:1:/rhev/data-center/mnt/storage\:_export_12/0e49277b-b13e-49d6-bb56-e3da6d0cc397/dom_md/ids:0 s 2b5b232f-4a59-4ace-9e18-3841c63505de:1:/rhev/data-center/mnt/storage\:_export_16/2b5b232f-4a59-4ace-9e18-3841c63505de/dom_md/ids:0 s 839db77f-fde3-4e13-bfb6-56be604631ed:1:/rhev/data-center/mnt/storage\:_export_01/839db77f-fde3-4e13-bfb6-56be604631ed/dom_md/ids:0 s cd205f88-9b6a-46c6-9993-8569a5dfb8f1:1:/rhev/data-center/mnt/storage\:_export_15/cd205f88-9b6a-46c6-9993-8569a5dfb8f1/dom_md/ids:0 s f53d7919-d275-4ad4-ba57-f280000de397:1:/dev/f53d7919-d275-4ad4-ba57-f280000de397/ids:0 s e673e145-d644-44ac-a416-a8fc14b029ca:1:/dev/e673e145-d644-44ac-a416-a8fc14b029ca/ids:0 s 06e43bfc-2ffe-419e-843b-59a5d23165e1:1:/dev/06e43bfc-2ffe-419e-843b-59a5d23165e1/ids:0 s 301a38ee-f20b-4eaa-9781-722434d8c8a0:1:/dev/301a38ee-f20b-4eaa-9781-722434d8c8a0/ids:0 s c202aeb7-771a-4f8c-b94b-39b805695b35:1:/dev/c202aeb7-771a-4f8c-b94b-39b805695b35/ids:0 s 31e3675e-302a-4a22-ad9a-402c0c1e9765:1:/dev/31e3675e-302a-4a22-ad9a-402c0c1e9765/ids:0 s 6749f0e2-848c-4b54-b206-51ae90109efc:1:/dev/6749f0e2-848c-4b54-b206-51ae90109efc/ids:0 s ebf4cf14-0a33-4c3d-bd83-9f161760fecb:1:/dev/ebf4cf14-0a33-4c3d-bd83-9f161760fecb/ids:0 s 829b9bed-1b74-4f74-9d18-1006c87ef620:1:/dev/829b9bed-1b74-4f74-9d18-1006c87ef620/ids:0 s f5597bd4-5c4f-4c4c-bd98-e8a33dace830:1:/dev/f5597bd4-5c4f-4c4c-bd98-e8a33dace830/ids:0 s 08c7e615-9fa0-4abd-8b0a-4e7f77bf3bd1:1:/dev/08c7e615-9fa0-4abd-8b0a-4e7f77bf3bd1/ids:0 s 8ef0cc64-77c5-49e0-a0f5-03cee9bbee20:1:/dev/8ef0cc64-77c5-49e0-a0f5-03cee9bbee20/ids:0 s 71702e1e-82d2-41d2-93df-678f2415da2a:1:/dev/71702e1e-82d2-41d2-93df-678f2415da2a/ids:0 s 747ea907-c24c-43ff-a389-660c729703b0:1:/dev/747ea907-c24c-43ff-a389-660c729703b0/ids:0 s ae5fe4a4-cd9e-421d-b501-93ea1c23bc73:1:/dev/ae5fe4a4-cd9e-421d-b501-93ea1c23bc73/ids:0 s 2e7daa04-2cd4-4e93-bab8-866227a8a37b:1:/dev/2e7daa04-2cd4-4e93-bab8-866227a8a37b/ids:0 s 2f8c68ef-d5c1-4f34-9a6a-f7cb2f0aef0a:1:/dev/2f8c68ef-d5c1-4f34-9a6a-f7cb2f0aef0a/ids:0 s 7cf51212-8247-41b0-ab38-da5954059d7a:1:/dev/7cf51212-8247-41b0-ab38-da5954059d7a/ids:0 s 103047b0-7650-4d31-89ee-6a39da955804:1:/dev/103047b0-7650-4d31-89ee-6a39da955804/ids:0 s 84dc4e3c-00fd-4263-84e8-fc246eeee6e9:1:/dev/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/ids:0 s 1f4e2acd-6224-4345-ae06-1db97c403a0c:1:/rhev/data-center/mnt/storage\:_export_11/1f4e2acd-6224-4345-ae06-1db97c403a0c/dom_md/ids:0 s c345fc52-f82e-4904-9ad4-1cc84b0fc639:1:/rhev/data-center/mnt/storage\:_export_03/c345fc52-f82e-4904-9ad4-1cc84b0fc639/dom_md/ids:0 s 980601ff-1e81-4e10-bf79-a29d2d5b0e9f:1:/rhev/data-center/mnt/storage\:_export_10/980601ff-1e81-4e10-bf79-a29d2d5b0e9f/dom_md/ids:0 s e0549942-c08c-478b-9c8d-c116e98b59d4:1:/rhev/data-center/mnt/storage\:_export_09/e0549942-c08c-478b-9c8d-c116e98b59d4/dom_md/ids:0 s 8f870f86-3e4e-44a3-9bc6-c0af6f9d7944:1:/rhev/data-center/mnt/storage\:_export_05/8f870f86-3e4e-44a3-9bc6-c0af6f9d7944/dom_md/ids:0 s a87b183d-df2d-4de3-b370-507c6261c328:1:/rhev/data-center/mnt/storage\:_export_02/a87b183d-df2d-4de3-b370-507c6261c328/dom_md/ids:0 s 1297b300-a126-45e8-8a7f-c69edcf057f1:1:/rhev/data-center/mnt/storage\:_export_08/1297b300-a126-45e8-8a7f-c69edcf057f1/dom_md/ids:0 s 09525896-e9c1-4d24-8e4d-dab18346a274:1:/rhev/data-center/mnt/storage\:_export_07/09525896-e9c1-4d24-8e4d-dab18346a274/dom_md/ids:0 s d046c651-ac7c-49ca-9541-31e47b7c38ef:1:/rhev/data-center/mnt/storage\:_export_06/d046c651-ac7c-49ca-9541-31e47b7c38ef/dom_md/ids:0 s b2ff7cd6-64a6-453d-b71e-22fa4b11f16e:1:/rhev/data-center/mnt/storage\:_export_00/b2ff7cd6-64a6-453d-b71e-22fa4b11f16e/dom_md/ids:0 s 2b5ccfe5-d05e-43d8-ae60-d36e3f7dd2e3:1:/rhev/data-center/mnt/storage\:_export_04/2b5ccfe5-d05e-43d8-ae60-d36e3f7dd2e3/dom_md/ids:0 s 27afe55d-68d6-49c1-a085-83425b4bdd67:1:/rhev/data-center/mnt/storage\:_export_14/27afe55d-68d6-49c1-a085-83425b4bdd67/dom_md/ids:0 s 5bb456c1-bcef-4fb5-adf2-01de35f46f6d:1:/rhev/data-center/mnt/storage\:_export_17/5bb456c1-bcef-4fb5-adf2-01de35f46f6d/dom_md/ids:0 r 5bb456c1-bcef-4fb5-adf2-01de35f46f6d:SDM:/rhev/data-center/mnt/storage\:_export_17/5bb456c1-bcef-4fb5-adf2-01de35f46f6d/dom_md/leases:1048576:38 p 3147 When removing lockspaces, we can see that we remove all lockspaces concurrently: # sanlock client status daemon 095bce26-9d5b-4ae2-b3bf-dddfc9cdf541 p -1 helper p -1 listener p -1 rem_lockspace p 3147 p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 rem_lockspace p -1 status s 5bb456c1-bcef-4fb5-adf2-01de35f46f6d:1:/rhev/data-center/mnt/storage\:_export_17/5bb456c1-bcef-4fb5-adf2-01de35f46f6d/dom_md/ids:0 REM s 27afe55d-68d6-49c1-a085-83425b4bdd67:1:/rhev/data-center/mnt/storage\:_export_14/27afe55d-68d6-49c1-a085-83425b4bdd67/dom_md/ids:0 REM s 2b5ccfe5-d05e-43d8-ae60-d36e3f7dd2e3:1:/rhev/data-center/mnt/storage\:_export_04/2b5ccfe5-d05e-43d8-ae60-d36e3f7dd2e3/dom_md/ids:0 REM s b2ff7cd6-64a6-453d-b71e-22fa4b11f16e:1:/rhev/data-center/mnt/storage\:_export_00/b2ff7cd6-64a6-453d-b71e-22fa4b11f16e/dom_md/ids:0 REM s d046c651-ac7c-49ca-9541-31e47b7c38ef:1:/rhev/data-center/mnt/storage\:_export_06/d046c651-ac7c-49ca-9541-31e47b7c38ef/dom_md/ids:0 REM s 09525896-e9c1-4d24-8e4d-dab18346a274:1:/rhev/data-center/mnt/storage\:_export_07/09525896-e9c1-4d24-8e4d-dab18346a274/dom_md/ids:0 REM s 1297b300-a126-45e8-8a7f-c69edcf057f1:1:/rhev/data-center/mnt/storage\:_export_08/1297b300-a126-45e8-8a7f-c69edcf057f1/dom_md/ids:0 REM s a87b183d-df2d-4de3-b370-507c6261c328:1:/rhev/data-center/mnt/storage\:_export_02/a87b183d-df2d-4de3-b370-507c6261c328/dom_md/ids:0 REM s 8f870f86-3e4e-44a3-9bc6-c0af6f9d7944:1:/rhev/data-center/mnt/storage\:_export_05/8f870f86-3e4e-44a3-9bc6-c0af6f9d7944/dom_md/ids:0 REM s e0549942-c08c-478b-9c8d-c116e98b59d4:1:/rhev/data-center/mnt/storage\:_export_09/e0549942-c08c-478b-9c8d-c116e98b59d4/dom_md/ids:0 REM s 980601ff-1e81-4e10-bf79-a29d2d5b0e9f:1:/rhev/data-center/mnt/storage\:_export_10/980601ff-1e81-4e10-bf79-a29d2d5b0e9f/dom_md/ids:0 REM s c345fc52-f82e-4904-9ad4-1cc84b0fc639:1:/rhev/data-center/mnt/storage\:_export_03/c345fc52-f82e-4904-9ad4-1cc84b0fc639/dom_md/ids:0 REM s 1f4e2acd-6224-4345-ae06-1db97c403a0c:1:/rhev/data-center/mnt/storage\:_export_11/1f4e2acd-6224-4345-ae06-1db97c403a0c/dom_md/ids:0 REM s 84dc4e3c-00fd-4263-84e8-fc246eeee6e9:1:/dev/84dc4e3c-00fd-4263-84e8-fc246eeee6e9/ids:0 REM s 103047b0-7650-4d31-89ee-6a39da955804:1:/dev/103047b0-7650-4d31-89ee-6a39da955804/ids:0 REM s 7cf51212-8247-41b0-ab38-da5954059d7a:1:/dev/7cf51212-8247-41b0-ab38-da5954059d7a/ids:0 REM s 2f8c68ef-d5c1-4f34-9a6a-f7cb2f0aef0a:1:/dev/2f8c68ef-d5c1-4f34-9a6a-f7cb2f0aef0a/ids:0 REM s 2e7daa04-2cd4-4e93-bab8-866227a8a37b:1:/dev/2e7daa04-2cd4-4e93-bab8-866227a8a37b/ids:0 REM s ae5fe4a4-cd9e-421d-b501-93ea1c23bc73:1:/dev/ae5fe4a4-cd9e-421d-b501-93ea1c23bc73/ids:0 REM s 747ea907-c24c-43ff-a389-660c729703b0:1:/dev/747ea907-c24c-43ff-a389-660c729703b0/ids:0 REM s 71702e1e-82d2-41d2-93df-678f2415da2a:1:/dev/71702e1e-82d2-41d2-93df-678f2415da2a/ids:0 REM s 8ef0cc64-77c5-49e0-a0f5-03cee9bbee20:1:/dev/8ef0cc64-77c5-49e0-a0f5-03cee9bbee20/ids:0 REM s 08c7e615-9fa0-4abd-8b0a-4e7f77bf3bd1:1:/dev/08c7e615-9fa0-4abd-8b0a-4e7f77bf3bd1/ids:0 REM s f5597bd4-5c4f-4c4c-bd98-e8a33dace830:1:/dev/f5597bd4-5c4f-4c4c-bd98-e8a33dace830/ids:0 REM s 829b9bed-1b74-4f74-9d18-1006c87ef620:1:/dev/829b9bed-1b74-4f74-9d18-1006c87ef620/ids:0 REM s ebf4cf14-0a33-4c3d-bd83-9f161760fecb:1:/dev/ebf4cf14-0a33-4c3d-bd83-9f161760fecb/ids:0 REM s 6749f0e2-848c-4b54-b206-51ae90109efc:1:/dev/6749f0e2-848c-4b54-b206-51ae90109efc/ids:0 REM s 31e3675e-302a-4a22-ad9a-402c0c1e9765:1:/dev/31e3675e-302a-4a22-ad9a-402c0c1e9765/ids:0 REM s c202aeb7-771a-4f8c-b94b-39b805695b35:1:/dev/c202aeb7-771a-4f8c-b94b-39b805695b35/ids:0 REM s 301a38ee-f20b-4eaa-9781-722434d8c8a0:1:/dev/301a38ee-f20b-4eaa-9781-722434d8c8a0/ids:0 REM s 06e43bfc-2ffe-419e-843b-59a5d23165e1:1:/dev/06e43bfc-2ffe-419e-843b-59a5d23165e1/ids:0 REM s e673e145-d644-44ac-a416-a8fc14b029ca:1:/dev/e673e145-d644-44ac-a416-a8fc14b029ca/ids:0 REM s f53d7919-d275-4ad4-ba57-f280000de397:1:/dev/f53d7919-d275-4ad4-ba57-f280000de397/ids:0 REM s cd205f88-9b6a-46c6-9993-8569a5dfb8f1:1:/rhev/data-center/mnt/storage\:_export_15/cd205f88-9b6a-46c6-9993-8569a5dfb8f1/dom_md/ids:0 REM s 839db77f-fde3-4e13-bfb6-56be604631ed:1:/rhev/data-center/mnt/storage\:_export_01/839db77f-fde3-4e13-bfb6-56be604631ed/dom_md/ids:0 REM s 2b5b232f-4a59-4ace-9e18-3841c63505de:1:/rhev/data-center/mnt/storage\:_export_16/2b5b232f-4a59-4ace-9e18-3841c63505de/dom_md/ids:0 REM s 0e49277b-b13e-49d6-bb56-e3da6d0cc397:1:/rhev/data-center/mnt/storage\:_export_12/0e49277b-b13e-49d6-bb56-e3da6d0cc397/dom_md/ids:0 REM s 0d38fccd-5585-40f7-9a73-d215b23bf7ed:1:/rhev/data-center/mnt/storage\:_export_18/0d38fccd-5585-40f7-9a73-d215b23bf7ed/dom_md/ids:0 REM s 7e25e16e-2cdf-4063-a638-3d61c0427ac6:1:/rhev/data-center/mnt/storage\:_export_19/7e25e16e-2cdf-4063-a638-3d61c0427ac6/dom_md/ids:0 REM s 82d93a3d-3d88-42ca-a328-f295dfc48ef2:1:/rhev/data-center/mnt/storage\:_export_13/82d93a3d-3d88-42ca-a328-f295dfc48ef2/dom_md/ids:0 REM Here some performance results: |------------------------|------------|-----------|------------| | operation | io_timeout | 8 workers | 50 workers | |-------------------------------------------------|------------| | activate host | 10 | 123 | 47 | | | 20 | 224 | 71 | | deactivate host | 10 | 16 | 8 | | | 20 | 16 | 5 | |------------------------|------------|-----------|------------| The time is measured from performing the action in RHV UI, until all lockspaces are added (ADD is removed in "sanlock client status"). Activate host after clean shutdown took less then 2 minutes. Based on tests with 20 storage domains and 8 workers, 40 storage domains would take about 9 minutes. Since RHV QE do not have capacity to test this now, and this is a tiny change, I think this is good enough. David, do we need to clone this bug to 8.3.z to get this fix in 8.3.z? Also we need an official build with this fix for testing bug 1903358. (In reply to Nir Soffer from comment #10) > David, do we need to clone this bug to 8.3.z to get this fix in 8.3.z? I'm not sure if the zstream process involves manually cloning the bug or if that's automated. I don't think PMs do this for RHEL 8... Bob Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (sanlock bug fix and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2021:1711 |