Bug 1957926

Description of problem
======================

When "Add Capacity" functionality (as provided by OCP Console) is used on
LSO StorageCluster, the cluster will be extended by just 3 (or 4 when stretch
cluster mode is enabled) devices from local device set, no matter how many free
local devices are actually available in selected ocs storage class (local
volume set).

This is not optimal, because it's not possible to add more devices at once.
I would expect that one can add any multiple of 3 (or 4) available.

Reproducer is done on arbiter stretch cluster, but the problem is general,
applicable to non arbiter deployments as well.

This is similar to BZ 1928319, but here we are discussing "Add Capacity"
instead of installation.

Version-Release number of selected component
============================================

OCP 4.7.0-0.nightly-2021-05-05-092347
LSO 4.7.0-202104250659.p0
OCS 4.7.0-377.ci (4.7.0-rc8)

How reproducible
================

100%

Steps to Reproduce
==================

1. Install OCP on vSphere, with 3 master and 6 worker nodes, with 2 local
   storage device per worker node (for LSO).

2. Install LSO and OCS operators.

3. Label nodes via `oc label node $NODE topology.kubernetes.io/zone=$ZONE`
   so that you have 3 worker nodes and 1 master in each data zone, and just
   single master in arbiter zone:

   ```
   $ oc get nodes -L topology.kubernetes.io/zone
   NAME              STATUS   ROLES    AGE   VERSION           ZONE
   compute-0         Ready    worker   8h    v1.20.0+c8905da   data-a
   compute-1         Ready    worker   8h    v1.20.0+c8905da   data-a
   compute-2         Ready    worker   8h    v1.20.0+c8905da   data-a
   compute-3         Ready    worker   8h    v1.20.0+c8905da   data-b
   compute-4         Ready    worker   8h    v1.20.0+c8905da   data-b
   compute-5         Ready    worker   8h    v1.20.0+c8905da   data-b
   control-plane-0   Ready    master   8h    v1.20.0+c8905da   data-a
   control-plane-1   Ready    master   8h    v1.20.0+c8905da   data-b
   control-plane-2   Ready    master   8h    v1.20.0+c8905da   foo-arbiter
   ```

4. Use "Create Storage Cluster" wizard in OCP Console to start setup of Storage
   Cluster in "Internal - Attached devices" mode. Make sure that the cluster
   will have just 4 OSDs (if BZ 1928319 is not fixed, you don't need to do
   anything, but if BZ 1928319 is fixed, you need to tweak local volume set
   accordingly).

5. Prepare local volume set for expand, so that there are 8 more local devices
   in OCS local volume set.

6. Go to OCP Console, locate StorageCluster/ocs-storagecluster and use
   "Add Capacity" feature to extend the cluster, selecting the LSO storage
   class.

Actual results
==============

Even though there are 8 more local devices in the LSO storage class, only 4
were consumed during "Add Capacity" operation.

Expected results
================

It's possible to add 8 OSDs at once (I don't expect this to be new default, but
that there is an option to do this if desired).