Bug 1889217

Description of problem:
The section at https://docs.openshift.com/container-platform/4.5/storage/persistent_storage/persistent-storage-local.html#local-tolerations_persistent-storage-local 
talks about only including tolerations in general to LocalVolume.

--------------------------------------------------------------------------------------------------------------
I find the documents fails to provide clarity to have local PVs in master-nodes in two main aspects as below,
--------------------------------------------------------------------------------------------------------------

Aspect-I:
---------

Inorder to have local PV in master node, the LocalVolume definition should explicity include the below toleration

- key: node-role.kubernetes.io/master
  operator: Exists

If it is not included the diskmaker, provisioner pods are not scheduled on the particular master and eventually the underlying PV is not located.

As a default OCP cluster's master nodes are mastersSchedulable=false, the user has to include this toleration inevitably.
But, the documentation fails to include any note on the needed toleration to be set in the LocalVolume for having the local PVs on master nodes.

--------------------------------------------------------------------------------------------------------------

Aspect-II:
----------

The section "2. Optional: Allow local storage creation on master and infrastructure nodes."
https://docs.openshift.com/container-platform/4.3/storage/persistent_storage/persistent-storage-local.html#local-storage-install_persistent-storage-local

The position of these instructions in the docs is mis-leading as it asks the user to apply the patch too early at a stage where the local storage operator is not even installed.

"oc get ds" at this point returns zero result and one cannot apply these patches.

And as I observed, 
oc ds resources are created only when LocalVolume is deployed and not when LSO operator is installed.
In general, applying the patches at this point helps the diskmaker, provisioner pods to be scheduled on master nodes and local PV is identified.

However, the patches donot have an effect when the LocalVolume already tolerates node-role.kubernetes.io/master - Exists.
I could see the local PVs on master nodes being displayed even before the patch is applied.

Version-Release number of selected component (if applicable):
oc version
Client Version: 4.6.0-0.nightly-s390x-2020-10-06-145952
Server Version: 4.6.0-0.nightly-s390x-2020-10-10-041058
Kubernetes Version: v1.19.0+d59ce34

How reproducible:
Add secondary disks to master nodes and proceed to install LSO, create local PV. 

Steps to Reproduce:
Please see above.

Actual results:
The documentation fails to provide clarity to have local PVs in master-nodes

Expected results:
1. The documentation should include clearer notes on the tolerations to be set to have local PVs in master nodes.
2. The section "2. Optional: Allow local storage creation on master and infrastructure nodes." has to be removed from the existing place and included later in the section towards "Using tolerations with Local Storage Operator Pods".

Master Log:

Node Log (of failed PODs):

PV Dump:

PVC Dump:

StorageClass Dump (if StorageClass used by PV/PVC):

Additional info:
The bug is identified as a part of https://bugzilla.redhat.com/show_bug.cgi?id=1887574