Document URL: https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html/openshift_virtualization/ Section Number and Name: Describe the issue: The OpenShift Virtualization storage feature matrix is not clear about what is supported, i.e. what to expect when opening a support case. From a conversation with Fabian Deutsch, we can import a virtual machine to OpenShift Virtualization using any supported storage provisioner with the following limitations: - If the provision is OCS, i.e. Ceph RBD, we only support block mode. Will be fixed in CNV 2.6.0. - Hostpath provisioner is only supported for import from VMware. Regarding the last two lines of the table, we support "Other multi-node writable storage" (RWX) and "Other single-node writable storage" (RWO). So, according to OCP 4.6 documentation about persistent storage, the supported persistent volume plugins listed in "Table 2.2. Supported access modes for PVs" all support the Filesystem volume mode. And the persistent volume plugins that support Block volume mode are listed in "Table 2.4. Block volume support". From these two tables, I should have the fully supported and tech preview access and volume modes that we can use when creating DataVolumes. Fabian told me that "For regular VMs on OpenShift Virtualization, a storage backend is needed that supports RWX access mode (shared storage) and block mode (alternative file-system mode). The recommended option is OCS." My understanding of this is that the RWX is required if we want to live migrate virtual machines. But RWO is enough for machines that won't migrate and there's even an eviction strategy to be able to upgrade CNV. So, it sounds more like something to consider when creating a VM than a requirement. For example, OpenStack Cinder is a supported storage backend for OpenShift Container Platform and is limited to RWO access mode. I would expect that it's also supported by OpenShift Virtualization, with features limitations for the VM. [1] https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html/openshift_virtualization/virtual-machines#importing-virtual-machines Suggestions for improvement: Additional information:
Here is a possible representation of the matrix for storage plug-ins that support dynamic provisioning (not fully valid markdown): | Storage Plug-in | Volume Mode | Dyn | | | Filesystem | Block | | | | RWO | ROX | RWX | RWO | ROX | RWX | | | :---------------------------------- | :-: | :-: | :-: | :-: | :-: | :-: | | | AWS EBS | x | - | - | x | - | - | x | | Azure Disk | x | - | - | x | - | - | x | | Azure File | x | x | x | - | - | - | x | | Cinder | x | - | - | x | - | - | x | | Fiber Channel | x | - | - | x | - | - | - | | GCE Persistent Disk | x | - | - | x | - | - | x | | HostPath | x | - | - | - | - | - | x | | iSCSI | x | - | - | x | - | - | - | | Local volume | x | - | - | x | - | - | - | | NFS | x | x | x | - | - | - | - | | OpenStack Manila | - | - | x | - | - | - | x | | Red Hat OpenShift Container Storage | x | - | x | x | - | - | x | | VMware vSphere | x | - | - | x | - | - | x | There is still some subtlety with the fact that manual provisioning of block PVs is Tech Preview for the following Cinder and Fiber Channel storage plug-ins.
As this enhancement has more to do with VM Import than the Features for Storage table, probably best for this bug to be assigned to Avital. Both Avital's and my content is single-sourced from one module, so it can be confusing :-) I'll discuss further with @ctomasko.
@apinnick Please review this BZ to determine whether this should be documented under migration.
The storage matrices are not explicit enough with regards to the supported features in CNV. They basically say that everything that OCP supports is supported, with some slight limitations. However, some features of CNV are not supported by all storage backends. For example, Live Migration requires RWX access mode, which limits it to Azure File, NFS, OpenStack Manila and OCS. If you don't deploy OpenShift on Azure or OpenStack, you're down to NFS and OCS. And given that NFS doesn't have a dynamic provisioner, if you really want the flexibility of OpenShift, your only choice is OCS. And that's not clear from reading the matrics.
Catharine and Bob, After discussion with Fabien, it is clear that this bug is for CNV storage, not VM import. > From a conversation with Fabian Deutsch, we can import a virtual machine to > OpenShift Virtualization using any supported storage provisioner with the > following limitations: > > - If the provision is OCS, i.e. Ceph RBD, we only support block mode. Will > be fixed in CNV 2.6.0. > - Hostpath provisioner is only supported for import from VMware. These issues have already been documented in VM import. (HPP was only for CNV 2.4.) The rest of Fabien's comments below apply to CNV in general, so I am re-assigning this BZ to Bob. > Regarding the last two lines of the table, we support "Other multi-node > writable storage" (RWX) and "Other single-node writable storage" (RWO). > So, according to OCP 4.6 documentation about persistent storage, the > supported persistent volume plugins listed in "Table 2.2. Supported access > modes for PVs" all support the Filesystem volume mode. And the persistent > volume plugins that support Block volume mode are listed in "Table 2.4. > Block volume support". > > From these two tables, I should have the fully supported and tech preview > access and volume modes that we can use when creating DataVolumes. > > Fabian told me that "For regular VMs on OpenShift Virtualization, a storage > backend is needed that supports RWX access mode (shared storage) and block > mode (alternative file-system mode). The recommended option is OCS." > > My understanding of this is that the RWX is required if we want to live > migrate virtual machines. But RWO is enough for machines that won't migrate > and there's even an eviction strategy to be able to upgrade CNV. So, it > sounds more like something to consider when creating a VM than a requirement. > > For example, OpenStack Cinder is a supported storage backend for OpenShift > Container Platform and is limited to RWO access mode. I would expect that > it's also supported by OpenShift Virtualization, with features limitations > for the VM. > > > [1] > https://access.redhat.com/documentation/en-us/openshift_container_platform/4. > 6/html/openshift_virtualization/virtual-machines#importing-virtual-machines
Since we have an epic (CNV-8626) in place in Jira and a plan from Adam, I am marking Closed/Not a Bug to clear this from the queue.