Bug 2091467

The primary disk refers to the boot disk, which includes the boot partition and normally also the root partition (unless the user is using Ignition to move the rootfs onto another disk, which is unusual). So the instructions available in the documentation does cover both the boot and root filesystems.

It is also possible to enable multipath for secondary disks, though we do not yet have official documentation for this. There is an RHBZ somewhere requesting that documentation which I can't find right now.

Thanks, Jonathan

I understand that the documentation setup covers both the boot and root areas. Thank you very much.

I am having trouble with multipath conversion in a customer environment. Currently there is no output in multipath -ll and I assume that the devices via multipath are not visible.

Hi Ryoji,

I had a look at the customer case and noticed references to `/dev/mapper/coreos-luks-root-nocrypt`. This indicates that they're working with hosts installed from RHCOS 4.6 (or earlier) images. Multipathing is supported starting from RHCOS 4.7, where the machines have to be installed from RHCOS 4.7 images. If this is a cluster installation, they need to make sure they're using the right RHCOS 4.7 images. If this is a cluster born before 4.7, multipath cannot be enabled on existing nodes, but it'd be possible to enable it on new nodes if they update the boot installation media used to 4.7 (but then, they'd have a mix of multipathed and non-multipathed nodes, which isn't great).

Thanks, Jonathan

The customer has upgraded OpenShift to 4.7, and the RHCOS version may have been out of date when the cluster was installed.
If that is the case, does that mean that the node must be built anew?

Correct. We do not support turning on multipath day-2 on nodes that have been upgraded from 4.6 or earlier. They can enable it on new nodes (that have been installed with 4.7 installation media) by manually using `rpm-ostree kargs --append rd.multipath=default --append root=/dev/disk/by-label/dm-mpath-root` directly on the machine (see the steps in https://docs.openshift.com/container-platform/4.7/nodes/nodes/nodes-nodes-working.html#nodes-nodes-working-evacuating_nodes-nodes-working to do this safely).

Thanks, Jonathan

> We do not support turning on multipath day-2 on nodes that have been upgraded from 4.6 or earlier. 

I think this should be clarified in the documentation, so I have reported a doc bug.

Bug 2094726 - [DOC]It should be clarified that multipath is not supported in environments upgraded from 4.6 or earlier. 
https://bugzilla.redhat.com/show_bug.cgi?id=2094726

Jonathan

The following document says that when adding a node, we must use the same ISO image when installing a cluster. The latest documents say the same thing too. Is it supported to use a different iso from the installation?

Creating Red Hat Enterprise Linux CoreOS (RHCOS) machines
https://docs.openshift.com/container-platform/4.10/machine_management/user_infra/adding-bare-metal-compute-user-infra.html#creating-rhcos-machines-bare-metal

```	
Note
You must use the same ISO image that you used to install a cluster to deploy all new nodes in a cluster. It is recommended to use the same Ignition config file. The nodes automatically upgrade themselves on the first boot before running the workloads. You can add the nodes before or after the upgrade.
```

In general, it's safer to keep reusing the same installation media. However, in some circumstances it is necessary and supported to use a newer ISO matching the target OCP release. For example, for new hardware enablement or to workaround Ignition versioning issues or situations like we have here.