Bug 2032243

There is not a lot we can do here. Newer releases CentOS and RHEL are carrying patches which create the device as /dev/md127, so this is what ironic-python-agent will discover. All we could suggest is some kind of documentation to raise awareness of when the discovered root device should be changed manually to /dev/md0.

Its possible we could create a code fallback in IPA which attempts /dev/md127 when /dev/md0 doesn't exist. This will require some research.

*** Bug 2031420 has been marked as a duplicate of this bug. ***

And trying to make assumptions around MD device ordering is problematic because they are unstable and upstream the code is modeled to support two software RAID devices. Plus the numbering goes by initialization order and kernel.

In other news, root device hints shouldn't be needed with software raid, at least based upon looking at the upstream code.  It looks like they also shouldn't really be needed on the Train release as well, at least based on the sorting/size management algorithm. This of course assumes there are no special block devices which would appear before a MD device based on sorting by name. The issue is the documentation, https://review.opendev.org/c/openstack/ironic/+/821782 should hopefully kind of clear this up.

Please let me know if that sorts out the question here.

(In reply to Julia Kreger from comment #4)
> And trying to make assumptions around MD device ordering is problematic
> because they are unstable and upstream the code is modeled to support two
> software RAID devices. Plus the numbering goes by initialization order and
> kernel.
> 
> In other news, root device hints shouldn't be needed with software raid, at
> least based upon looking at the upstream code.  It looks like they also
> shouldn't really be needed on the Train release as well, at least based on
> the sorting/size management algorithm. This of course assumes there are no
> special block devices which would appear before a MD device based on sorting
> by name. The issue is the documentation,
> https://review.opendev.org/c/openstack/ironic/+/821782 should hopefully kind
> of clear this up.
> 
> Please let me know if that sorts out the question here.

No.

(In reply to Nilesh from comment #6)
> (In reply to Julia Kreger from comment #4)
[trim]

> > 
> > Please let me know if that sorts out the question here.
> 
> No.

Wait, are you introspecting after creating the software raid device? If you are, that is not correct. If it is being manually added, it is invalid.

What steps are precisely being taken to result in the root device hint being added?

* Deployed environment however, md device created with /dev/md127, 
  * Would like to understand the reason or any limitation, 
  * Is it hardcoded somewhere. 


* Import the node

```
openstack overcloud node import /home/stack/instackenv2.json
```

* Introspect the node.

```
openstack overcloud node introspect --provide 5a6342b6-4386-4633-b558-ba4900789879
```

* Set sw-raid agent on the node.

```
openstack baremetal node set --raid-interface agent  5a6342b6-4386-4633-b558-ba4900789879
```

* move the node into manage, for further configuration.

```
openstack baremetal node manage  5a6342b6-4386-4633-b558-ba4900789879
```

* Set the target configuration [sw raid]

```
openstack baremetal node set   --target-raid-config /home/stack/sw-raid/target.yaml  5a6342b6-4386-4633-b558-ba4900789879
```

* cat  /home/stack/sw-raid/target.yaml

```
{
   "logical_disks": [
     {
       "size_gb": "MAX",
       "raid_level": "1",
       "controller": "software",
       "is_root_volume": true,
       "disk_type": "hdd"
     }
   ]
}
```

* Clean the node before deployment, [During this step the nodes will turn on and will start cleaning o the disk]

```
openstack baremetal node clean  --clean-steps /home/stack/sw-raid/cleanstate.yaml 5a6342b6-4386-4633-b558-ba4900789879
```

* Move node from manage to available. [provide]

```
openstack baremetal node provide 5a6342b6-4386-4633-b558-ba4900789879
```

* Set the flavor as well,

```
openstack flavor set \
--property "capabilities:raid_level"="1"  \
baremetal
```




```
[root@overcloud-swnovacompute-0 ~]# mdadm --detail /dev/md127
/dev/md127:
           Version : 1.2
     Creation Time : Wed Feb  2 09:35:23 2022
        Raid Level : raid1
        Array Size : 62878720 (59.97 GiB 64.39 GB)
     Used Dev Size : 62878720 (59.97 GiB 64.39 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Thu Feb  3 17:03:32 2022
             State : clean
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : localhost.localdomain:0
              UUID : 078b1d94:543da54c:df879980:a0c81fa8
            Events : 257

    Number   Major   Minor   RaidDevice State
       0     252        1        0      active sync   /dev/vda1
       1     252       17        1      active sync   /dev/vdb1
[root@overcloud-swnovacompute-0 ~]#
```

(In reply to Nilesh from comment #9)
> * Deployed environment however, md device created with /dev/md127, 
>   * Would like to understand the reason or any limitation, 
>   * Is it hardcoded somewhere. 
> 

The python-tripleoclient tooling has an algorithm to *always* put a root device hint in for every node which is explicitly introspected. It uses introspection data to forcibly populate this value, and using it's algorithm it determines the smallest "viable" block device by default. This is because it is unaware of raid devices, and it really shouldn't *need* to be anyway. It is just trying to be helpful, and in this specific use case, it is not exactly helpful.

Because of the way it does this, this value is also always wrong for software raid uses because device names for software raid device names are not stable across the reboot, but can be easily navigated by either resetting the root device manually after executing the overcloud node configuration step, or by not passing the "--overwrite-root-device-hints" if explicitly set *before* the overcloud node configuration step.

We've merged a patch in upstream which will be available in OSP18 which explicitly looks for software raid partitions, and explicitly skips guessing the root device hint as a result. I do not intend to backport this patch at this time because this issue can be worked around easily.

> 
> * Import the node
> 
> ```
> openstack overcloud node import /home/stack/instackenv2.json
> ```
> 
> * Introspect the node.
> 
> ```
> openstack overcloud node introspect --provide
> 5a6342b6-4386-4633-b558-ba4900789879
> ```
> 
> * Set sw-raid agent on the node.
> 
> ```
> openstack baremetal node set --raid-interface agent 
> 5a6342b6-4386-4633-b558-ba4900789879
> ```
> 
> * move the node into manage, for further configuration.
> 
> ```
> openstack baremetal node manage  5a6342b6-4386-4633-b558-ba4900789879
> ```
> 
> * Set the target configuration [sw raid]
> 
> ```
> openstack baremetal node set   --target-raid-config
> /home/stack/sw-raid/target.yaml  5a6342b6-4386-4633-b558-ba4900789879
> ```
> 
> * cat  /home/stack/sw-raid/target.yaml
> 
> ```
> {
>    "logical_disks": [
>      {
>        "size_gb": "MAX",
>        "raid_level": "1",
>        "controller": "software",
>        "is_root_volume": true,
>        "disk_type": "hdd"
>      }
>    ]
> }
> ```
> 
> * Clean the node before deployment, [During this step the nodes will turn on
> and will start cleaning o the disk]
> 
> ```
> openstack baremetal node clean  --clean-steps
> /home/stack/sw-raid/cleanstate.yaml 5a6342b6-4386-4633-b558-ba4900789879
> ```
> 
> * Move node from manage to available. [provide]
> 
> ```
> openstack baremetal node provide 5a6342b6-4386-4633-b558-ba4900789879
> ```
> 
> * Set the flavor as well,
> 
> ```
> openstack flavor set \
> --property "capabilities:raid_level"="1"  \
> baremetal
> ```

For what it is worth, this is redundant and unrelated to setting nodes to deploy with software raid.

> 
> 


> 
> 
> ```
> [root@overcloud-swnovacompute-0 ~]# mdadm --detail /dev/md127
> /dev/md127:
>            Version : 1.2
>      Creation Time : Wed Feb  2 09:35:23 2022
>         Raid Level : raid1
>         Array Size : 62878720 (59.97 GiB 64.39 GB)
>      Used Dev Size : 62878720 (59.97 GiB 64.39 GB)
>       Raid Devices : 2
>      Total Devices : 2
>        Persistence : Superblock is persistent
> 
>        Update Time : Thu Feb  3 17:03:32 2022
>              State : clean
>     Active Devices : 2
>    Working Devices : 2
>     Failed Devices : 0
>      Spare Devices : 0
> 
> Consistency Policy : resync
> 
>               Name : localhost.localdomain:0
>               UUID : 078b1d94:543da54c:df879980:a0c81fa8
>             Events : 257
> 
>     Number   Major   Minor   RaidDevice State
>        0     252        1        0      active sync   /dev/vda1
>        1     252       17        1      active sync   /dev/vdb1
> [root@overcloud-swnovacompute-0 ~]#
> ```

Closing as upstream as change will be in python-tripleoclient which will short circuit in-appropriate root device hinting in this specific case. This change should make it into OSP18.