Bug 2079796

Description of problem:

My customer is having the following issue:

They are trying to get IPv4 hash-based multipath routing to work with nmstate but it looks like it's not possible yet. 
I can create a manual multipath route on the worker node with:

[core@worker-0 ~]# sudo ip route add 2.2.2.2/32 proto static scope global  nexthop via 10.123.0.56 dev eno2.100 weight 1  nexthop via 10.123.0.184 dev eno1.100 weight 1

#Route is set:
[core@worker-0 ~]# ip r
<snip>
2.2.2.2 proto static 
	nexthop via 10.123.0.56 dev eno2.100 weight 1 
	nexthop via 10.123.0.184 dev eno1.100 weight 1 


Now trying to do the same with nmstate:
---
apiVersion: nmstate.io/v1beta1
kind: NodeNetworkConfigurationPolicy
metadata:
  name: bond-vlan-worker-0
spec:
  nodeSelector:
    kubernetes.io/hostname: worker-0
  desiredState:
    interfaces:
    - name: eno1.100
      type: vlan
      state: up
      mtu: 1500
      vlan:
        base-iface: eno1
        id: 100
      ipv4:
        address:
        - ip: 10.123.0.185
          prefix-length: 31
        dhcp: false
        enabled: true
    - name: eno2.100
      type: vlan
      state: up
      mtu: 1500
      vlan:
        base-iface: eno2
        id: 100
      ipv4:
        address:
        - ip: 10.123.0.57
          prefix-length: 31
        dhcp: false
        enabled: true
    routes:
      config:
      - destination: 2.2.2.2/32
        metric: 451
        next-hop-address: 10.123.0.184
        next-hop-interface: eno1.100
        table-id: 254
      - destination: 2.2.2.2/32
        metric: 451
        next-hop-address: 10.123.0.56
        next-hop-interface: eno2.100
        table-id: 254
        
# But it fails to create a multipath route and now it's only routing to eno1.100:        
```
[core@worker-0 /]# ip r
<snip>
2.2.2.2 via 10.123.0.184 dev eno1.100 proto static metric 451 
2.2.2.2 via 10.123.0.56 dev eno2.100 proto static metric 451 
```

Am I'm doing something wrong, or is this not implemented? I see some references in the nmstate code but not sure if we're running the version that contains this fix: https://github.com/nmstate/nmstate/commit/0eb2c3378f7df6586b0cadb4df734c95e47761bf

The image the customer is using is:

image: registry.redhat.io/openshift4/ose-kubernetes-nmstate-handler-rhel8@sha256:2f410c46c75ee4c35027ab76b1d8378aa1edf8a9ceea6eef784cc7fbae81a4aa which does contain the above fix



Version-Release number of selected component (if applicable):

OCP 4.10


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info: