RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2136820 - [RFE] Allow policy based routing based on marking in nmstate
Summary: [RFE] Allow policy based routing based on marking in nmstate
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 9
Classification: Red Hat
Component: nmstate
Version: 9.1
Hardware: Unspecified
OS: Unspecified
unspecified
urgent
Target Milestone: rc
: ---
Assignee: Fernando F. Mancera
QA Contact: Mingyu Shi
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2022-10-21 13:30 UTC by sfaye
Modified: 2023-12-09 11:10 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2023-05-09 07:31:50 UTC
Type: Bug
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
BaselinePOC (24.02 KB, image/png)
2022-10-21 13:30 UTC, sfaye 		no flags Details
View All


Links
System ID Private Priority Status Summary Last Updated
Github nmstate nmstate pull 2072 0 None open Add support to fwmark and fwmask in routing policy 2022-10-21 14:44:18 UTC
Red Hat Issue Tracker NMT-121 0 None None None 2023-01-22 15:28:42 UTC
Red Hat Issue Tracker RHELPLAN-137286 0 None None None 2022-10-21 13:34:20 UTC
Red Hat Issue Tracker RHELPLAN-137292 0 None None None 2022-10-21 13:45:25 UTC
Red Hat Product Errata RHBA-2023:2190 0 None None None 2023-05-09 07:32:04 UTC

Description sfaye 2022-10-21 13:30:32 UTC 
Created attachment 1919405 [details]
BaselinePOC

Context : 

The baseline is the scenario described in the attachement (BaselinePOC) : 
The clients are not on the same subnet of the node
Reaches the node from a vlan interface different from br-ex
The service’s return traffic goes via br-ex (the default gw) and doesn’t reach the client. 

Requirements : 

As a system administrator, I would like to segregate traffic between VRFs and avoid static routes on the OCP nodes while handling the overlapping scenarios.




The overlapping scenarios are :

Client1  / Client2 may have overlapping IPs
A SVC2 may have overlapping IP with Client1




To overcome this problem, the attachement (VRFSolution) can be a solution (Wrapping the interface inside a VRF)

In order to drive the return traffic via the selected interface using policy based routing based on marking. 


For that, we need to direct the traffic coming with the service’s ClusterIP via the veth leg on the default VRF. 

Nmstate should be able to : 

Change the priority of ip rules : We need to have the VRF ip rule have more priority over the local one
Policy based routing based on marking.
Comment 3 Fernando F. Mancera 2022-10-24 14:49:39 UTC 
Merged upstream!
Comment 6 Fernando F. Mancera 2023-02-20 09:49:25 UTC 
The description of the bug is confusing.

This bug is asking for routing policy support based on marking which means that the user should be to specify fwmark and fwmask for each routing policy.

This is an examples desired state from Nmstate:

```
route-rules:
  config:
    - ip-to: 192.0.2.0/24
      ip-from: 198.51.100.0/24
      priority: 100
      route-table: 254
      fwmark: 0x30
      fwmask: 0x10
```

Thanks!
Comment 7 Mingyu Shi 2023-02-27 03:43:09 UTC 
Verified with:
nmstate-2.2.7-1.el9.x86_64
nispor-1.2.10-1.el9.x86_64
NetworkManager-1.42.0-1.el9.x86_64

[11:41:27@dell-per740-68 ~]0# nmstatectl set newrules.yaml 
Using 'set' is deprecated, use 'apply' instead.
[2023-02-27T03:41:33Z INFO  nmstate::query_apply::net_state] Created checkpoint /org/freedesktop/NetworkManager/Checkpoint/1
[2023-02-27T03:41:33Z INFO  nmstate::ifaces::inter_ifaces] Ignoring interface eno3 type ethernet
[2023-02-27T03:41:33Z INFO  nmstate::ifaces::inter_ifaces] Ignoring interface ens3f0np0 type ethernet
[2023-02-27T03:41:33Z INFO  nmstate::ifaces::inter_ifaces] Ignoring interface eno4 type ethernet
[2023-02-27T03:41:33Z INFO  nmstate::ifaces::inter_ifaces] Ignoring interface enp59s0 type ethernet
[2023-02-27T03:41:33Z INFO  nmstate::ifaces::inter_ifaces] Ignoring interface ens3f1np1 type ethernet
[2023-02-27T03:41:33Z INFO  nmstate::ifaces::inter_ifaces] Ignoring interface eno2 type ethernet
[2023-02-27T03:41:33Z INFO  nmstate::nm::query_apply::profile] Creating connection UUID Some("1c5543c4-e80e-46b1-8101-cd37e8562003"), ID Some("veth0"), type Some("802-3-ethernet") name Some("veth0")
[2023-02-27T03:41:33Z INFO  nmstate::nm::query_apply::profile] Activating connection 1c5543c4-e80e-46b1-8101-cd37e8562003: veth0/802-3-ethernet
[2023-02-27T03:41:33Z INFO  nmstate::query_apply::net_state] Destroyed checkpoint /org/freedesktop/NetworkManager/Checkpoint/1
dns-resolver: {}
route-rules:
  config:
  - ip-from: 198.51.100.0/24
    ip-to: 192.0.2.0/24
    priority: 100
    route-table: 254
    fwmark: '0x30'
    fwmask: '0x10'
    iif: veth0
routes: {}
interfaces: []
ovs-db: {}

[11:41:33@dell-per740-68 ~]0# ip rule
0:      from all lookup local
100:    from 198.51.100.0/24 to 192.0.2.0/24 fwmark 0x30/0x10 iif veth0 lookup main proto static
32766:  from all lookup main
32767:  from all lookup default
Comment 9 errata-xmlrpc 2023-05-09 07:31:50 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (nmstate bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:2190
Note You need to log in before you can comment on or make changes to this bug.