Bug 1996731

Summary: [RFE] Extend rhel-system-roles.network feature set to support routing rules
Product: Red Hat Enterprise Linux 8 Reporter: Roman Bobek <rbobek>
Component: rhel-system-rolesAssignee: Rich Megginson <rmeggins>
Status: CLOSED ERRATA QA Contact: Jon Trossbach <jtrossba>
Severity: medium Docs Contact: Mayur Patil <maypatil>
Priority: unspecified    
Version: 8.7CC: djez, ferferna, fge, gfialova, jtrossba, maypatil, nhosoi, rmeggins, spetrosi, thaller, till, wenliang, zhguan
Target Milestone: rcKeywords: FutureFeature, Triaged
Target Release: 8.7   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: role:network
Fixed In Version: rhel-system-roles-1.18.0-1.el8 Doc Type: Enhancement
Doc Text:
.The `network` RHEL System Role now configures network settings for routing rules Previously, you could route the packet based on the destination address field in the packet, but you could not define the source routing and other policy routing rules. With this enhancement, `network` RHEL System Role supports routing rules so that the users have control over the packet transmission or route selection.
Story Points: ---
Clone Of:
: 2079622 (view as bug list) Environment:
Last Closed: 2022-11-08 09:40:46 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 2079622    

Description Roman Bobek 2021-08-23 14:29:56 UTC
Description of problem:

rhel-system-roles.network is not able to configure the following network settings:

* configure more routing tables
* configure routing rules
* some bonding opts are missing, such as: acp_rate=, updelay=, xmit_hash_policy=, etc...
* configure secondary interface such as eth0:1234. Config file does not contain :1234.


Actual results:
Unable to use rhel-system-roles.network to configure above mentioned features.

Expected results:
rhel-system-roles.network is able to configure above mentioned features.

Comment 6 Wen Liang 2021-12-09 15:05:15 UTC
(In reply to Roman Bobek from comment #0)
> Description of problem:
> 
> rhel-system-roles.network is not able to configure the following network
> settings:
> 
> * configure more routing tables
> * configure routing rules
> * some bonding opts are missing, such as: acp_rate=, updelay=,
> xmit_hash_policy=, etc...
> * configure secondary interface such as eth0:1234. Config file does not
> contain :1234.
> 

For the IP-Aliasing feature request "configure secondary interface such as eth0:1234", it is already deprecated by kernel, and NetworkManager allows multiple IP addresses assigned to single interface without creating new sub-interface. For additional infomation around IP-Aliasing, you can take a look at here, https://www.kernel.org/doc/html/latest/networking/alias.html.

Comment 7 Wen Liang 2021-12-12 14:33:16 UTC
Hi rbobek,

We are planning to implement the routing table support in the network role in rhel-8.6 and postpone other features in the rhel-8.7. If time permits, we will also implement the source routing (part of policy routing) in rhel-8.6. 

I created a separate bug for tracking the progress of routing table support, https://bugzilla.redhat.com/show_bug.cgi?id=2031521. You may take a look there and see if the design suffices to fulfill the needs from the customer. Your feedback is precious to us.

Best,
Wen

Comment 8 Wen Liang 2022-02-14 22:08:14 UTC
The bonding options support has been implemented, which can be tracked in https://bugzilla.redhat.com/show_bug.cgi?id=2008931.

Comment 10 Wen Liang 2022-04-04 11:35:58 UTC
The following is the User Story and the Acceptance Criteria for supporting the routing rules in the network role. 


General Requirement (User Story) 
As a user, I can specify the policy routing rule which forwards the packet based on the other criteria except for the destination address field in the packet, so that I can have flexible control over the packet transmission or route selection.


Acceptance Criteria:
Given a Linux system with NetworkManager, when a user applies the following playbook
```
        network_connections:
          - name: "{{ interface }}"
            interface_name: "{{ interface }}"
            state: up
            type: ethernet
            autoconnect: yes
            ip:
              dhcp4: no
              address:
                - 198.51.100.3/26
              route:
                - network: 198.51.100.64
                  prefix: 26
                  gateway: 198.51.100.6
                  metric: 4
                  table: 30200
                - network: 198.51.100.128
                  prefix: 26
                  gateway: 198.51.100.1
                  metric: 2
                  table: 30400
              routing_rule:
                - priority: 30200
                  from: 198.51.100.58/26
                  table: 30200
                - priority: 30201
                  family: ipv4
                  fwmark: 1
                  fwmask: 1
                  table: 30200
                - priority: 30202
                  family: ipv4
                  sport: 128 - 256
                  table: 30200
                - priority: 30203
                  family: ipv4
                  tos: 8
                  table: 30200
                - priority: 30400
                  to: 198.51.100.128/26
                  table: 30400
                - priority: 30401
                  family: ipv4
                  iif: iiftest
                  table: 30400
                - priority: 30402
                  family: ipv4
                  oif: oiftest
                  table: 30400
```



Then  `ip rule list table 30200` will show the following routing rules:
30200:  from 198.51.100.58/26 lookup 30200
30201:  from all fwmark 0x1/0x1 lookup 30200
30202:  from all sport 128-256 lookup 30200
30203:  from all tos 0x08 lookup 30200

Then  `ip rule list table 30400` will show the following routing rules:
30400:  from all to 198.51.100.128/26 lookup 30400
30401:  from all iif iiftest [detached] lookup 30400
30402:  from all oif oiftest [detached] lookup 30400

Comment 11 Till Maas 2022-04-04 19:03:46 UTC
Regarding the acceptance criteria, it does not seem to me to be a good idea to use all the abbreviations since they make the settings harder to understand compared to using identifiers such as source_port, firewall_mark (or just mark, and maybe using the slash-notation for mark/mask), in_interface/out_interface. Also, It only shows family: ipv4 - will there be no ipv6 support?

Comment 12 Wen Liang 2022-04-04 19:43:21 UTC
(In reply to Till Maas from comment #11)
> Regarding the acceptance criteria, it does not seem to me to be a good idea
> to use all the abbreviations since they make the settings harder to
> understand compared to using identifiers such as source_port, firewall_mark
> (or just mark, and maybe using the slash-notation for mark/mask),
> in_interface/out_interface. Also, It only shows family: ipv4 - will there be
> no ipv6 support?

`family: ipv6` will also be supported, I will add the acceptance criteria for the family ipv6.

Comment 13 Thomas Haller 2022-04-04 20:17:20 UTC
(In reply to Till Maas from comment #11)
> Regarding the acceptance criteria, it does not seem to me to be a good idea
> to use all the abbreviations since they make the settings harder to
> understand compared to using identifiers such as source_port, firewall_mark
> (or just mark, and maybe using the slash-notation for mark/mask),
> in_interface/out_interface. Also, It only shows family: ipv4 - will there be
> no ipv6 support?

Routing-rules (and policy routing) are a rather special feature, few people will use it
or care about it. But those who do, will be necessarily familiar with `iproute2`'s `ip rule add`
command and the output of `ip rule`. These names come from there, so we should not add
different names than what we are familiar with.

Comment 14 Wen Liang 2022-04-06 00:44:49 UTC
(In reply to Till Maas from comment #11)
> Regarding the acceptance criteria, it does not seem to me to be a good idea
> to use all the abbreviations since they make the settings harder to
> understand compared to using identifiers such as source_port, firewall_mark
> (or just mark, and maybe using the slash-notation for mark/mask),
> in_interface/out_interface. Also, It only shows family: ipv4 - will there be
> no ipv6 support?

The acceptance criteria for IPv6:

Acceptance Criteria:
Given a Linux system with NetworkManager, when a user applies the following playbook
```
        network_connections:
          - name: "{{ interface }}"
            interface_name: "{{ interface }}"
            state: up
            type: ethernet
            autoconnect: yes
            ip:
              dhcp4: no
              address:
               - 2001:db8::2/32
              route:
                - network: 2001:db8::4
                  prefix: 32
                  gateway: 2001:db8::1
                  metric: 2
                  table: 30600   
              routing_rule:
                - priority: 30600
                  to: 2001:db8::4/32
                  table: 30600                  
                - priority: 30601
                  family: ipv6
                  dport: 128 - 256
                  invert: True
                  table: 30600
```


Then  `ip -6 rule list table 30600` will show the following routing rules:
30600:  from all to 2001:db8::4/32 lookup 30600
30601:  not from all dport 128-256 lookup 30600

Comment 15 Till Maas 2022-04-06 14:05:41 UTC
(In reply to Thomas Haller from comment #13)

> or care about it. But those who do, will be necessarily familiar with
> `iproute2`'s `ip rule add`
> command and the output of `ip rule`. These names come from there, so we
> should not add
> different names than what we are familiar with.

There are always new people coming, so they might not know ip rule, yet. And everyone who knows iif will also understand what incoming_interface means. Nevertheless, I can also accept to use the abbreviations.

Comment 16 Fernando F. Mancera 2022-04-21 10:21:22 UTC
Upstream PR merged. Thanks!

Comment 21 Jon Trossbach 2022-05-06 04:14:15 UTC
All tests were good on 8.7 for this MR.

# export ANSIBLE_VER=2.9
# export IMAGE=RHEL_8_7_TESTING
# export SYSTEM_ROLES_ONLY_TESTS=network/tests_routing_rules_nm.yml
# cd /mnt/tests/CoreOS/rhel-system-roles/Sanity/Upstream-testsuite/

# make run

...

::   Assertions: 5 good, 0 bad
::   RESULT: PASS (Test)

Comment 22 Jon Trossbach 2022-05-06 13:59:26 UTC
Additional system information for clarification:

# make run
...
rhel-system-roles-1.17.0-0.1.el8.noarch
:: [ 09:49:08 ] :: [   PASS   ] :: Checking for the presence of rhel-system-roles rpm 
:: [ 09:49:09 ] :: [   LOG    ] :: Package versions:
:: [ 09:49:09 ] :: [   LOG    ] ::   rhel-system-roles-1.17.0-0.1.el8.noarch
...



[root@netqe18 ~]# ls -la /etc/sysconfig/network-scripts/
total 52
drwxr-xr-x. 2 root root  236 May  5 22:49 .
drwxr-xr-x. 5 root root 4096 May  5 23:11 ..
-rw-r--r--. 1 root root  244 May  5 22:49 ifcfg-eno1
-rw-r--r--. 1 root root  243 May  5 22:49 ifcfg-eno2
-rw-r--r--. 1 root root  243 May  5 22:49 ifcfg-eno3
-rw-r--r--. 1 root root  243 May  5 22:49 ifcfg-eno4
-rw-r--r--. 1 root root  245 May  5 22:49 ifcfg-eno49
-rw-r--r--. 1 root root  245 May  5 22:49 ifcfg-eno50
-rw-r--r--. 1 root root  247 May  5 22:49 ifcfg-ens1f0
-rw-r--r--. 1 root root  247 May  5 22:49 ifcfg-ens1f1
-rw-r--r--. 1 root root  247 May  5 22:49 ifcfg-ens3f0
-rw-r--r--. 1 root root  247 May  5 22:49 ifcfg-ens3f1
-rw-r--r--. 1 root root  247 May  5 22:49 ifcfg-ens6f0
-rw-r--r--. 1 root root  247 May  5 22:49 ifcfg-ens6f1

Still pending are the additional qcow2 results for this 8.7 host image; those results should be in later today:
RHEL 6.10 
RHEL 7.9
RHEL 8.5
RHEL 8.6
RHEL 8.7
RHEL 9.0
RHEL 9.1

Comment 23 Jon Trossbach 2022-05-06 17:48:45 UTC
All qcow2 images listed above have now been verified on an 8.7 host. Please let me know when this has been merged.

Comment 24 Jon Trossbach 2022-05-06 17:53:47 UTC
The verification playbook ran PASSED with the following for each qcow2 image:

:: [ 13:46:09 ] :: [   PASS   ] :: Test network/tests_routing_rules_nm.yml (roles/rhel-system-roles.network/tests/tests_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_9_1_TESTING (Expected 0, got 0)

Comment 25 Fedora Update System 2022-05-11 01:17:43 UTC
FEDORA-2022-2da3a47337 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 26 Fedora Update System 2022-05-11 01:23:52 UTC
FEDORA-2022-d6e6c7c37c has been pushed to the Fedora 35 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 27 Fedora Update System 2022-05-11 01:34:10 UTC
FEDORA-2022-671a5d63e6 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

Comment 30 Jon Trossbach 2022-06-06 19:58:05 UTC
It looks like all VMs have healthy test results. Please review:

root@netqe16 Upstream-testsuite]# uname -r
4.18.0-398.el8.x86_64
[root@netqe16 Upstream-testsuite]# rpm -q rhel-system-roles
rhel-system-roles-1.18.0-1.el8.noarch

From http://download.eng.bos.redhat.com/brewroot/vol/rhel-9/packages/rhel-system-roles/1.18.0/1.el9/noarch/rhe\
l-system-roles-1.18.0-1.el9.noarch.rpm

:: [ 15:32:07 ] :: [   PASS   ] :: Test network (/tmp/tmp.bJx9BSOGca/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_6_10_GA (Expected 0, got 0)
:: [ 15:36:28 ] :: [   PASS   ] :: Test network (/tmp/tmp.nvtshNIkTy/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_7_9_GA (Expected 0, got 0)
:: [ 15:41:51 ] :: [   PASS   ] :: Test network (/tmp/tmp.HtPquWKtIB/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_8_5_0_GA (Expected 0, got 0)
:: [ 15:47:09 ] :: [   PASS   ] :: Test network (/tmp/tmp.I7sZDx82a3/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_8_6_TESTING (Expected 0, got 0)
:: [ 15:49:03 ] :: [   PASS   ] :: Test network (/tmp/tmp.8c8x4ZsCvB/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_8_7_TESTING (Expected 0, got 0)
:: [ 15:50:51 ] :: [   PASS   ] :: Test network (/tmp/tmp.WeG82sp8JO/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_9_0_TESTING (Expected 0, got 0)
:: [ 15:52:40 ] :: [   PASS   ] :: Test network (/tmp/tmp.ixjeuF5ns8/roles/rhel-system-roles.network/tests/tes\
ts_change_indication_on_repeat_run.yml) with ANSIBLE-2.9 against RHEL_9_1_TESTING (Expected 0, got 0)

Will move to VERIFIED.

Comment 31 Jon Trossbach 2022-06-06 20:01:06 UTC
Moved back to on QA test results are flawed.

Comment 32 Jon Trossbach 2022-06-06 20:30:18 UTC
Please ignore previous results they were not reported correctly.

It looks like all VMs have healthy test results. Please review the following:

[root@netqe16 Upstream-testsuite]# uname -r
4.18.0-398.el8.x86_64
[root@netqe16 Upstream-testsuite]# rpm -q rhel-system-roles
rhel-system-roles-1.18.0-1.el8.noarch

From http://download.eng.bos.redhat.com/brewroot/vol/rhel-8/packages/rhel-system-roles/1.18.0/1.el8/noarch/rhe\
l-system-roles-1.18.0-1.el8.noarch.rpm

:: [ 16:03:43 ] :: [   PASS   ] :: Test network (/tmp/tmp.k4frKkIlv8/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_9_1_TESTING (Expected 0, got 0)
:: [ 16:05:30 ] :: [   PASS   ] :: Test network (/tmp/tmp.eYJb7Q8PnK/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_9_0_TESTING (Expected 0, got 0)
:: [ 16:07:36 ] :: [   PASS   ] :: Test network (/tmp/tmp.5qA1G5V4Tk/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_8_7_TESTING (Expected 0, got 0)
:: [ 16:25:45 ] :: [   PASS   ] :: Test network (/tmp/tmp.ajFpTYwJg1/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_8_6_TESTING (Expected 0, got 0)
:: [ 16:16:46 ] :: [   PASS   ] :: Test network (/tmp/tmp.ib35Etv83R/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_8_5_0_GA (Expected 0, got 0)
:: [ 16:22:13 ] :: [   PASS   ] :: Test network (/tmp/tmp.L62BGdBDsQ/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_7_9_GA (Expected 0, got 0)
:: [ 16:23:40 ] :: [   PASS   ] :: Test network (/tmp/tmp.VBLVQPqpJf/roles/rhel-system-roles.network/tests/tes\
ts_routing_rules_nm.yml) with ANSIBLE-2.9 against RHEL_6_10_GA (Expected 0, got 0)

It will now be put back to VERIFIED.

Comment 40 errata-xmlrpc 2022-11-08 09:40:46 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (rhel-system-roles bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:7568