also issue #1384799 does not seem to have effect, as it still gives me the message "Cannot modify a connection that has an associated 'rule-' or 'rule6-' file" when I attempt to fix ipv4.route-table with nmcli after boot.

It's a missing feature.

Traditionally, NetworkManager did not support specifying a routing-table at all. For that, there is a workaround in form of a dispatcher script [1] [2]. The dispatcher script becomes active, when a rule- / rule6- file exists, and it calls ifup-routes command accordingly.

That is, the dispatcher script calls ifup-routes, but that cannot only handle rules, it also handles route files.

For that reason, when you configure a rule- file, it is assumed you are using initscripts to manage the rules and the routes. For that reason, the presence of a rule-file prevents NetworkManager from configuring any static-routes (ipv4.routes) or a routing table (ipv4.route-table).

This limitation was present for a long time, it was slightly relaxed with bug 1384799, but it is still essentially there.

Yes, this results in a catch22, because NetworkManager currently does not support configuring routing rules, so users are required to configure them themselvs. But users cannot use initscripts for that, because than NetworkManager does not handle static routes or ipv4.route-table.

The long term solution is to natively support routing rules in NetworkManager.
The restriction that we see exists, is so that later when we introduce support for this, we won't change existing behaviour.

The short term workarounds are:

 - don't rely on NetworkManager's ipv4.route-table and do it all with initscripts (writing rule- and route- files).
 - create the rules any other way, for example with a dispatcher scripts. If you look at /etc/sysconfig/network-scripts/ifup-routes, it's no magic.



[1] NetworkManager-dispatcher-routing-rules.rpm package
[2] /etc/NetworkManager/dispatcher.d/10-ifcfg-rh-routes.sh

> "ipv4.route-table" is reset to value 0 after reboot, if rule- file is present

btw, what the subject describes, is correct but misleading.

What happens is:

1) first, user creates profile with ipv4.route-table=42
2) then, user creates rule-file. Unless user issues `nmcli connection load $FILE` or `nmcli connection reload`, creating files has no immediate effects to NetworkManager.
3) after reboot, the profile is interpreted differently, with ipv4.route-table=0 (due to the now-present rule-file).



I think, the way forward here is to add native support to rules to NetworkManager. Which is however a large effort. If we agree, I would re-purpose this bug as feature request for supporting routing rules.

(In reply to Thomas Haller from comment #4)
> > "ipv4.route-table" is reset to value 0 after reboot, if rule- file is present
> 
> btw, what the subject describes, is correct but misleading.
> 
> What happens is:
> 
> 1) first, user creates profile with ipv4.route-table=42
> 2) then, user creates rule-file. Unless user issues `nmcli connection load
> $FILE` or `nmcli connection reload`, creating files has no immediate effects
> to NetworkManager.

agree but: in my described scenario, issuing `nmcli connection up` with the rule file present _does_ setup rules correctly (review the `ip ru` and `ip ro` output right after `nmcli c up`)!
it breaks only after reboot - but running the commands on a live system it works immediately.

> 3) after reboot, the profile is interpreted differently, with
> ipv4.route-table=0 (due to the now-present rule-file).
> 
> 
> 
> I think, the way forward here is to add native support to rules to
> NetworkManager. Which is however a large effort. If we agree, I would
> re-purpose this bug as feature request for supporting routing rules.

*full* ip rule + rt_tables support for n-m would be awesome (so no more n-m dispatcher helper for rule- files and no more manual editing of rt_tables - if I understand correctly when we say "native" support).

but until then, I'd be happy just to have the running config work after reboot.

RFE for adding native support for routing rules to NetworkManager: bug 1652653

> agree but: in my described scenario, issuing `nmcli connection up` with the rule file present _does_ setup rules correctly (review the `ip ru` and `ip ro` output right after `nmcli c up`)!
it breaks only after reboot - but running the commands on a live system it works immediately.

If I understand correctly, that is because as long as you don't reload the file, NetworkManager is not aware of the rule file.
On the other hand, the dispatcher scripts is aware of the rule file. So, in that case NM configures routes while also the dispatcher script configures them.

After reboot (or `nmcli con reload`) NetworkManager sees the rule file. This causes the profile in NetworkManager to have no routes/rules (because those are assume to be handled by the dispatcher scripts). Then it works as expected.

The problem is that the file was modified on disk without telling NetworkManager.

I don't see anything unexpected here -- aside that of course this legacy behavior with the dispatcher script is not stellar. But it's not clear how to fix it without moving away from the dispatcher script completely. And that is what happened...


> I'd like to avoid editing files and only use nmcli for everything! Or at least make it work partially by not resetting the ipv4.route-table at boot and live in peace with rule- files!

Yes, this is now possible as NetworkManager gained full support for routing rules. Note that it will not store rules in the "rule-*" file, because those are reserved for the legacy mode of operation. So, either use nmcli to edit rules ([1]) or use the new style format (if you wish to edit files).


[1] e.g. nmcli connection modify "$PROFILE" +ipv4.routing-rules "priority 10 from 192.168.6.24/32 table 66"

(the syntax for rules follows iproute2's syntax).




I will close this bug as duplicate of bug 1652653. Usually, I would move the bug to ON-QA for verification, but I don't think there are any special parts that aren't yet covered by bug 1652653. Please open a new bug if something is not working as expected.

*** This bug has been marked as a duplicate of bug 1652653 ***