Description of problem: Unable to add multiple static routes for logical network using rhevm portal. It is possible to create them manually but the changes should persist after reinstallation or upgrade of hypervisor.
Another example of a required routing can be seen on https://bugzilla.redhat.com/show_bug.cgi?id=1301879#c14
*** Bug 1301879 has been marked as a duplicate of this bug. ***
I wonder if it can be easily done via RHEVH NGN Cockpit interface?
*** Bug 1406951 has been marked as a duplicate of this bug. ***
Dan, haven't we done any work in this area in 4.2?
Static routes are not on our list for 4.3, but it is still a feature that we'd like to support, eventually - even if only via Ansible.
I am interested in the feedback of all subscribers to this bug about a new workaround. The idea is to improve the workaround from Knowledge Base Solution 46060. In RHV 4.4 VDSM will utilize the NetworkManager (bug 1107803) to manage the network configuration. This will enable other software than VDSM to modify the network configuration via NetworkManager. The purposed new workaround is to add the static route via nmstate on each host manually. This will work without any interference with VDSM. nmstate will be called like this: nmstatectl set add_static_route.yml with add_static_route.yml like this: --- routes: config: - destination: 192.168.123.0/24 metric: 100 next-hop-address: 192.168.178.1 next-hop-interface: ovirtmgmt table-id: 254 interfaces: [] Do you think that this would be acceptable for customers? Thanks for your feedback
Hi Dominik, can I share a few thoughts? I have a customer that needs the ability to create static routes... After looking briefly at 'nmstate' it seems to be too heavy for this use-case, but maybe useful for other situations I'm not familiar with... Also, will the effect of 'nmstate set ...' persist beyond reboots? nmstate has a few dependencies that will need to be present in the RHV-H ISO image. nmstate also requires too much yml input from the customer. I believe the yml file should only require minimal input, similar to creating a static route via "ip route add ..." command. [root@jcall-laptop ~]# ip route add 192.168.2.0/24 via 192.168.0.99 ---------------------------------------------------------------------------------------------------- [root@jcall-laptop ~]# cat ~jcall/add_static_route.yml --- routes: config: - destination: 192.168.3.0/24 next-hop-address: 192.168.0.99 [root@jcall-laptop ~]# nmstatectl set /home/jcall/add_static_route.yml Traceback (most recent call last): File "/usr/bin/nmstatectl", line 11, in <module> load_entry_point('nmstate==0.0.8', 'console_scripts', 'nmstatectl')() File "/usr/lib/python3.7/site-packages/nmstatectl/nmstatectl.py", line 62, in main return args.func(args) File "/usr/lib/python3.7/site-packages/nmstatectl/nmstatectl.py", line 220, in apply statedata, args.verify, args.commit, args.timeout File "/usr/lib/python3.7/site-packages/nmstatectl/nmstatectl.py", line 240, in apply_state checkpoint = libnmstate.apply(state, verify_change, commit, timeout) File "/usr/lib/python3.7/site-packages/libnmstate/netapplier.py", line 57, in apply validator.validate_capabilities(desired_state, netinfo.capabilities()) File "/usr/lib/python3.7/site-packages/libnmstate/validator.py", line 59, in validate_capabilities validate_interface_capabilities(state[Constants.INTERFACES], capabilities) KeyError: 'interfaces' ---------------------------------------------------------------------------------------------------- Has anybody considered using a simple NetworkManager dispatcher script like this? https://developer.gnome.org/NetworkManager/stable/NetworkManager.html [root@jcall-laptop ~]# chown root.root /etc/NetworkManager/dispatcher.d/999-add-static-route.sh [root@jcall-laptop ~]# chmod 0755 /etc/NetworkManager/dispatcher.d/999-add-static-route.sh [root@jcall-laptop ~]# cat /etc/NetworkManager/dispatcher.d/999-add-static-route.sh #!/bin/bash # $1 is the interface name; $2 is the new state of the interface ROUTE_INTERFACE="enp0s31f6" ROUTE_DESTINATION="192.168.4.0/24" ROUTE_NEXTHOP="192.168.0.99" if [[ "$1" =~ $WIRED_INTERFACES ]]; then case "$2" in up) ip route add $ROUTE_DESTINATION via $ROUTE_NEXTHOP ;; down) ip route del $ROUTE_DESTINATION via $ROUTE_NEXTHOP ;; esac fi [root@jcall-laptop ~]# Any thoughts about this? Or am I way off? Thanks, John
> Hi Dominik, can I share a few thoughts? Thank you! > I have a customer that needs the > ability to create static routes... > > After looking briefly at 'nmstate' it seems to be too heavy for this use-case, > but maybe useful for other situations I'm not familiar with... nmstate will be also used by VDSM to communicate with NetworkManager, so nmstate comes with no extra costs here. > Also, will the effect of 'nmstate set ...' persist beyond reboots? > Yes, it modifies the network connection profile managed by NetworkManager. > nmstate has a few dependencies that will need to be present in the RHV-H ISO > image. nmstate also requires too much yml input from the customer. I believe > the yml file should only require minimal input, similar to creating a static > route via "ip route add ..." command. > > [root@jcall-laptop ~]# ip route add 192.168.2.0/24 via 192.168.0.99 > > ---------------------------------------------------------------------------------------------------- > [root@jcall-laptop ~]# cat ~jcall/add_static_route.yml > --- > routes: > config: > - destination: 192.168.3.0/24 > next-hop-address: 192.168.0.99 > > [root@jcall-laptop ~]# nmstatectl set /home/jcall/add_static_route.yml > Traceback (most recent call last): > File "/usr/bin/nmstatectl", line 11, in <module> > load_entry_point('nmstate==0.0.8', 'console_scripts', 'nmstatectl')() > File "/usr/lib/python3.7/site-packages/nmstatectl/nmstatectl.py", line 62, in > main > return args.func(args) > File "/usr/lib/python3.7/site-packages/nmstatectl/nmstatectl.py", line 220, > in apply > statedata, args.verify, args.commit, args.timeout > File "/usr/lib/python3.7/site-packages/nmstatectl/nmstatectl.py", line 240, > in apply_state > checkpoint = libnmstate.apply(state, verify_change, commit, timeout) > File "/usr/lib/python3.7/site-packages/libnmstate/netapplier.py", line 57, in > apply > validator.validate_capabilities(desired_state, netinfo.capabilities()) > File "/usr/lib/python3.7/site-packages/libnmstate/validator.py", line 59, in > validate_capabilities > validate_interface_capabilities(state[Constants.INTERFACES], capabilities) > KeyError: 'interfaces' > ---------------------------------------------------------------------------------------------------- > If nmstate will be accepted as the workaround, I will try to push https://nmstate.atlassian.net/projects/NMSTATE/issues/NMSTATE-243 which will enable a centralized management via ansible with an interface similar what you suggested. > > Has anybody considered using a simple NetworkManager dispatcher script like > this? > https://developer.gnome.org/NetworkManager/stable/NetworkManager.html > > [root@jcall-laptop ~]# chown root.root > /etc/NetworkManager/dispatcher.d/999-add-static-route.sh > [root@jcall-laptop ~]# chmod 0755 > /etc/NetworkManager/dispatcher.d/999-add-static-route.sh > [root@jcall-laptop ~]# cat > /etc/NetworkManager/dispatcher.d/999-add-static-route.sh > #!/bin/bash > # $1 is the interface name; $2 is the new state of the interface > > ROUTE_INTERFACE="enp0s31f6" > ROUTE_DESTINATION="192.168.4.0/24" > ROUTE_NEXTHOP="192.168.0.99" > > if [[ "$1" =~ $WIRED_INTERFACES ]]; then > case "$2" in > up) > ip route add $ROUTE_DESTINATION via $ROUTE_NEXTHOP > ;; > down) > ip route del $ROUTE_DESTINATION via $ROUTE_NEXTHOP > ;; > esac > fi > [root@jcall-laptop ~]# > > > Any thoughts about this? Or am I way off? The routes should be managed by NetworkManager. With plain nmcli this would be nmcli connection modify ovirtmgmt +ipv4.routes "192.168.124.0/24 10.10.10.4" && nmcli device reapply ovirtmgmt Even setting the route using cockpit would work. Using nmstate with the yml file will help to document how the route was set. > Thanks, > John >
(In reply to Dominik Holler from comment #24) > If nmstate will be accepted as the workaround, I will try to push > https://nmstate.atlassian.net/projects/NMSTATE/issues/NMSTATE-243 > which will enable a centralized management via ansible with an interface > similar what you suggested. > I don't quite understand how issue #243, and the related ansible module for net_static_route would help. That module doesn't appear to support RHEL. It looks to only support switches/routers from Cisco, Juniper, etc... > > Has anybody considered using a simple NetworkManager dispatcher script like > > this? > > https://developer.gnome.org/NetworkManager/stable/NetworkManager.html > > > The routes should be managed by NetworkManager. > With plain nmcli this would be > > nmcli connection modify ovirtmgmt +ipv4.routes "192.168.124.0/24 10.10.10.4" > && nmcli device reapply ovirtmgmt > > Even setting the route using cockpit would work. Oh yes, I would much prefer to use normal "nmcli con mod ovirtmgmt ..." commands. That's much cleaner than my silly idea of a dispatch script.
(In reply to John Call from comment #25) > (In reply to Dominik Holler from comment #24) > > If nmstate will be accepted as the workaround, I will try to push > > https://nmstate.atlassian.net/projects/NMSTATE/issues/NMSTATE-243 > > which will enable a centralized management via ansible with an interface > > similar what you suggested. > > > > I don't quite understand how issue #243, and the related ansible module for > net_static_route would help. That module doesn't appear to support RHEL. > It looks to only support switches/routers from Cisco, Juniper, etc... > The net_static_route ansible role, which is more an interface than an implementation could be implemented by https://github.com/nmstate/ansible-nmstate to enable RHEL. But at first, we have to check if this would solve the issue discussed in this bug, by using a workaround which works in a similar way.
(In reply to Dominik Holler from comment #20) > I am interested in the feedback of all subscribers to this bug about a new > workaround. > The idea is to improve the workaround from Knowledge Base Solution 46060. > > In RHV 4.4 VDSM will utilize the NetworkManager (bug 1107803) to manage the > network configuration. > This will enable other software than VDSM to modify the network > configuration via NetworkManager. > > The purposed new workaround is to add the static route via nmstate on each > host manually. > This will work without any interference with VDSM. > nmstate will be called like this: > nmstatectl set add_static_route.yml > with add_static_route.yml like this: > --- > routes: > config: > - destination: 192.168.123.0/24 > metric: 100 > next-hop-address: 192.168.178.1 > next-hop-interface: ovirtmgmt > table-id: 254 > interfaces: [] > > Do you think that this would be acceptable for customers? > Thanks for your feedback Germano, can you please review this and modify your solution? https://access.redhat.com/solutions/46060
(In reply to Marina Kalinin from comment #27) > (In reply to Dominik Holler from comment #20) > > I am interested in the feedback of all subscribers to this bug about a new > > workaround. > > The idea is to improve the workaround from Knowledge Base Solution 46060. > > > > In RHV 4.4 VDSM will utilize the NetworkManager (bug 1107803) to manage the > > network configuration. > > This will enable other software than VDSM to modify the network > > configuration via NetworkManager. > > > > The purposed new workaround is to add the static route via nmstate on each > > host manually. > > This will work without any interference with VDSM. > > nmstate will be called like this: > > nmstatectl set add_static_route.yml > > with add_static_route.yml like this: > > --- > > routes: > > config: > > - destination: 192.168.123.0/24 > > metric: 100 > > next-hop-address: 192.168.178.1 > > next-hop-interface: ovirtmgmt > > table-id: 254 > > interfaces: [] > > > > Do you think that this would be acceptable for customers? > > Thanks for your feedback > > Germano, can you please review this and modify your solution? > https://access.redhat.com/solutions/46060 Since we are currently not using NetworkManager, I think we can only modify the solution after we release 4.4 on EL8. No? Other than that, I think the workaround if good enough.
This bug is spit into: - bug 1771969 to document proposed approach from comment 20 - bug 1771977 to ensure that the approach is working - bug 1771981 to add ansible support for static routes This bug will be used to track the RHVM portal UI for the functionality.
Adjusting the flags based on the conversation above.
The manual procedure is documented in https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4-beta/html-single/administration_guide/index#proc-Adding-a-static-route-on-a-host should enable the functionality in RHV-4.4 using nmstate/NetworkManager.
The workaround documented in Bug 1771969 - [Docs] Document defining static routes on RHV hosts seems to work quite well.