Bug 1305465 - Unble to persist the 'ip route' in RHEV-H
Unble to persist the 'ip route' in RHEV-H
Status: CLOSED WORKSFORME
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: rhev-hypervisor (Show other bugs)
3.5.3
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Fabian Deutsch
wanghui
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-08 05:40 EST by Sarvesh Pandit
Modified: 2016-03-23 09:48 EDT (History)
17 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-03-23 09:48:25 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Node
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)


External Trackers
Tracker ID Priority Status Summary Last Updated
Red Hat Knowledge Base (Solution) 2157361 None None None 2016-02-10 04:22 EST

  None (edit)
Description Sarvesh Pandit 2016-02-08 05:40:21 EST
Description of problem:
Able to persist the 'ip route' in  RHEV-H

Version-Release number of selected component (if applicable):
Red Hat Enterprise Virtualization Hypervisor release 7.1 (20150603.0.el7ev)
rhevm-3.5.3.1-1.4.el6ev.noarch

How reproducible:
100%

Steps to Reproduce:
1. SSH to hypervisor

2. Check current "ip route" 

# ip route show
default via 10.XX.XXX.XXX dev rhevm 
10.XX.XXX.0/24 dev rhevm  proto kernel  scope link  src 10.XX.XXX.XXX

3. To enable the fix to disable TCP delayed ACK:

# /sbin/ip route change default via 10.XX.XXX.XXXdev rhevm quickack 1
# /sbin/ip route change 10.XX.XXX.0/24/24 dev rhevm  proto kernel  scope link  src 10.XX.XXX.XXX  quickack 1

4. Check again "ip route" 

# ip route show
default via 10.XX.XXX.XXX dev rhevm quickack 1
10.XX.XXX.0/24 dev rhevm  proto kernel  scope link  src 10.XX.XXX.XXX quickack 1

5. Persist the changes:

# persist /etc/sysconfig/network-scripts/route-rhevm

6. Reboot the hypervisor

7. Check the "route-rhevm"

# cat /etc/sysconfig/network-scripts/route-rhevm 
/sbin/ip route change default via 10.xx.xxx.xxx dev rhevm quickack 1
/sbin/ip route change 10.xx.xxx.0/24 dev rhevm  proto kernel  scope link  src 10.xx.xxx.xxx  quickack 1

8. Check "ip route"

# ip route show
default via 10.XX.XXX.XXX dev rhevm 
10.XX.XXX.0/24 dev rhevm  proto kernel  scope link  src 10.XX.XXX.XXX

Actual results:
disable TCP delayed ACK i.e. changes are not persisted even though the "route-rhevm" persisted

Expected results:
Changes should be persisted across reboot.

Additional info:
Comment 3 Fabian Deutsch 2016-02-08 06:26:15 EST
Persistence can just be applied to configurations files, changes to the runtime (in-kernel) configuration is not persisted.
Calling the ip utility will only change the runtime.

Thus persisting the ifcfg file after performing some ip calls did not have any effect after a reboot.

If this tuning using ip is done often, the user can consider to open an RFE against vdsm to allow seeting this through RHEV-M.

For the time beeing, the necessary ip calls can be written to /etc/rc.d/rc.local and will then be performed during the boot of RHEV-H.

To persist the rc.local script:

1. Write /etc/rc.d/rc.local
2. chmod a+x /etc/rc.d/rc.local
3. persist /etc/rc.d/rc.local

After reboot the rc.local script will be executed.

Please let me know if this solution is viable to the customer.
Comment 12 Fabian Deutsch 2016-02-17 14:30:03 EST
It could be that we are seeing a race here, a race between the script which is setting up the routes and vdsm. Because vdsm will be taking care of the networking.

Dan, is there a way to configure custom routes as described in comment 0 with vdsm?
Comment 13 Dan Kenigsberg 2016-02-18 02:56:46 EST
Vdsm is expected to rewrite route-rhevm; it cannot be safely persisted without the use of an after_network_setup hook (which is not at all hard to write).

Sarvesh, would you please read bug 1301879 and explain why your own route is needed? I suspect this bug is a dup of rfe 1301879.
Comment 14 Pavel Zhukov 2016-02-18 05:45:34 EST
(In reply to Sarvesh Pandit from comment #0)
> # cat /etc/sysconfig/network-scripts/route-rhevm 
> /sbin/ip route change default via 10.xx.xxx.xxx dev rhevm quickack 1
> /sbin/ip route change 10.xx.xxx.0/24 dev rhevm  proto kernel  scope link 
> src 10.xx.xxx.xxx  quickack 1
are you sure the syntax is correct? If I remember correctly you cannot put ip call inside because it's called from the parser [See 1]
Can you please check documentation.

[1]

handle_ip_file() {
    local f t type= file=$1 proto="-4"
    f=${file##*/}
    t=${f%%-*}
    type=${t%%6}
    if [ "$type" != "$t" ]; then
        proto="-6"
    fi
    { cat "$file" ; echo ; } | while read line; do
        if [[ ! "$line" =~ $MATCH ]]; then
            /sbin/ip $proto $type add $line   <==== !!!!!!
        fi
    done
}
Comment 15 Pavel Zhukov 2016-02-18 05:47:38 EST
(In reply to Dan Kenigsberg from comment #13)
> Vdsm is expected to rewrite route-rhevm; it cannot be safely persisted
> without the use of an after_network_setup hook (which is not at all hard to
> write).
> 
> Sarvesh, would you please read bug 1301879 and explain why your own route is
> needed? I suspect this bug is a dup of rfe 1301879.

Hi Dan, 

I'm sorry for dummy question.
Doesn't vdsm call ifup after generating of ifcfg-rhevm file? If so the route-XXX approach should work because ifup will call ifup-routes (see my previous comment).
Thank you in advance!
Comment 16 Dan Kenigsberg 2016-02-18 07:42:24 EST
Pavel, the problem is that vdsm would overwrite route-rhevm and rule-rhevm. If the customer would like to change the default routing (the one set by http://www.ovirt.org/Features/Multiple_Gateways ) they have to place a hook script in /usr/libexec/vdsm/after_network_setup which can modify route-rhevm (and source it) or call the needed `ip route` commands explicitly.

This can be implemented based on the example of https://gerrit.ovirt.org/#/c/29738/5/vdsm_hooks/extra_ipv4_addrs/extra_ipv4_addrs.py
Comment 18 Fabian Deutsch 2016-02-22 08:55:49 EST
Is there anything else to discuss, or can I close this issue?
Comment 20 Fabian Deutsch 2016-02-22 14:06:51 EST
Sarvesh, to my understanding setting of multiple routes is handled in bug 1301879 and according to comment 13 there is a workaround by using a hook.
Comment 21 Dan Kenigsberg 2016-02-22 15:34:25 EST
Sarvesh, with rhev-3.6 you can also add a hook that modifies route-* files, where you could add "quickack 1" [untested] to the route defined there by Vdsm.
See bug 965929.
Comment 22 Fabian Deutsch 2016-03-08 04:33:06 EST
Sarvesh, are these suggestions enough to solve your issue?
Comment 23 wanghui 2016-03-14 04:17:02 EDT
Hi Sarvesh Pandit,

I have tested this follow you steps but I can not found the /etc/sysconfig/network-scripts/route-rhevm file to persist. So I need to confirm with you that where can I find this file?

Test version:
Red Hat Enterprise Virtualization Hypervisor release 7.1 (20150603.0.el7ev)
Red Hat Enterprise Virtualization Manager Version: 3.5.8-0.1.el6ev

Test steps:
1. SSH to hypervisor
2. Check current "ip route" 
3. To enable the fix to disable TCP delayed ACK:
4. Check again "ip route" 

5. Persist the changes:   
# persist /etc/sysconfig/network-scripts/route-rhevm   -- can not find it

Thanks!
Hui Wang
Comment 25 Fabian Deutsch 2016-03-23 06:27:05 EDT
Sarvesh, do you need anything else?
Comment 27 Fabian Deutsch 2016-03-23 09:48:25 EDT
Thanks. Closing according to comment 26.

Note You need to log in before you can comment on or make changes to this bug.