Description of problem: For an interface with multiple VLAN interfaces, rhev Host assigns highest mtu of a vlan interface to all vlan interface under the interfaces . Version-Release number of selected component (if applicable): * RHEV 3.2 How reproducible: Steps to Reproduce: 1. Create multiple Logical Interface on vlan, make sure atleast one vlan has mtu value more that 1500 ( say 9000) 2. Add the host to the Cluster . & confirm the network configuration on RHEV Host. Actual results: ifconfig will show all vlan interfaces clubbed under same interface has MTU as 9000 . The ifcfg file will show that the vlan interface selected with higher MTU has vlan set & also the parent interface ( like eth1 ) also has mtu set . The issue here is that that when other vlans are not configured to use 9000 MTU, it ends with network issue & host going Non-responsive / non-operational . Expected results: Additional info: I could confirm the following behaviour which caused this issue . i. By default MTU of an interface is 1500 . TO enable any MTU over this value 'MTU=' syntax needs to be mentioned in its ifcfg file . ( /etc/sysconfig/network-scripts/ifcfg-* ) ii. A vlan interface cannot be assigned a higher MTU than its parent interface over which it lies . ( Hence by default rhevm make sure that the em1 interface gets the highest mtu among all the vlan interface it would create on it ) iii. For a logical network in rhevm , if no MTU is mentioned , rhevm does not add 'MTU' syntax in ifcfg configuration files . iv. If an MTU is not mentioned in ifcfg configuration of a vlan , the network scripts assigns in the mtu of the underlying interfaces . All the above behaviour is expected as a Linux networking perspective . Here the main issue is that rhevm goes ahead & assigns assigns 9000 mtu for interfaces without making the admin aware of this . I propose that rhevm should throw a warning message while setting up network on RHEV Host , if rhevm is clubbing multiple Logical Networks on one interface & atleast one of these networks has an mtu set & another one does not have any mtu set . The message should mention the Logical Networks assigned on the interface & what default MTU will be set for Networks for which Override MTU is not opted for . Since modifying Linux Networking component for this behaviour may not be a feasible solution , hence the above modification would be the best option to make the admin aware of the implications when clubbing vlans with & without mtu set . Expected behaviour : i. While clubbing multiple vlans on RHEV Host ( there is atleast 1 network with mtu set & 1 with default) OR ii. while adding multiple logical networks to cluster which will result in the above i. scenario , there should be a popup warning in web admin GUI .
Why is the reported issue a bug? What averse effects does it have? Why is a warning needed/useful?
Answering myself, by reading comment 0 more carefully. If eth0.100 was defined with no MTU whatsoever, and eth0.200 has MTU=9000, eth0.100 would not see its former default of 1500, but the high MTU of eth0. To avoid that, Vdsm could explicitly interpret "no MTU" as MTU=1500. That's quite easy to do, but we went a long distance in the original MTU support to avoid that: we wanted not to add Vdsm-specific default on top of the one in the kernel. Personally, I do not mind over-riding the kernel default value: it is not going to change from its current 1500 too soon. However, Engine can have its own default, and send it explicitly instead of sending nothing. I find this cleaner - if oVirt needs a default value for MTU, let us keep it at the top, where it is guaranteed to be shared by all clustered hosts or changed in a distant future.
(In reply to Dan Kenigsberg from comment #3) > Answering myself, by reading comment 0 more carefully. If eth0.100 was > defined with no MTU whatsoever, and eth0.200 has MTU=9000, eth0.100 would > not see its former default of 1500, but the high MTU of eth0. > > To avoid that, Vdsm could explicitly interpret "no MTU" as MTU=1500. That's > quite easy to do, but we went a long distance in the original MTU support to > avoid that: we wanted not to add Vdsm-specific default on top of the one in > the kernel. > > Personally, I do not mind over-riding the kernel default value: it is not > going to change from its current 1500 too soon. > > However, Engine can have its own default, and send it explicitly instead of > sending nothing. I find this cleaner - if oVirt needs a default value for > MTU, let us keep it at the top, where it is guaranteed to be shared by all > clustered hosts or changed in a distant future. From engine pov, the state of mind of the internal discussions around the 'jumbo frame' feature was not to define any host's default on engine side, as it may vary between different OSs/HW. If no MTU is configured on a specific network, its value value will be presented to the user as "MTU: Host's default" in the network general sub-tab, stating the engine doesn't guess the host specific default MTU. http://www.ovirt.org/Features/Design/Network/Jumbo_frames
Moti, the only question is whether our former resolution is worth the confusion to the customer. I see three options: 1. Keep things as they are. A customer that cares about his MTU should define it explicitly. 2. Keep thinkgs as they are, but add a warning when Engine expects that the meaning of "default mtu" is about to change for an existing network. That's what Jaison Rju has asked. 3.a. Eliminate the concept of "MTU: Host's default", and use an oVirt-level default. 3.b. implement the oVirt-level default on Vdsm side. I like option 3.a. What is your opinion?
Until this issue is fixed, I can only ask the customer to set an explicit MTU value on his non-jambo networks, too. Option 3.b is as simple as the following patch, but there's some more dead code to be removed, due to handling of non-existing mtu. I am reluctant to implement it since it introduces an ovirt deafult mtu in the back door. --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -234,6 +234,8 @@ def addNetwork(network, vlan=None, bonding=None, nics=None, ipaddr=None, if mtu: mtu = int(mtu) + else: + mtu = netinfo.DEFAULT_MTU if prefix: if netmask:
(In reply to Dan Kenigsberg from comment #5) > Moti, the only question is whether our former resolution is worth the > confusion to the customer. I see three options: > > 1. Keep things as they are. A customer that cares about his MTU should > define it explicitly. > 2. Keep thinkgs as they are, but add a warning when Engine expects that the > meaning of "default mtu" is about to change for an existing network. That's > what Jaison Rju has asked. > 3.a. Eliminate the concept of "MTU: Host's default", and use an oVirt-level > default. > 3.b. implement the oVirt-level default on Vdsm side. > > I like option 3.a. What is your opinion? I'm in favor of 3.a as well. Note that as part of implementing 3.a, we should include the following: 1. An upgrade script to update the network's mtu column (in network table) to 1500. This will simplify the 'sync network' check. 2. Provide the user the ability to define its desired default MTU (1500 will be the system default). This will reflect the actual configured MTU when adding/updating a network via the webadmin/rest.
ovirt-engine-3.5.0-0.0.master.20140715172116.git4687dc1.el6.noarch vdsm-4.16.0-27.git00146ed.el6.x86_64
Alona, please properly document the behavior - including the possible/probable marking of networks as out-of-sync when upgrading the engine to 3.5.
Hi Alona, I've edited the doc text but not sure if I have understand it correctly. Can you please have a look and let me know if anything needs to be changed. Kind regards, Julie
Hi Julie, I would also mention that because of this fix, upgrade to 3.5 engine can cause networks with default mtu to be marked as out-of-sync. (As I explained in the previous doc-text).
(In reply to Alona Kaplan from comment #11) > Hi Julie, > I would also mention that because of this fix, upgrade to 3.5 engine can > cause networks with default mtu to be marked as out-of-sync. (As I explained > in the previous doc-text). hi Alona, Upgrading to 3.5 will get out of sync because of this reason: 'Also note that the default host level MTU must be the same as the data center level MTU, otherwise the network is considered out of synchronization.' Is this not enough to cover this message? Do you also want to suggest here how to fix the out of sync issue after upgrade? BTW, what's your IRC nick? probably easier if I just ping you.
(In reply to Julie from comment #12) > (In reply to Alona Kaplan from comment #11) > > Hi Julie, > > I would also mention that because of this fix, upgrade to 3.5 engine can > > cause networks with default mtu to be marked as out-of-sync. (As I explained > > in the previous doc-text). > > hi Alona, > Upgrading to 3.5 will get out of sync because of this reason: 'Also note > that the default host level MTU must be the same as the data center level > MTU, otherwise the network is considered out of synchronization.' > Is this not enough to cover this message? > It is the reason. But I still think it worth mentioning the upgrade issue to avoid the user be surprised that some of his networks are out-of-sync after the upgrade. Please also mention that setting the value of the default mtu is done by setting 'DefaultMTU' using ovirt-engine-config. > Do you also want to suggest here how to fix the out of sync issue after > upgrade? The fix is just to mark the network as 'has to be synced' via the setup networks. It is trivial, I don't think it worth mentioning it. > > BTW, what's your IRC nick? probably easier if I just ping you. alkaplan, I'm available on ovirt channel.
(In reply to Alona Kaplan from comment #13) > (In reply to Julie from comment #12) > > (In reply to Alona Kaplan from comment #11) > > > Hi Julie, > > > I would also mention that because of this fix, upgrade to 3.5 engine can > > > cause networks with default mtu to be marked as out-of-sync. (As I explained > > > in the previous doc-text). > > > > hi Alona, > > Upgrading to 3.5 will get out of sync because of this reason: 'Also note > > that the default host level MTU must be the same as the data center level > > MTU, otherwise the network is considered out of synchronization.' > > Is this not enough to cover this message? > > > > It is the reason. But I still think it worth mentioning the upgrade issue to > avoid the user be surprised that some of his networks are out-of-sync after > the upgrade. > Please also mention that setting the value of the default mtu is done by > setting 'DefaultMTU' using ovirt-engine-config. engine-config -s DefaultMTU=1500 > > > Do you also want to suggest here how to fix the out of sync issue after > > upgrade? > > The fix is just to mark the network as 'has to be synced' via the setup > networks. It is trivial, I don't think it worth mentioning it. > > > > > BTW, what's your IRC nick? probably easier if I just ping you. > > alkaplan, I'm available on ovirt channel.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-0158.html