Bug 1292364 - Engine reports Network error after changing the management network bootproto from dhcp to static. although the change took place
Summary: Engine reports Network error after changing the management network bootproto ...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: ovirt-engine
Version: 3.5.7
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: ---
Assignee: Alona Kaplan
QA Contact: Michael Burman
URL:
Whiteboard: network
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-17 08:27 UTC by Michael Burman
Modified: 2022-07-13 08:10 UTC (History)
10 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-01-25 13:35:58 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
engine log (437.37 KB, application/x-gzip)
2015-12-17 08:27 UTC, Michael Burman
no flags Details
vdsm logs (447.42 KB, application/x-gzip)
2015-12-17 11:01 UTC, Michael Burman
no flags Details


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHV-47646 0 None None None 2022-07-13 08:10:04 UTC

Description Michael Burman 2015-12-17 08:27:25 UTC
Created attachment 1106601 [details]
engine log

Description of problem:
Changing the management network bootproto from dhcp to static ip and vise versa, ends up with "Error while executing action Setup Networks: Network error during communication with the Host." although the change took place in UI - setup networks dialog, in ifcfg-rhevm file and in vdsCaps:

1) Change the rhevm's bootproto from dhcp to static ip:
- error is thrown after few seconds
- UI report static ip
- [root@pink-vds2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-rhevm 
# Generated by VDSM version 4.16.31-1.el7ev
IPADDR=10.35.128.19
NETMASK=255.255.255.0
GATEWAY=10.35.128.254
BOOTPROTO=none
- vdscaps report:
'rhevm': {'addr': '10.35.128.19',
                              'bootproto4': 'none',
                              'bridged': True,
                              'cfg': {'BOOTPROTO': 'none',

2) Change the rhevm's bootproto from static ip back to dhcp:
- error is thrown after few seconds
- UI reports dhcp
- [root@pink-vds2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-rhevm 
# Generated by VDSM version 4.16.31-1.el7ev
BOOTPROTO=dhcp
- vdsCaps report:
'rhevm': {'addr': '10.35.128.19',
                              'bootproto4': 'dhcp',
                              'bridged': True,
                              'cfg': {'BOOTPROTO': 'dhcp',

engine.log:

2015-12-17 10:02:17,243 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand] (ajp-/127.0.0.1:8702-6) [3eb7e877] org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNe
tworkException: Heartbeat exeeded
2015-12-17 10:02:17,262 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-68) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkExce
ption: VDSGenericException: VDSNetworkException: Heartbeat exeeded


2015-12-17 10:02:19,679 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler_Worker-64) [4ff7c13b] ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: XmlRpcRunTimeException: Connection issues during send request, log id: 2fbbd562: org.ovirt.engine.core.vdsbroker.xmlrpc.XmlRpcRunTimeException: Connection issues during send request


2015-12-17 10:02:27,425 ERROR [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-22) Failure to refresh Vds runtime info: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.rmi.ConnectException: Connection timeout


Version-Release number of selected component (if applicable):
3.5.7-0.1.el6ev
vdsm 4.16.31-1.el7ev

How reproducible:
100

Steps to Reproduce:
1. Change rhevm's bootproto from dhcp to static ip and vise versa

Actual results:
- Operation canceled
"Error while executing action Setup Networks: Network error during communication with the Host."
Change took place in:
UI - setup networks dialog
ifcfg-rhevm
vdsCaps

Expected results:
No error message should be displayed.
It's a regression from previous version 3.5.6
 

Additional info:
Maybe relevant to the fix in BZ 1279824

Comment 1 Dan Kenigsberg 2015-12-17 09:47:01 UTC
What Engine version is it? Does the bug show up when you are editing non-management network?

Comment 2 Michael Burman 2015-12-17 09:50:47 UTC
Hi Dan

engine version is provided in description and bug verson ^^ 3.5.7-0.1.el6ev

No, only for management network.

Comment 3 Dan Kenigsberg 2015-12-17 10:10:23 UTC
Can you attach super,vdsm,connectivity.log as well?
Did Vdsm persist the new config?

Comment 4 Michael Burman 2015-12-17 11:01:02 UTC
Created attachment 1106693 [details]
vdsm logs

Comment 5 Michael Burman 2015-12-17 11:01:23 UTC
It seems that vdsm is not persist the new config -->

'rhevm': {'addr': '10.35.128.19',
                              'bootproto4': 'none',
                              'bridged': True,
                              'cfg': {'BOOTPROTO': 'none'

[root@pink-vds2 ~]# cat /var/lib/vdsm/persistence/netconf/nets/rhevm 
{"nic": "enp4s0", "mtu": "1500", "bootproto": "dhcp", "STP": "no", "bridged": "true", "defaultRoute": true}

[root@pink-vds2 ~]# cat /var/run/vdsm/netconf/nets/rhevm 
{"nic": "enp4s0", "ipaddr": "10.35.128.19", "mtu": "1500", "netmask": "255.255.255.0", "STP": "no", "bridged": "true", "gateway": "10.35.128.254", "defaultRoute": true}

Attaching logs.

Comment 6 Michael Burman 2015-12-17 11:07:03 UTC
Important thing i didn't mentioned, the operation cause the server to become non responsive for few seconds, host and engine loosing connectivity for few seconds.

Comment 8 Dan Kenigsberg 2015-12-21 13:35:56 UTC
does the bug show up with engine-3.5.6 and new Vdsm?
An engine-3.5.7 and and old Vdsm?

I don't have a clue were the regression may lie (Engine/Vdsm)

lowering severity as editing management network is not recommended anyway.

Comment 9 Michael Burman 2015-12-21 14:09:58 UTC
Hi Dan, 

The bug doesn't show up in engine 3.5.6 and vdsm 3.5.7.
Managed to reproduce in engine 3.5.7 with older vdsm versions like :
vdsm  4.16.30-1.el7ev.x86_64 and 4.16.28-1.el7ev.x86_64 (reproduced not 100%)

Dan, please note, that this report no longer 100 reproducible, because i had several attempts which succeeded. But it is still failing in 60-70% of the attempts.

Comment 10 Dan Kenigsberg 2015-12-31 13:16:27 UTC
Alona, is there anything new in Engine-3.5.7 that might have triggered this bug?

Comment 11 RHEL Program Management 2016-01-25 13:35:58 UTC
Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.

Comment 12 Dan Kenigsberg 2016-01-26 08:07:01 UTC
Alona has verified that nothing has changed since ovirt-3.5.6 that could have triggered this bug. Moreover, as it reproduces very inconsistently, we believe it is not a regression. It seems like an issue that has been lurking since the introduction of jsonrpc, and should be fixed with bug 1277988 which would stop Engine's change of jsonrpc connection during setupNetworks command.

I do not believe this issue is urgent enough to merit entry to 3.5.8, hence my devel_ack-.


Note You need to log in before you can comment on or make changes to this bug.