Bug 861701
| Summary: | 3.2 - need to sync networks between vdsm and libvirt. | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Meni Yakove <myakove> | ||||||
| Component: | vdsm | Assignee: | Antoni Segura Puimedon <asegurap> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Meni Yakove <myakove> | ||||||
| Severity: | high | Docs Contact: | |||||||
| Priority: | high | ||||||||
| Version: | 3.2.0 | CC: | bazulay, cpelland, danken, hateya, iheim, lpeer, mavital, sgrinber, ykaul, zdover | ||||||
| Target Milestone: | --- | ||||||||
| Target Release: | 3.2.0 | ||||||||
| Hardware: | x86_64 | ||||||||
| OS: | Linux | ||||||||
| Whiteboard: | network | ||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||
| Doc Text: |
Previously, when a network device was removed from the system through a means that was not VDSM, VDSM recognized that the device was removed and it stopped reporting the device to the engine. But libvirt still held a reference to the removed device. If the user attempted to create the device again, the attempt failed because libvirt still holds a reference to the non-existent device.
Broken networks are now removed from libvirt before new networks are set up. The re-creation of broken networks no longer causes exceptions.
|
Story Points: | --- | ||||||
| Clone Of: | Environment: | ||||||||
| Last Closed: | 2013-06-10 20:32:13 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | Network | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Embargoed: | |||||||||
| Attachments: |
|
||||||||
Another scenario:
1. Create network vmnet1 and attach it to the cluster.
2. On SetupNetworks attach vmnet1 to nic - Don't save network configuration.
3. Create VM and use vmnet1 as nic for this VM.
4. Start the VM
5. On the host restart VDSm service.
Actual results:
Because we didn't save network configuration VDSM rollback the network configuration and now vmnet1 is deleted but the VM is still using vmnet1 network.
brctl show
bridge name bridge id STP enabled interfaces
rhevm 8000.00145e17d5b0 no eth2
vnet1
vmnet1 8000.fe1a4a236168 no vnet0
virsh -r net-list
Name State Autostart
-----------------------------------------
vdsm-rhevm active yes
vdsm-vmnet1 active yes
vmnet1 is attached to vnet0 (the VM network) and vdsm-vmnet1 exist in libvirt.
When stopping the VM vmnet1 is still exist under brctl and in libvirt but without nic attache to it (on brctl).
bridge name bridge id STP enabled interfaces
rhevm 8000.00145e17d5b0 no eth2
vmnet1 8000.000000000000 no
Even when restarting VDSM service again (when the VM is not running) vmnet1 exist in brctl and in libvirt, the only way to return to normal stat is to delete the network from brctl and from libvirt.
Also now when opening SetupNetworks vmnet1 is not attached to any nic and can be use again but when trying to attach vmnet1 to nic there is error:
MainProcess|Thread-173::ERROR::2012-10-03 14:32:04,801::configNetwork::1368::setupNetworks::(setupNetworks) [Errno 2] No such file or directory: '/etc/sysconfig/network-scripts/ifcfg-vmnet1'
Traceback (most recent call last):
File "/usr/share/vdsm/configNetwork.py", line 1310, in setupNetworks
implicitBonding=False)
File "/usr/share/vdsm/configNetwork.py", line 1069, in delNetwork
configWriter.setNewMtu(network=network, bridged=bridged)
File "/usr/share/vdsm/configNetwork.py", line 617, in setNewMtu
currmtu = self._getConfigValue(cf, 'MTU')
File "/usr/share/vdsm/configNetwork.py", line 541, in _getConfigValue
with open(conffile) as f:
IOError: [Errno 2] No such file or directory: '/etc/sysconfig/network-scripts/ifcfg-vmnet1'
MainProcess|Thread-173::ERROR::2012-10-03 14:32:04,802::supervdsmServer::61::SuperVdsm.ServerCallback::(wrapper) Error in setupNetworks
Traceback (most recent call last):
File "/usr/share/vdsm/supervdsmServer.py", line 59, in wrapper
return func(*args, **kwargs)
File "/usr/share/vdsm/supervdsmServer.py", line 107, in setupNetworks
return configNetwork.setupNetworks(networks, bondings, **options)
File "/usr/share/vdsm/configNetwork.py", line 1310, in setupNetworks
implicitBonding=False)
File "/usr/share/vdsm/configNetwork.py", line 1069, in delNetwork
configWriter.setNewMtu(network=network, bridged=bridged)
File "/usr/share/vdsm/configNetwork.py", line 617, in setNewMtu
currmtu = self._getConfigValue(cf, 'MTU')
File "/usr/share/vdsm/configNetwork.py", line 541, in _getConfigValue
with open(conffile) as f:
IOError: [Errno 2] No such file or directory: '/etc/sysconfig/network-scripts/ifcfg-vmnet1'
Thread-173::DEBUG::2012-10-03 14:32:04,803::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/kill -9 19518' (cwd None)
Thread-173::DEBUG::2012-10-03 14:32:04,826::__init__::1164::Storage.Misc.excCmd::(_log) SUCCESS: <err> = ''; <rc> = 0
Thread-173::DEBUG::2012-10-03 14:32:04,833::supervdsm::81::SuperVdsmProxy::(_launchSupervdsm) Launching Super Vdsm
Thread-173::DEBUG::2012-10-03 14:32:04,838::__init__::1164::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/bin/python /usr/share/vdsm/supervdsmServer.py 1c711907-f40d-41d7-bf5b-32d6fe524ba7 19465' (cwd None)
MainThread::DEBUG::2012-10-03 14:32:05,011::supervdsmServer::247::SuperVdsm.Server::(main) Making sure I'm root
MainThread::DEBUG::2012-10-03 14:32:05,011::supervdsmServer::251::SuperVdsm.Server::(main) Parsing cmd args
MainThread::DEBUG::2012-10-03 14:32:05,012::supervdsmServer::254::SuperVdsm.Server::(main) Creating PID file
MainThread::DEBUG::2012-10-03 14:32:05,012::supervdsmServer::258::SuperVdsm.Server::(main) Cleaning old socket
MainThread::DEBUG::2012-10-03 14:32:05,013::supervdsmServer::262::SuperVdsm.Server::(main) Setting up keep alive thread
MainThread::DEBUG::2012-10-03 14:32:05,013::supervdsmServer::267::SuperVdsm.Server::(main) Creating remote object manager
MainThread::DEBUG::2012-10-03 14:32:05,015::supervdsmServer::278::SuperVdsm.Server::(main) Started serving super vdsm object
Thread-176::DEBUG::2012-10-03 14:32:05,116::BindingXMLRPC::864::vds::(wrapper) client [10.35.97.119]::call ping with () {} flowID [7df120fa]
Thread-176::DEBUG::2012-10-03 14:32:05,117::BindingXMLRPC::870::vds::(wrapper) return ping with {'status': {'message': 'Done', 'code': 0}}
Thread-177::DEBUG::2012-10-03 14:32:05,635::BindingXMLRPC::864::vds::(wrapper) client [10.35.97.119]::call ping with () {} flowID [7df120fa]
Thread-177::DEBUG::2012-10-03 14:32:05,636::BindingXMLRPC::870::vds::(wrapper) return ping with {'status': {'message': 'Done', 'code': 0}}
Thread-178::DEBUG::2012-10-03 14:32:06,155::BindingXMLRPC::864::vds::(wrapper) client [10.35.97.119]::call ping with () {} flowID [7df120fa]
Thread-178::DEBUG::2012-10-03 14:32:06,155::BindingXMLRPC::870::vds::(wrapper) return ping with {'status': {'message': 'Done', 'code': 0}}
Thread-179::DEBUG::2012-10-03 14:32:06,673::BindingXMLRPC::864::vds::(wrapper) client [10.35.97.119]::call ping with () {} flowID [7df120fa]
Thread-179::DEBUG::2012-10-03 14:32:06,674::BindingXMLRPC::870::vds::(wrapper) return ping with {'status': {'message': 'Done', 'code': 0}}
Thread-173::DEBUG::2012-10-03 14:32:06,849::supervdsm::102::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm
Thread-173::ERROR::2012-10-03 14:32:06,853::BindingXMLRPC::879::vds::(wrapper) unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/BindingXMLRPC.py", line 869, in wrapper
res = f(*args, **kwargs)
File "/usr/share/vdsm/BindingXMLRPC.py", line 346, in setupNetworks
return api.setupNetworks(networks, bondings, options)
File "/usr/share/vdsm/API.py", line 1115, in setupNetworks
supervdsm.getProxy().setupNetworks(networks, bondings, options)
File "/usr/share/vdsm/supervdsm.py", line 62, in __call__
return callMethod()
File "/usr/share/vdsm/supervdsm.py", line 57, in <lambda>
callMethod = lambda : getattr(self._supervdsmProxy._svdsm, self._funcName)(*args, **kwargs)
File "<string>", line 2, in setupNetworks
File "/usr/lib64/python2.6/multiprocessing/managers.py", line 725, in _callmethod
conn.send((self._id, methodname, args, kwds))
IOError: [Errno 32] Broken pipe
Created attachment 620822 [details]
vdsm.log
Verified on: rhevm-3.2.0-10.16.master.el6ev.noarch vdsm-4.10.2-12.0.el6ev.x86_64 Verified on: rhevm-3.2.0-10.16.master.el6ev.noarch vdsm-4.10.2-12.0.el6ev.x86_64 This bug is currently attached to errata RHBA-2012:14332. If this change is not to be documented in the text for this errata please either remove it from the errata, set the requires_doc_text flag to minus (-), or leave a "Doc Text" value of "--no tech note required" if you do not have permission to alter the flag. Otherwise to aid in the development of relevant and accurate release documentation, please fill out the "Doc Text" field above with these four (4) pieces of information: * Cause: What actions or circumstances cause this bug to present. * Consequence: What happens when the bug presents. * Fix: What was done to fix the bug. * Result: What now happens when the actions or circumstances above occur. (NB: this is not the same as 'the bug doesn't present anymore') Once filled out, please set the "Doc Type" field to the appropriate value for the type of change made and submit your edits to the bug. For further details on the Cause, Consequence, Fix, Result format please refer to: https://bugzilla.redhat.com/page.cgi?id=fields.html#cf_release_notes Thanks in advance. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHSA-2013-0886.html |
Created attachment 619333 [details] vdsm.log Description of problem: When bridge on virsh and on the host (brctl) do not match the network can be used in SetupNetworks again (the network is not attached to any interface). and when trying to assing the network there is error from libvirt: MainProcess|Thread-848::ERROR::2012-09-27 12:16:30,051::configNetwork::1368::setupNetworks::(setupNetworks) operation failed: network 'vdsm-vmnet1' already exists with uuid 205bfb07-9b78-8d00-a6a6-aafe0b90947b Version-Release number of selected component (if applicable): vdsm-4.9.6-36.0.el6_3.x86_64 How reproducible: 100% Steps to Reproduce: 1.Open SetupNetworks and attach vmnet1 to nic. 2.On the host delete the bridge - brctl delbr vmnet1. 3.Open setupNetworks and attach the network net1 to another nic. Actual results: operation failed with errors. Expected results: operation should finished wothout errors. Additional info: