Bug 1419931
| Summary: | Failed to destroy partially-initialized VM with port mirroring | ||||||
|---|---|---|---|---|---|---|---|
| Product: | [oVirt] ovirt-engine | Reporter: | Israel Pinto <ipinto> | ||||
| Component: | BLL.Virt | Assignee: | Dan Kenigsberg <danken> | ||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Israel Pinto <ipinto> | ||||
| Severity: | high | Docs Contact: | |||||
| Priority: | medium | ||||||
| Version: | 4.0.6.3 | CC: | bugs, danken, edwardh, gklein, ipinto, michal.skrivanek, tjelinek, ylavi | ||||
| Target Milestone: | ovirt-4.1.1 | Flags: | rule-engine:
ovirt-4.1+
|
||||
| Target Release: | 4.1.1.2 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | |||||||
| : | 1430658 (view as bug list) | Environment: | |||||
| Last Closed: | 2017-04-21 09:41:17 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | Network | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1430658 | ||||||
| Attachments: |
|
||||||
|
Description
Israel Pinto
2017-02-07 12:32:30 UTC
Not sure what does it have to do with the tasks. What I see is this: - try to stop a VM - stop VM fails due to Internal JSON-RPC error. (for this I'd like to see the VDSM logs to see what happened) - the VM is still running, try to start it again - start VM fails since the VM is running The issue I see here is the failed stop attempt. Can you please attach the VDSM logs? I was not able to dig them out from jenkins... This VM is down for about 30 min and when trying to start it is failed with "Desktop already exist", which is very wired. I see that tasks are not cleaned and this VM is reported as failed to stop/destroy. I guess that can be relate to the fact it failed to run. I will try to add all logs but there is limitation of size ... (In reply to Israel Pinto from comment #2) > This VM is down for about 30 min and when trying to start it is failed with > "Desktop already exist", which is very wired. > I see that tasks are not cleaned and this VM is reported as failed to > stop/destroy. I guess that can be relate to the fact it failed to run. > > I will try to add all logs but there is limitation of size ... Use Google drive. from the failed stop you can see that it was running on
hostId='db68f77f-551f-465e-bc8d-7d1711d27f0e' name = host_mixed_1
ip = 10.35.117.48
-- Engine log failed to stop VM:
2017-02-06 06:28:45,935 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (DefaultQuartzScheduler4) [3d3c11bf] Command 'DestroyVDSCommand(HostName = host_mixed_1, DestroyVmVDSCommandParameters:{runAsync='true', hostId='db68f77f-551f-465e-bc8d-7d1711d27f0e', vmId='2707de71-e71f-4a33-b057-7205b06d548c', force='false', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'})' execution failed: VDSGenericException: VDSErrorException: Failed to DestroyVDS, error = name, code = -32603
vdsm log (see vdsm.log.1 in host-logs:
1. There are a lot of server internal errors
jsonrpc.Executor/5::DEBUG::2017-02-06 04:07:48,846::API::314::vds::(destroy) About to destroy VM 2707de71-e71f-4a33-b057-7205b06d548c
jsonrpc.Executor/5::DEBUG::2017-02-06 04:07:48,846::vm::4217::virt.vm::(destroy) vmId=`2707de71-e71f-4a33-b057-7205b06d548c`::destroy Called
jsonrpc.Executor/5::ERROR::2017-02-06 04:07:48,846::__init__::550::jsonrpc.JsonRpcServer::(_handle_request) Internal server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 545, in _handle_request
res = method(**params)
File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 195, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/API.py", line 320, in destroy
res = v.destroy(gracefulAttempts)
File "/usr/share/vdsm/virt/vm.py", line 4219, in destroy
result = self.doDestroy(gracefulAttempts)
File "/usr/share/vdsm/virt/vm.py", line 4237, in doDestroy
return self.releaseVm(gracefulAttempts)
File "/usr/share/vdsm/virt/vm.py", line 4125, in releaseVm
nic.name)
AttributeError: name
2. And at the same time of stop on the engine same error:
jsonrpc.Executor/4::DEBUG::2017-02-06 06:28:44,928::API::314::vds::(destroy) About to destroy VM 2707de71-e71f-4a33-b057-7205b06d548c
jsonrpc.Executor/4::DEBUG::2017-02-06 06:28:44,929::vm::4217::virt.vm::(destroy) vmId=`2707de71-e71f-4a33-b057-7205b06d548c`::destroy Called
jsonrpc.Executor/4::ERROR::2017-02-06 06:28:44,929::__init__::550::jsonrpc.JsonRpcServer::(_handle_request) Internal server error
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 545, in _handle_request
res = method(**params)
File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 195, in _dynamicMethod
result = fn(*methodArgs)
File "/usr/share/vdsm/API.py", line 320, in destroy
res = v.destroy(gracefulAttempts)
File "/usr/share/vdsm/virt/vm.py", line 4219, in destroy
result = self.doDestroy(gracefulAttempts)
File "/usr/share/vdsm/virt/vm.py", line 4237, in doDestroy
return self.releaseVm(gracefulAttempts)
File "/usr/share/vdsm/virt/vm.py", line 4125, in releaseVm
nic.name)
AttributeError: name
jsonrpc.Executor/4::INFO::2017-02-06 06:28:44,930::__init__::513::jsonrpc.JsonRpcServer::(_serveRequest) RPC call VM.destroy failed (error -32603) in 0.01 seconds
google drive logs:
https://drive.google.com/drive/folders/0BxBXxc5waimaUkU3RjdwY3VNdGc
Michal - can someone look at this? This is an exception in networking code. Dan, any recent changes in that area? vdsm-4.18.21.1-1.el7ev.x86_64 is not so recent... (2 months old) Neither me not `git log --oneline ovirt/ovirt-4.0 -- vdsm/virt/vmdevices/network.py` recall any related change. Eddy, can you look deeper into the vdsm.log to tell how could we have a nameless nic? I don't see it as a blocker. It pops up in a very rare case (qemu crash right after startup) of a rather rare use case (port mirroring). Verify with: Red Hat Virtualization Manager Version: 4.1.1.2-0.1.el7 |