Bug 1443078
| Summary: | USB legacy VM originating in 3.5 env fails to start - Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [oVirt] ovirt-engine | Reporter: | Jiri Belka <jbelka> | ||||||||||
| Component: | BLL.Virt | Assignee: | jniederm | ||||||||||
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Israel Pinto <ipinto> | ||||||||||
| Severity: | high | Docs Contact: | |||||||||||
| Priority: | high | ||||||||||||
| Version: | 4.1.1.8 | CC: | bugs, dougsland, ipinto, jniederm, michal.skrivanek, mtessun, nashok, nsimsolo, stirabos, tjelinek | ||||||||||
| Target Milestone: | ovirt-4.1.3 | Flags: | rule-engine:
ovirt-4.1+
rule-engine: exception+ mtessun: planning_ack+ tjelinek: devel_ack+ ipinto: testing_ack+ |
||||||||||
| Target Release: | 4.1.3.3 | ||||||||||||
| Hardware: | Unspecified | ||||||||||||
| OS: | Unspecified | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
| Doc Text: |
Cause:
Consequence: VMs started in engine version 4.0 or older with "USB Support" set to "Legacy (deprecated)" might fail to run after shutdown in engine version 4.1. The problem was caused by collision of bus addresses of usb controllers - set of USB controllers explicitly defined by engine and USB controller implicitly added by libvirt.
Fix: Implicit USB controller added by libvirt is removed.
Result:
|
Story Points: | --- | ||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2017-07-06 13:35:10 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | Virt | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Embargoed: | |||||||||||||
| Attachments: |
|
||||||||||||
it has the same root cause as 1438188. Since the outcome is quite different, leaving this bug here for independent verification. this has actually been fixed in 4.1.2 by the same patches as: https://bugzilla.redhat.com/show_bug.cgi?id=1438188 moving to the same state. Failed QA, checked with: RHVM Version: 4.1.2.2-0.1.el7 Host: OS Version:RHEL - 7.3 - 7.el7 Kernel Version:3.10.0 - 327.28.3.el7.x86_64 KVM Version:2.6.0 - 28.el7_3.9 LIBVIRT Version:libvirt-2.0.0-10.el7_3.5 VDSM Version:vdsm-4.19.15-1.el7ev Steps: 1. 3.5 vm in 3.5 env with above settings (usb legacy,spice, 4 monitors, 2 netifaces ...) 2. upgrade the env till 4.1 while keeping the VM running 3. stop the vm in 4.1 env and try to start it again Results: Vm failed to start Engine log: 2017-05-22 21:07:10,939+03 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler2) [] EVENT_ID: VM_DOWN_ERROR(119), Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM BZ_1443078_usb is down with error. Exit message: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0). Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release. adding sosreport from engine and hosts Created attachment 1281177 [details]
engine sosreport
Created attachment 1281178 [details]
host1 sosreport
Created attachment 1281179 [details]
host2 sosreport
Hi Israel, could you please keep that testing vm and create another 3.5 setup? Thanks we need to see the VM definition in 3.5, either please provide a log showing original creation and startup of that 3.5 VM (the logs only contain the situation after 4.1 upgrade) or ideally an environment prior the engine upgrade Created attachment 1281583 [details]
vm_devices-of-vm-after-stop-in-4.1-env.txt
This happens if the VM is set with usb: enabled_legacy (which behaves same as disabled so it will have an unmanaged usb). After update the enabled_legacy is considered to be enabled, consequently the acquireUnmanagedUsbController is not called to adopt this unmanaged controller. a workaround is not difficult - just edit the VM and change USB settings and save, then change back *** Bug 1455124 has been marked as a duplicate of this bug. *** Can you please check the status of this bug? It's on modified but it references a patch not yet merged. The patch was abandoned. Sorry for the confusion. Verify with: Red Hat Virtualization Manager Version: 4.1.3.4-0.1.el7 Host/s: OS Version:RHEL - 7.4 - 11.el7 Kernel Version:3.10.0 - 327.28.3.el7.x86_64 KVM Version:2.9.0 - 12.el7 LIBVIRT Version:libvirt-3.2.0-4.el7 VDSM Version:vdsm-4.19.19-1.el7ev SPICE Version:0.12.8 - 2.el7 Steps: 1. 3.5 vm in 3.5 env with above settings (usb legacy,spice, 4 monitors, 2 netifaces ...) 2. upgrade the env till 4.1 while keeping the VM running 3. stop the vm in 4.1 env and try to start it again Results: PASS |
Description of problem: A VM originating in 3.5 env with "spice, 4 monitors, 2 netifaces, usb legacy, vioscsi, viomb, watchdog, smartcard, soundcard" configuration fails to start in 4.1 env - it was all the time running since 3.5 and "survived" all cluster compat levels changes and finally it was shutdown in 4.1 env era and attempted to start again... 2017-04-18 13:56:25,262+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-6-thread-45) [] START, CreateVDSCommand(HostName = slot-5d, CreateVmVDSCommandParameters:{runAsync='true', hostId='998cf6db-ad07-4a42-a103-fa018c3e1120', vmId='27f1a8d4-1f81-49e6-bd83-4ecb0a462b54', vm='VM [w7-1]'}), log id: 178cdeb0 ... 2017-04-18 13:56:30,532+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-8) [] EVENT_ID: VM_DOWN_ERROR(119), Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM w7-1 is down with error. Exit message: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0). Apr 18 13:56:28 slot-5d journal: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0) 2017-04-18 13:56:28,096+0200 ERROR (vm/27f1a8d4) [virt.vm] (vmId='27f1a8d4-1f81-49e6-bd83-4ecb0a462b54') The vm start process failed (vm:617) Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 553, in _startUnderlyingVm self._run() File "/usr/share/vdsm/virt/vm.py", line 2006, in _run self._connection.createXML(domxml, flags), File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 941, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3777, in createXML if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self) libvirtError: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0) 2017-04-18 13:56:28,100+0200 INFO (vm/27f1a8d4) [virt.vm] (vmId='27f1a8d4-1f81-49e6-bd83-4ecb0a462b54') Changed state to Down: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0) (code=1) (vm:1207) Version-Release number of selected component (if applicable): ovirt-engine-4.1.1.8-0.1.el7.noarch How reproducible: just happened Steps to Reproduce: 1. 3.5 vm in 3.5 env with above settings (usb legacy!) 2. upgrade the env till 4.1 while keeping the VM running 3. stop the vm in 4.1 env and try to start it again Actual results: fails to start - Message: VM w7-1 is down with error. Exit message: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0). Expected results: either it should run or there should be a warning that such "legacy" vm configuration is not supported anymore Additional info: