Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1455124

Summary: VMs with legacy USB device will fail to start with "PCI slot duplicate error" if these VMs are not down while upgrading from 3.6 to 4.1
Product: Red Hat Enterprise Virtualization Manager Reporter: nijin ashok <nashok>
Component: ovirt-engineAssignee: Nobody <nobody>
Status: CLOSED DUPLICATE QA Contact: Israel Pinto <ipinto>
Severity: high Docs Contact:
Priority: high    
Version: 4.1.1CC: lsurette, nashok, rbalakri, Rhev-m-bugs, srevivo, tjelinek, ykaul
Target Milestone: ovirt-4.1.3   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-25 07:51:03 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Virt RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description nijin ashok 2017-05-24 10:15:28 UTC
Description of problem:

If the VM is started with legacy USB device, it will create following entries in the database on 3.6 RHEV-M.

engine=# select device_id,spec_params from vm_device where vm_id = '4d000ed5-d371-41a6-b347-c29f0f9c45b4' and device='usb';
              device_id               | spec_params 
--------------------------------------+-------------
 f62d59d2-cbf2-4677-a5e3-2cb7fea5c66e | { }
(1 row)

The same entry will be there in the database for these VMs after upgrading to 4.0 . 

Now change the cluster compatibility of the cluster to 4.0 and then Upgrade RHV-M to 4.1. 

If we shutdown the VM at this state, it will create devices for both legacy and native USB device in the database.


              device_id               |       spec_params        
--------------------------------------+--------------------------
 f62d59d2-cbf2-4677-a5e3-2cb7fea5c66e | { }
 81d8f783-84c5-43ec-8b0c-a6d6abaf82a3 | {                       +
                                      |   "index" : "0",        +
                                      |   "model" : "ich9-uhci2"+
                                      | }
 ced4809e-59ec-42ef-98a7-134531a26cc4 | {                       +
                                      |   "index" : "0",        +
                                      |   "model" : "ich9-uhci3"+
                                      | }
 4b54563e-a2a6-4e10-a518-274ac73a31df | {                       +
                                      |   "index" : "0",        +
                                      |   "model" : "ich9-uhci1"+
                                      | }
 4f4649a8-f772-41e6-bf1b-89d1a573abcb | {                       +
                                      |   "index" : "0",        +
                                      |   "model" : "ich9-ehci1"+
                                      | }

Because of the presence of the legacy device, VM will fail to start with below errors with duplicate PCI slot error.


2017-05-24 09:35:19,138+0530 ERROR (vm/4d000ed5) [virt.vm] (vmId='4d000ed5-d371-41a6-b347-c29f0f9c45b4') The vm start process failed (vm:617)
Traceback (most recent call last):
  File "/usr/share/vdsm/virt/vm.py", line 553, in _startUnderlyingVm
    self._run()
  File "/usr/share/vdsm/virt/vm.py", line 2006, in _run
    self._connection.createXML(domxml, flags),
  File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 123, in wrapper
    ret = f(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 941, in wrapper
    return func(inst, *args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3777, in createXML
    if ret is None:raise libvirtError('virDomainCreateXML() failed', conn=self)
libvirtError: internal error: Attempted double use of PCI slot 0000:00:01.0 (may need "multifunction='on'" for device on function 0)

To resolve the issue, we have to manually remove the legacy USB device from the vm_device  table ( f62d59d2-cbf2-4677-a5e3-2cb7fea5c66e ).


Version-Release number of selected component (if applicable):

Version after upgrade:

rhevm-4.1.1.8-0.1.el7.noarch
ovirt-engine-4.1.1.8-0.1.el7.noarch

Before upgrade:

rhevm-3.6.11.1-0.1.el6.noarch

Host version:

vdsm-4.19.10.1-1.el7ev.x86_64
libvirt-daemon-2.0.0-10.el7_3.5.x86_64


How reproducible:

100%

Steps to Reproduce:

1. create a 3.6 RHV-M environment with 4.1 host.

2. Start a VM in this environment with USB device as legacy. This should create a USB device for the VM in the database.

3. Upgrade RHV-M to 4.0 by keeping the VM as up.

4. Change the cluster compatibility to 4.0 . Don't shutdown the VM.

5. Upgrade the RHV-M to 4.1 .

6. Shutdown the VM . Check the vm_device table if it now contains both legacy and native USB devices.

7. Start the VM. It will fail with error "Attempted double use of PCI slot"

Actual results:

VM is not able to start with PCI slot error

Expected results:

VM should be able to start without any error.

Additional info:

Issue is not reproduced if,

[1] VM is down while migrating to 4.0.

[2] VM is made down before upgrading to 4.1.

Comment 1 Yaniv Kaul 2017-05-24 10:42:48 UTC
Duplicate of bug 1443078 ?

Comment 2 Yaniv Kaul 2017-05-24 10:43:29 UTC
(In reply to Yaniv Kaul from comment #1)
> Duplicate of bug 1443078 ?

And if it is, see workaround at https://bugzilla.redhat.com/show_bug.cgi?id=1443078#c16

Comment 3 Tomas Jelinek 2017-05-24 11:23:46 UTC
yes, a duplicate.

@Nijin, can you please try to workaround?

Comment 5 nijin ashok 2017-05-25 04:57:51 UTC
(In reply to Tomas Jelinek from comment #3)
> yes, a duplicate.
> 
> @Nijin, can you please try to workaround?

I missed the other bug. It's the same issue. 

Yes, workaround works fine. Just disabling and enabling USB removed the legacy device from the database. Then the VM boot up fine.

Comment 6 Yaniv Kaul 2017-05-25 07:51:03 UTC

*** This bug has been marked as a duplicate of bug 1443078 ***