Bug 1154737

Summary: [nova][PCI-Passthrough] TypeError: pop() takes at most 1 argument (2 given)
Product: Red Hat OpenStack Reporter: Stephen Gordon <sgordon>
Component: openstack-novaAssignee: Sahid Ferdjaoui <sferdjao>
Status: CLOSED ERRATA QA Contact: Sean Toner <stoner>
Severity: high Docs Contact:
Priority: medium    
Version: 5.0 (RHEL 7)CC: ajeain, beagles, berrange, dasmith, eglynn, jtaleric, ndipanov, pbrady, sbauza, sclewis, sferdjao, sgordon, slong, vromanso, yeylon
Target Milestone: z3Keywords: ZStream
Target Release: 6.0 (Juno)   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-nova-2014.2.2-21 Doc Type: Bug Fix
Doc Text:
The Compute service's pci_passthrough_whitelist configuration option (/etc/nova/nova.conf) is not compatible between the icehouse and juno releases. This resulted in an error being raised when a user migrated from icehouse to juno without updating the pci_passthrough_whitelist configuration. With this update, pci_passthrough_whitelist can now handle both icehouse and juno versions, and no migration errors are raised.
Story Points: ---
Clone Of: 1154201 Environment:
Last Closed: 2015-05-05 13:29:55 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1154201    
Bug Blocks:    

Description Stephen Gordon 2014-10-20 16:13:28 UTC
+++ This bug was initially created as a clone of Bug #1154201 +++

Description of problem:
Setting the configuration to passthrough a device, I get the following:
 
CRITICAL nova [-] TypeError: pop() takes at most 1 argument (2 given) 
2014-10-17 15:28:59.968 7153 CRITICAL nova [-] TypeError: pop() takes at most 1 argument (2 given)
2014-10-17 15:28:59.968 7153 TRACE nova Traceback (most recent call last):
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/bin/nova-compute", line 10, in <module>
2014-10-17 15:28:59.968 7153 TRACE nova     sys.exit(main())
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/cmd/compute.py", line 72, in main
2014-10-17 15:28:59.968 7153 TRACE nova     db_allowed=CONF.conductor.use_local)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/service.py", line 275, in create
2014-10-17 15:28:59.968 7153 TRACE nova     db_allowed=db_allowed)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/service.py", line 148, in __init__
2014-10-17 15:28:59.968 7153 TRACE nova     self.manager = manager_class(host=self.host, *args, **kwargs)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 631, in __init__
2014-10-17 15:28:59.968 7153 TRACE nova     self.driver = driver.load_compute_driver(self.virtapi, compute_driver)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 1402, in load_compute_driver
2014-10-17 15:28:59.968 7153 TRACE nova     virtapi)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/openstack/common/importutils.py", line 50, in import_object_ns
2014-10-17 15:28:59.968 7153 TRACE nova     return import_class(import_value)(*args, **kwargs)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 405, in __init__
2014-10-17 15:28:59.968 7153 TRACE nova     self.dev_filter = pci_whitelist.get_pci_devices_filter()
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/pci/pci_whitelist.py", line 88, in get_pci_devices_filter
2014-10-17 15:28:59.968 7153 TRACE nova     return PciHostDevicesWhiteList(CONF.pci_passthrough_whitelist)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/pci/pci_whitelist.py", line 68, in __init__
2014-10-17 15:28:59.968 7153 TRACE nova     self.specs = self._parse_white_list_from_config(whitelist_spec)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/pci/pci_whitelist.py", line 49, in _parse_white_list_from_config
2014-10-17 15:28:59.968 7153 TRACE nova     spec = pci_devspec.PciDeviceSpec(jsonspec)
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/pci/pci_devspec.py", line 132, in __init__
2014-10-17 15:28:59.968 7153 TRACE nova     self._init_dev_details()
2014-10-17 15:28:59.968 7153 TRACE nova   File "/usr/lib/python2.7/site-packages/nova/pci/pci_devspec.py", line 137, in _init_dev_details
2014-10-17 15:28:59.968 7153 TRACE nova     self.vendor_id = details.pop("vendor_id", ANY)

Version-Release number of selected component (if applicable):
RHEL7 w/ RDO Juno

How reproducible:
100%

Steps to Reproduce:
1. Modify /etc/nova/nova.conf to have the pci_alias and pci_passthrough_whitelist
2. restart openstack services


Actual results:
Failure to start nova-compute

Expected results:
Nova compute start...

Additional info:
The example states in nova.conf for :
# White list of PCI devices available to VMs. For example:
# pci_passthrough_whitelist =  [{"vendor_id": "8086",
# "product_id": "0443"}] (multi valued)
#pci_passthrough_whitelist=

A list [{json....}]

However, to make nova-compute not bomb out, I used : 

pci_passthrough_whitelist={"vendor_id":"8086","product_id":"10fb"}

In the python code: 
/usr/lib/python2.7/site-packages/nova/pci/pci_devspec.py

There is a code reference on line 53 that doesn't show the list either. However, upstream documentation shows to use the list : https://wiki.openstack.org/wiki/Pci_passthrough

Brent Eagles might have more to add, as he was watching me debug the system on bluejeans.

--- Additional comment from Joe Talerico on 2014-10-17 17:03:28 EDT ---

This is n RHEL7

--- Additional comment from Daniel Berrange on 2014-10-20 03:46:19 EDT ---

I get the impression that they have mistakenly changed the config option format in Juno. If correct this needs to be fixed to be backwards compatible with the old format.

--- Additional comment from Dan Smith on 2014-10-20 10:40:09 EDT ---

Yeah, that code was touched at the last minute too.

Can you file a bug upstream against nova?

Comment 5 Sean Toner 2015-04-23 16:18:39 UTC
I am no longer hitting this particular issue.  I assigned pci_passthrough_whitelist=[{"vendor_id": "8086", "product_id": "10c9"}], and did not get the exception anymore

Comment 7 errata-xmlrpc 2015-05-05 13:29:55 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0931.html