Bug 1418614

Summary:	Unexpected exception in API method: got an unexpected keyword argument 'prompt'
Product:	Red Hat OpenStack	Reporter:	Yurii Prokulevych <yprokule>
Component:	openstack-nova	Assignee:	Diana Clarke <dclarke>
Status:	CLOSED NOTABUG	QA Contact:	Prasanth Anbalagan <panbalag>
Severity:	high	Docs Contact:
Priority:	high
Version:	9.0 (Mitaka)	CC:	berrange, dasmith, dclarke, eglynn, jdennis, kchamart, sbauza, sferdjao, sgordon, srevivo, vromanso, yprokule
Target Milestone:	---	Keywords:	ZStream
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-02-08 13:41:34 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Yurii Prokulevych 2017-02-02 10:38:44 UTC

Description of problem:
-----------------------
Fail to spawn a vm in RHOS-9:
nova  boot --poll --flavor m1.ultra --image cirros-0.3.4 --nic net-id=7e002663-44ec-4380-93cf-c618edf91d39 --key-name stack-undercloud --security-group allow-all Admin-VM-1 

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 10.0.0.101 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 10.0.0.101 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
ERROR (ClientException): Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'> (HTTP 500) (Request-ID: req-504d292a-306e-4814-abf2-8d698d9b4c15)

And traceback in nova/nova-api.log:
-----------------------------------
2017-02-02 10:34:14.418 26803 WARNING oslo_config.cfg [req-504d292a-306e-4814-abf2-8d698d9b4c15 788988690f1c463f927b0a8adba8ff5b 25768a1b1fa949f1991b06fe5059d723 - - -] Option "auth_plugin" from group "neutron" 
is deprecated. Use option "auth_type" from group "neutron".
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions [req-504d292a-306e-4814-abf2-8d698d9b4c15 788988690f1c463f927b0a8adba8ff5b 25768a1b1fa949f1991b06fe5059d723 - - -] Unexpected exception in API me
thod
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 629, in create
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     **create_kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/hooks.py", line 154, in inner
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     rv = f(*args, **kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 1563, in create
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     check_server_group_quota=check_server_group_quota)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 1146, in _create_instance
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     reservation_id, max_count)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 873, in _validate_and_build_base_options
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     pci_request_info, requested_networks)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1125, in create_pci_requests_for_sriov_ports
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     neutron = get_client(context, admin=True)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 183, in get_client
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     _ADMIN_AUTH = _load_auth_plugin(CONF)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 116, in _load_auth_plugin
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     auth_plugin = ks_loading.load_auth_from_conf_options(conf, NEUTRON_GROUP)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/conf.py", line 127, in load_from_conf_options
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     plugin_opts = plugin.get_options()
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/keystoneauth1/loading/_plugins/identity/v3.py", line 53, in get_options
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     help="User's password"),
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/positional/__init__.py", line 94, in inner
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions TypeError: __init__() got an unexpected keyword argument 'prompt'
2017-02-02 10:34:14.420 26803 ERROR nova.api.openstack.extensions 
2017-02-02 10:34:14.424 26803 INFO nova.api.openstack.wsgi [req-504d292a-306e-4814-abf2-8d698d9b4c15 788988690f1c463f927b0a8adba8ff5b 25768a1b1fa949f1991b06fe5059d723 - - -] HTTP exception thrown: Unexpected API
 Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'>
2017-02-02 10:34:14.424 26803 DEBUG nova.api.openstack.wsgi [req-504d292a-306e-4814-abf2-8d698d9b4c15 788988690f1c463f927b0a8adba8ff5b 25768a1b1fa949f1991b06fe5059d723 - - -] Returning 500 to user: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<type 'exceptions.TypeError'> __call__ /usr/lib/python2.7/site-packages/nova/api/openstack/wsgi.py:1070
2017-02-02 10:34:14.425 26803 INFO nova.osapi_compute.wsgi.server [req-504d292a-306e-4814-abf2-8d698d9b4c15 788988690f1c463f927b0a8adba8ff5b 25768a1b1fa949f1991b06fe5059d723 - - -] 172.17.1.14 "POST /v2.1/25768a1b1fa949f1991b06fe5059d723/servers HTTP/1.1" status: 500 len: 472 time: 0.6320770


Version-Release number of selected component (if applicable):
-------------------------------------------------------------

openstack-nova-scheduler-13.1.2-9.el7ost.noarch
openstack-nova-api-13.1.2-9.el7ost.noarch
python-nova-13.1.2-9.el7ost.noarch
openstack-nova-console-13.1.2-9.el7ost.noarch
openstack-nova-novncproxy-13.1.2-9.el7ost.noarch
openstack-nova-common-13.1.2-9.el7ost.noarch
openstack-nova-conductor-13.1.2-9.el7ost.noarch
openstack-nova-cert-13.1.2-9.el7ost.noarch
python-novaclient-3.3.2-1.el7ost.noarch
openstack-nova-compute-13.1.2-9.el7ost.noarch

openstack-keystone-9.0.2-1.el7ost.noarch
python-keystoneclient-2.3.1-3.el7ost.noarch
python-keystoneauth1-2.18.0-0.20170120194254.9365dbe.el7ost.noarch
python-keystonemiddleware-4.4.1-1.el7ost.noarch
python-keystone-9.0.2-1.el7ost.noarch

openstack-tripleo-heat-templates-liberty-2.0.0-42.el7ost.noarch
openstack-heat-templates-0-0.3.96a0b0bgit.el7ost.noarch
openstack-tripleo-heat-templates-2.0.0-42.el7ost.noarch


Steps to Reproduce:
-------------------
1. Deploy RHOS-9 with InfraRed (overcloud+ssl+ipv4 network isolation)
2. Create networks/routers
3. Try to spawn a vm

Actual results:
---------------
Fail to spawn a vm

Expected results:
-----------------
VM is successfully spawned

Comment 2 Sahid Ferdjaoui 2017-02-02 14:19:08 UTC

I'm not an expert on Keystone and identity but in nova.conf, the section 'keystone_authtoken', and finally for the option 'auth_uri'. It seems that it's specifically set to use keystone v2.

[keystone_authtoken]
auth_uri=http://172.17.1.10:5000/v2.0

Don't you have to just let the API decide which version to use by removing 'v2.0' ?

[keystone_authtoken]
auth_uri=http://172.17.1.10:5000/

We probably need a Keystone expert to confirm the right configuration.

Comment 3 Diana Clarke 2017-02-03 18:53:37 UTC

Hi Yuri:

Can you please describe the steps you used to install OSP9 + InfraRed? If you followed documentation for that InfraRed install, can you please link to it?

I'm especially curious about anything keystone related because the following version of keystoneauth1 is too new for OSP9/Mitaka, and would explain the errors you're seeing.

- python-keystoneauth1-2.18.0-0.20170120194254.9365dbe.el7ost.noarch

In OSP9/Mitaka, the upper constraint for keystoneauth1 was 2.4.2, so I was surprised to see keystoneauth1 2.18.0 in your listing.

- https://github.com/openstack/requirements/blob/stable/mitaka/upper-constraints.txt

Have a great weekend!

Comment 6 Diana Clarke 2017-02-08 13:41:34 UTC

Thanks for providing those InfraRed install steps & docs, Yurii.

I suspect this bug should be reassigned to the InfraRed team, and that either the ir-installer is grabbing an incompatible keystoneauth1 version, or someone has manually installed an incompatible keystoneauth1 version. Either way, this doesn't appear to be a Nova bug.

I'll close this bug for now, but if you're still able to reproduce these dependency issues in a fresh environment with ir-installer, I would open an InfraRed issue on GitHub since they don't appear to be using Bugzilla to track their issues.

    https://github.com/redhat-openstack/infrared/issues/new

As always, please feel free to re-open this bug if turns out to be a Nova issue after all.

Comment 7 Red Hat Bugzilla 2023-09-14 03:53:03 UTC

The needinfo request[s] on this closed bug have been removed as they have been unresolved for 1000 days