Description of problem:
During one of the upgrade steps I have included additional template called "major-upgrade-pacemaker.yaml". After this step I am not able to boot new instance on overcloud. It seems like the auth_plugin parameter (under [neutron] section in nova.conf) is not set. Nova-api throws the following traceback:

2016-10-25 13:58:11.204 20299 WARNING keystonemiddleware.auth_token [-] Using the in-process token cache is deprecated as of the 4.2.0 release and may be removed in the 5.0.0 release or the 'O' development cycle. The in-process cache causes inconsistent results and high memory usage. When the feature is removed the auth_token middleware will not cache tokens by default which may result in performance issues. It is recommended to use  memcache for the auth_token token cache by setting the memcached_servers option.
2016-10-25 13:58:11.206 20299 WARNING oslo_config.cfg [-] Option "memcached_servers" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions [req-d17a7110-2fb3-473f-8dda-0540d8176316 3de032dcc73c4c418fb02c5634d279c3 da845199f7fd4532adc6904aaebe3350 - - -] Unexpected exception in API method
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions Traceback (most recent call last):
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/extensions.py", line 478, in wrapped
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     return f(*args, **kwargs)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/validation/__init__.py", line 73, in wrapper
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     return func(*args, **kwargs)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/api/openstack/compute/servers.py", line 629, in create
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     **create_kwargs)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/hooks.py", line 154, in inner
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     rv = f(*args, **kwargs)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 1563, in create
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     check_server_group_quota=check_server_group_quota)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 1146, in _create_instance
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     reservation_id, max_count)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/compute/api.py", line 873, in _validate_and_build_base_options
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     pci_request_info, requested_networks)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1125, in create_pci_requests_for_sriov_ports
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     neutron = get_client(context, admin=True)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 183, in get_client
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     _ADMIN_AUTH = _load_auth_plugin(CONF)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions   File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 122, in _load_auth_plugin
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions     raise neutron_client_exc.Unauthorized(message=err_msg)
2016-10-25 13:58:14.062 20327 ERROR nova.api.openstack.extensions Unauthorized: Unknown auth type: None


Version-Release number of selected component (if applicable):
[root@overcloud-controller-0 ~]# rpm -qa | grep nova
openstack-nova-scheduler-13.1.1-7.el7ost.noarch
openstack-nova-compute-13.1.1-7.el7ost.noarch
openstack-nova-cert-13.1.1-7.el7ost.noarch
openstack-nova-common-13.1.1-7.el7ost.noarch
openstack-nova-console-13.1.1-7.el7ost.noarch
openstack-nova-conductor-13.1.1-7.el7ost.noarch
python-novaclient-3.3.2-1.el7ost.noarch
openstack-nova-api-13.1.1-7.el7ost.noarch
python-nova-13.1.1-7.el7ost.noarch
openstack-nova-novncproxy-13.1.1-7.el7ost.noarch

How reproducible:


Steps to Reproduce:
1. execute major version upgrade from director including "major-upgrade-pacemaker.yaml" step.
2. Try to boot VM from director using
[stack@osp9-director ~]$ source overcloudrc
[stack@osp8-director ~]$ nova --debug boot --flavor 2 --image rhel7 --nic net-id=cc5c67be-1ce0-4606-9d63-5f31fe361ffa after_step5

Actual results:
DEBUG (session:248) REQ: curl -g -i -X POST http://10.2.116.50:8774/v2.1/da845199f7fd4532adc6904aaebe3350/servers -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.25" -H "X-Auth-Token: {SHA1}afb1d5fe6c9f31f4d5921fd2b102aa4360f1b4ee" -d '{"server": {"name": "after_step5", "imageRef": "7cba2c2f-a87c-420d-b05b-fcd790c0d11b", "flavorRef": "2", "max_count": 1, "min_count": 1, "networks": [{"uuid": "cc5c67be-1ce0-4606-9d63-5f31fe361ffa"}]}}'
DEBUG (connectionpool:395) "POST /v2.1/da845199f7fd4532adc6904aaebe3350/servers HTTP/1.1" 500 218
DEBUG (session:277) RESP: [500] X-Openstack-Nova-Api-Version: 2.25 Vary: X-OpenStack-Nova-API-Version Content-Type: application/json; charset=UTF-8 Content-Length: 218 X-Compute-Request-Id: req-d17a7110-2fb3-473f-8dda-0540d8176316 Date: Tue, 25 Oct 2016 13:58:14 GMT
RESP BODY: {"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'neutronclient.common.exceptions.Unauthorized'>", "code": 500}}

DEBUG (shell:1082) Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'neutronclient.common.exceptions.Unauthorized'> (HTTP 500) (Request-ID: req-d17a7110-2fb3-473f-8dda-0540d8176316)
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1080, in main
    OpenStackComputeShell().main(argv)
  File "/usr/lib/python2.7/site-packages/novaclient/shell.py", line 1007, in main
    args.func(self.cs, args)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/shell.py", line 589, in do_boot
    server = cs.servers.create(*boot_args, **boot_kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 1233, in create
    **boot_kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/v2/servers.py", line 667, in _boot
    return_raw=return_raw, **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/base.py", line 345, in _create
    resp, body = self.api.client.post(url, body=body)
  File "/usr/lib/python2.7/site-packages/keystoneauth1/adapter.py", line 179, in post
    return self.request(url, 'POST', **kwargs)
  File "/usr/lib/python2.7/site-packages/novaclient/client.py", line 94, in request
    raise exceptions.from_response(resp, body, url, method)
ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.
<class 'neutronclient.common.exceptions.Unauthorized'> (HTTP 500) (Request-ID: req-d17a7110-2fb3-473f-8dda-0540d8176316)


Expected results:
VM started (available as building/active in "nova list").


Additional info: