Bug 1325476

Summary:	rhel-osp-director: 7.3->8.0. upgrade, nova/cinder/glance backend on NFS - unable to launch instance"
Product:	Red Hat OpenStack	Reporter:	Alexander Chuzhoy <sasha>
Component:	rhosp-director	Assignee:	Angus Thomas <athomas>
Status:	CLOSED NOTABUG	QA Contact:	Arik Chernetsky <achernet>
Severity:	high	Docs Contact:
Priority:	high
Version:	8.0 (Liberty)	CC:	dbecker, mburns, morazi, rhel-osp-director-maint
Target Milestone:	async	Keywords:	Reopened
Target Release:	8.0 (Liberty)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2016-04-14 00:45:10 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Alexander Chuzhoy 2016-04-09 04:18:24 UTC

rhel-osp-director: 7.3->8.0. upgrade, nova/cinder/glance backend on NFS - nova-compute appears down in "nova service-list"


Environment:
openstack-tripleo-heat-templates-0.8.14-7.el7ost.noarch
instack-undercloud-2.2.7-4.el7ost.noarch
openstack-puppet-modules-7.0.17-1.el7ost.noarch
openstack-tripleo-heat-templates-kilo-0.8.14-7.el7ost.noarch


Steps to reproduce:

1. Deploy 7.3 with cinder/glance on NFS.
2. Manually configure Nova with NFS backend.
3. Populate the OC (launch instance).
4. Upgrade the OC to 8.0


Result:
The upgrade succeeded, but unable to launch an instance.

[stack@instack ~]$ nova service-list
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 2  | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-09T03:47:49.000000 | -               |
| 5  | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-09T03:47:56.000000 | -               |
| 8  | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-09T03:47:51.000000 | -               |
| 11 | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-09T03:47:54.000000 | -               |
| 14 | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-09T03:47:54.000000 | -               |
| 17 | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-09T03:47:52.000000 | -               |
| 20 | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-09T03:47:53.000000 | -               |
| 23 | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-09T03:47:54.000000 | -               |
| 29 | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-09T03:47:54.000000 | -               |
| 35 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | down  | 2016-04-08T22:56:36.000000 | -               |
| 38 | nova-compute     | overcloud-compute-1.localdomain    | nova     | enabled | down  | 2016-04-08T22:56:17.000000 | -               |
| 39 | nova-compute     | localhost                          | nova     | enabled | up    | 2016-04-09T03:47:54.000000 | -               |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+

Comment 2 Alexander Chuzhoy 2016-04-09 04:33:03 UTC

nova-compute.log on compute has many occurences of:
2016-04-09 04:25:23.747 31590 WARNING nova.compute.resource_tracker [req-3aab040a-2809-4b43-a2f3-cebb16ae69df - - - - -] No compute node record for localhost:overcloud-compute-1.localdomain
2016-04-09 04:27:26.444 31590 ERROR nova.compute.manager [req-3aab040a-2809-4b43-a2f3-cebb16ae69df - - - - -] No compute node record for host localhost
2016-04-09 04:27:26.448 31590 WARNING nova.compute.monitors [req-3aab040a-2809-4b43-a2f3-cebb16ae69df - - - - -] Excluding nova.compute.monitors.cpu monitor virt_driver. Not in the list of enabled monitors (CONF.compute_monitors).
2016-04-09 04:27:26.693 31590 WARNING nova.compute.resource_tracker [req-3aab040a-2809-4b43-a2f3-cebb16ae69df - - - - -] No compute node record for localhost:overcloud-compute-1.localdomain
2016-04-09 04:29:27.446 31590 ERROR nova.compute.manager [req-3aab040a-2809-4b43-a2f3-cebb16ae69df - - - - -] No compute node record for host localhost

Comment 4 Mike Burns 2016-04-11 13:39:56 UTC

This appears to be due to a different bug where the nova hostname got changed.

*** This bug has been marked as a duplicate of bug 1324739 ***

Comment 5 Daniel Becker 2016-04-13 11:34:05 UTC

Not sure why this is not a dupe any more?
Moving to async and removing blocker as per the duplicate

Comment 6 Alexander Chuzhoy 2016-04-13 12:55:30 UTC

Because with the "duplicate" bug - there's no issue launching an instance.

Comment 7 Alexander Chuzhoy 2016-04-13 22:24:30 UTC

Didn't reproduce.

Comment 8 Mike Burns 2016-04-14 00:45:10 UTC

closing per comment 7, please reopen if it reproduces