Bug 1365789 - Duplicate nova hypervisors show up in nova hypervisor-list post 9 -> 10 upgrade
Summary: Duplicate nova hypervisors show up in nova hypervisor-list post 9 -> 10 upgrade
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: rc
: 10.0 (Newton)
Assignee: Marios Andreou
QA Contact: Marius Cornea
URL:
Whiteboard:
Depends On: 1352057
Blocks: 1333977 1337794
TreeView+ depends on / blocked
 
Reported: 2016-08-10 08:38 UTC by Marius Cornea
Modified: 2016-12-29 16:57 UTC (History)
12 users (show)

Fixed In Version: openstack-tripleo-heat-templates-5.0.0-0.20161008015357.0d3e3e3.1.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of: 1352057
: 1406826 (view as bug list)
Environment:
Last Closed: 2016-12-14 15:50:27 UTC


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHEA-2016:2948 normal SHIPPED_LIVE Red Hat OpenStack Platform 10 enhancement update 2016-12-14 19:55:27 UTC

Description Marius Cornea 2016-08-10 08:38:05 UTC
In OSP10 ComputeHostnameFormat is set to '%stackname%-novacompute-%index% so we'll most probably hit the same issue with it as in the past.

+++ This bug was initially created as a clone of Bug #1352057 +++

+++ This bug was initially created as a clone of Bug #1324739 +++

This is the same as bug 1324739 but occuring in 8->9 upgrade.

[stack@instack ~]$ nova service-list
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 3  | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-06-29T16:06:37.000000 | -               |
| 6  | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-06-29T16:06:41.000000 | -               |
| 9  | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-06-29T16:06:44.000000 | -               |
| 12 | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-06-29T16:06:45.000000 | -               |
| 15 | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-06-29T16:06:40.000000 | -               |
| 18 | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-06-29T16:06:40.000000 | -               |
| 21 | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-06-29T16:06:37.000000 | -               |
| 24 | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-06-29T16:06:45.000000 | -               |
| 30 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | down  | 2016-06-29T16:02:10.000000 | -               |
| 33 | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-06-29T16:06:39.000000 | -               |
| 62 | nova-compute     | localhost                          | nova     | enabled | up    | 2016-06-29T16:06:41.000000 | -               |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
[stack@instack ~]$ nova hypervisor-list
+----+---------------------------------+-------+---------+
| ID | Hypervisor hostname             | State | Status  |
+----+---------------------------------+-------+---------+
| 3  | overcloud-compute-0.localdomain | down  | enabled |
| 5  | overcloud-compute-0.localdomain | up    | enabled |
+----+---------------------------------+-------+---------+

--- Additional comment from marios on 2016-07-06 12:28:17 EDT ---

I hit this today, doing the 8..9 upgrade and specifically after the first step, the aodh migration:


Deploy like:

    openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml  --control-scale 3 --compute-scale 1 --libvirt-type qemu -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e network_env.yaml --ntp-server '0.fedora.pool.ntp.org'

Apply the aodh migration:

    openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates -e /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml  --control-scale 3 --compute-scale 1 --libvirt-type qemu -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e network_env.yaml --ntp-server '0.fedora.pool.ntp.org' -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-aodh.yaml


On the undercloud nova list shows "overcloud-novacompute-0". You can see related discussion in the bug this is cloned from and pointers to proposed upstream fixes. But I think we can/should ultimately land this downstream as it is a downstream only problem, as we did for 8, see the review pointed at in the tracker above.

thanks, marios

--- Additional comment from errata-xmlrpc on 2016-07-20 11:40:53 EDT ---

Bug report changed to ON_QA status by Errata System.
A QE request has been submitted for advisory RHEA-2016:23454-01
https://errata.devel.redhat.com/advisory/23454

--- Additional comment from Omri Hochman on 2016-07-26 17:10:32 EDT ---

Verified .

Looks good with openstack-tripleo-heat-templates-2.0.0-21.el7ost.noarch


[stack@undercloud72 ~]$ . overcloudrc 
[stack@undercloud72 ~]$ nova hypervisor-list
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 10.19.184.210 has no `subjectAltName`, falling back to check for a `commo
ature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:303: SubjectAltNameWarning: Certificate for 10.19.184.210 has no `subjectAltName`, falling back to check for a `commo
ature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SubjectAltNameWarning
+----+---------------------------------+-------+---------+
| ID | Hypervisor hostname             | State | Status  |
+----+---------------------------------+-------+---------+
| 2  | overcloud-compute-0.localdomain | up    | enabled |
+----+---------------------------------+-------+---------+


+-----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id  | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+-----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 2   | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-07-26T21:07:20.000000 | -               |
| 5   | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-07-26T21:07:20.000000 | -               |
| 8   | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-07-26T21:07:20.000000 | -               |
| 11  | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-07-26T21:07:19.000000 | -               |
| 14  | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-07-26T21:07:18.000000 | -               |
| 17  | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-07-26T21:07:18.000000 | -               |
| 20  | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-07-26T21:07:22.000000 | -               |
| 56  | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-07-26T21:07:22.000000 | -               |
| 86  | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-07-26T21:07:22.000000 | -               |
| 104 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | up    | 2016-07-26T21:07:19.000000 | -               |
+-----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+


nova list : 

+--------------------------------------+-------------------------+--------+------------+-------------+-----------------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks              |
+--------------------------------------+-------------------------+--------+------------+-------------+-----------------------+
| eff0cbec-79fa-4d67-b185-216d0cf45f4c | overcloud-cephstorage-0 | ACTIVE | -          | Running     | ctlplane=192.168.0.8  |
| 13889a06-8be4-49fa-a0dc-2e1a7f9fb893 | overcloud-cephstorage-1 | ACTIVE | -          | Running     | ctlplane=192.168.0.7  |
| 3028f323-6cd7-4e54-a605-f061741ada20 | overcloud-compute-0     | ACTIVE | -          | Running     | ctlplane=192.168.0.9  |
| e6313445-96cf-4fe2-b70d-6386dc64f6d4 | overcloud-controller-0  | ACTIVE | -          | Running     | ctlplane=192.168.0.11 |
| a62f2db7-64d3-4762-87a2-851db25157ef | overcloud-controller-1  | ACTIVE | -          | Running     | ctlplane=192.168.0.10 |
| af1b33ca-9cef-49ac-a724-654fad68f841 | overcloud-controller-2  | ACTIVE | -          | Running     | ctlplane=192.168.0.12 |

--- Additional comment from Jiri Stransky on 2016-08-01 07:15:30 EDT ---

BZ not in released code -> no doc text.

Comment 5 Omri Hochman 2016-10-10 16:19:35 UTC
After a discussion with Marios, we've realized that by using the "environment file which landed upstream that allows us to keep the existing overcloud-compute hostnames " - > there will be many cases in Update/Upgrade  
that Unless including this environment file the operation will fail,  

In order to avoid this complexity, and potential hazard to Upgrade/Update,
the fix to this bug would be to keep the 'Compute-node naming format' as it was before OSP10 :    

overcloud-compute-0    (should remain)  
overcloud-nova-compute-0 (remove this format)

Comment 8 Jaromir Coufal 2016-10-11 14:13:47 UTC
Ack. Thanks guys

Comment 14 errata-xmlrpc 2016-12-14 15:50:27 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2948.html


Note You need to log in before you can comment on or make changes to this bug.