Bug 1324739 - Duplicate nova hypervisors show up in nova hypervisor-list post 7.3 -> 8 upgrade
Summary: Duplicate nova hypervisors show up in nova hypervisor-list post 7.3 -> 8 upgrade
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-tripleo-heat-templates
Version: 8.0 (Liberty)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: async
: 8.0 (Liberty)
Assignee: Marios Andreou
QA Contact: Marius Cornea
URL:
Whiteboard:
: 1325536 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-04-07 07:27 UTC by Marius Cornea
Modified: 2019-03-14 17:09 UTC (History)
10 users (show)

Fixed In Version: openstack-tripleo-heat-templates-0.8.14-9.el7ost
Doc Type: Bug Fix
Doc Text:
Previously, there was a divergence in the default value for the ComputeHostnameFormat parameter between what was originally deployed for Red Hat Enterprise Linux OpenStack Platform 7.3 and the current default value in Red Hat OpenStack Platform 8.0 ("overcloud-compute-N" instead of "overcloud-novacompute-N"). This would cause upgraded compute nodes to have an incorrect setting for their host name, resulting in issues such as the node being listed twice in the output of 'nova hypervisor-list', and issues restarting existing instances. With this update, the value of ComputeHostnameFormat is now preserved during upgrade so that it does not change from the overcloud-compute-N pattern, allowing compute nodes to be upgraded correctly and to retain the same host name as that prior to the upgrade.
Clone Of:
: 1352057 (view as bug list)
Environment:
Last Closed: 2016-04-20 13:04:44 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 304342 0 None ABANDONED change default compute hostnames to compute 2020-09-30 21:34:39 UTC
Red Hat Product Errata RHBA-2016:0653 0 normal SHIPPED_LIVE Red Hat Enterprise Linux OSP 8 director Bug Fix Advisory 2016-04-20 17:02:07 UTC

Description Marius Cornea 2016-04-07 07:27:25 UTC
Description of problem:
I've got a 3 ctrls + 1 compute + 2 ceph nodes; IPv6 + SSL deployment.

After upgrading from 7.3 -> 8 and rebooting all the nodes nova shows duplicate hypervisors of the same compute node and it doesn't refresh the instances status(they show as ACTIVE even though the compute node has been restarted and no VMs are running on it). 

Version-Release number of selected component (if applicable):
openstack-tripleo-heat-templates-0.8.14-5.el7ost.noarch

How reproducible:

Steps to Reproduce:
1. Upgrade 7.3 -> 8 following the procedure
2. Reboot all the nodes in the cluster serially
3. Verify the overcloud nova instances status 

Actual results:

stack@instack:~>>> nova hypervisor-list
+----+---------------------------------+-------+---------+
| ID | Hypervisor hostname             | State | Status  |
+----+---------------------------------+-------+---------+
| 2  | overcloud-compute-0.localdomain | down  | enabled |
| 3  | overcloud-compute-0.localdomain | up    | enabled |
+----+---------------------------------+-------+---------+

stack@instack:~>>> nova service-list
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 2  | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-07T07:22:48.000000 | -               |
| 5  | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-07T07:22:45.000000 | -               |
| 8  | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-07T07:22:50.000000 | -               |
| 11 | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-07T07:22:48.000000 | -               |
| 14 | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-07T07:22:44.000000 | -               |
| 17 | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-07T07:22:48.000000 | -               |
| 20 | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-07T07:22:47.000000 | -               |
| 26 | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-07T07:22:40.000000 | -               |
| 32 | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-07T07:22:41.000000 | -               |
| 38 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | down  | 2016-04-06T13:23:53.000000 | -               |
| 39 | nova-compute     | localhost                          | nova     | enabled | up    | 2016-04-07T07:22:45.000000 | -               |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+

stack@instack:~>>> nova list
+--------------------------------------+----------------------------------------------------+--------+------------+-------------+--------------------------------------------------------------------------------------------------+
| ID                                   | Name                                               | Status | Task State | Power State | Networks                                                                                         |
+--------------------------------------+----------------------------------------------------+--------+------------+-------------+--------------------------------------------------------------------------------------------------+
| 957900be-ff9b-4d58-85ae-1a1f4de2d102 | stack01-vm01-iniq47w6yudz-my_instance-bnjfhsulyfw4 | ACTIVE | -          | Running     | stack01-tenant_net_ext_tagged-2taq4jb4esmh-private_network-4a5ljbjlrhwv=10.10.10.5, 172.29.0.102 |
| 43edf30d-e66b-4a4b-b497-a39157493703 | stack01-vm02-db7ex52fohdk-my_instance-me6zyibue5pe | ACTIVE | -          | Running     | stack01-tenant_net_ext_flat-dhpw5q2dkzcm-private_network-fojno5stcgp5=10.10.10.5, 172.29.1.102   |
| b75d9f51-eed2-4493-8660-6d58a59239df | stack01-vm03-v5dtcxa6em75-my_instance-fui57say4j42 | ACTIVE | -          | Running     | provider01=172.29.2.103                                                                          |
+--------------------------------------+----------------------------------------------------+--------+------------+-------------+--------------------------------------------------------------------------------------------------+

Expected results:
There are no duplicate nova hypervisors and the state of the instances reflects that no VMs are running on the host.

Additional info:

The new nova hypervisor which shows with the 'Up' state shows 'localhost' for the service_host:

stack@instack:~>>> nova hypervisor-show 2
+---------------------------+------------------------------------------+
| Property                  | Value                                    |
+---------------------------+------------------------------------------+
| cpu_info_arch             | x86_64                                   |
| cpu_info_features         | ["pge", "clflush", "sep", "syscall",     |
|                           | "tsc", "cmov", "fpu", "pat", "lm",       |
|                           | "msr", "nx", "fxsr", "pae", "mmx",       |
|                           | "cx8", "mce", "de", "mca", "pse", "pni", |
|                           | "abm", "apic", "sse", "lahf_lm", "sse2", |
|                           | "hypervisor", "cx16", "pse36", "mtrr",   |
|                           | "x2apic"]                                |
| cpu_info_model            | cpu64-rhel6                              |
| cpu_info_topology_cells   | 1                                        |
| cpu_info_topology_cores   | 1                                        |
| cpu_info_topology_sockets | 4                                        |
| cpu_info_topology_threads | 1                                        |
| cpu_info_vendor           | Intel                                    |
| current_workload          | 0                                        |
| disk_available_least      | 69                                       |
| free_disk_gb              | 66                                       |
| free_ram_mb               | 6255                                     |
| host_ip                   | 172.16.22.10                             |
| hypervisor_hostname       | overcloud-compute-0.localdomain          |
| hypervisor_type           | QEMU                                     |
| hypervisor_version        | 2003000                                  |
| id                        | 2                                        |
| local_gb                  | 81                                       |
| local_gb_used             | 15                                       |
| memory_mb                 | 9839                                     |
| memory_mb_used            | 3584                                     |
| running_vms               | 3                                        |
| service_disabled_reason   | None                                     |
| service_host              | overcloud-compute-0.localdomain          |
| service_id                | 38                                       |
| state                     | down                                     |
| status                    | enabled                                  |
| vcpus                     | 4                                        |
| vcpus_used                | 3                                        |
+---------------------------+------------------------------------------+
stack@instack:~>>> nova hypervisor-show 3
+---------------------------+------------------------------------------+
| Property                  | Value                                    |
+---------------------------+------------------------------------------+
| cpu_info_arch             | x86_64                                   |
| cpu_info_features         | ["pge", "clflush", "sep", "syscall",     |
|                           | "tsc", "cmov", "fpu", "pat", "lm",       |
|                           | "msr", "nx", "fxsr", "pae", "mmx",       |
|                           | "cx8", "mce", "de", "mca", "pse", "pni", |
|                           | "abm", "apic", "sse", "lahf_lm", "sse2", |
|                           | "hypervisor", "cx16", "pse36", "mtrr",   |
|                           | "x2apic"]                                |
| cpu_info_model            | cpu64-rhel6                              |
| cpu_info_topology_cells   | 1                                        |
| cpu_info_topology_cores   | 1                                        |
| cpu_info_topology_sockets | 4                                        |
| cpu_info_topology_threads | 1                                        |
| cpu_info_vendor           | Intel                                    |
| current_workload          | 0                                        |
| disk_available_least      | 68                                       |
| free_disk_gb              | 81                                       |
| free_ram_mb               | 7791                                     |
| host_ip                   | 172.16.22.10                             |
| hypervisor_hostname       | overcloud-compute-0.localdomain          |
| hypervisor_type           | QEMU                                     |
| hypervisor_version        | 2003000                                  |
| id                        | 3                                        |
| local_gb                  | 81                                       |
| local_gb_used             | 0                                        |
| memory_mb                 | 9839                                     |
| memory_mb_used            | 2048                                     |
| running_vms               | 0                                        |
| service_disabled_reason   | None                                     |
| service_host              | localhost                                |
| service_id                | 39                                       |
| state                     | up                                       |
| status                    | enabled                                  |
| vcpus                     | 4                                        |
| vcpus_used                | 0                                        |
+---------------------------+------------------------------------------+

Comment 3 Marius Cornea 2016-04-07 07:52:05 UTC
This looks to be have pretty much the same cause as BZ#1320777 since I got an empty resolv.conf due to BZ#1324160.

But I rebooted the nodes and now they got proper resolv.conf since the network scripts were run correctly at boot so I'd expect the instances status to be reported correctly. I'm not sure how I can bring the instances back on at this point. Doing nova stop on one of the instances gets it stuck in the powering-off task state.

Comment 4 Marius Cornea 2016-04-07 10:39:24 UTC

*** This bug has been marked as a duplicate of bug 1320777 ***

Comment 6 Marius Cornea 2016-04-08 14:53:14 UTC
stack@instack:~>>> nova service-list
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 3  | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-08T14:52:57.000000 | -               |
| 6  | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-08T14:52:57.000000 | -               |
| 9  | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-08T14:52:51.000000 | -               |
| 12 | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-08T14:52:54.000000 | -               |
| 15 | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-08T14:52:57.000000 | -               |
| 18 | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-08T14:52:50.000000 | -               |
| 21 | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-08T14:52:52.000000 | -               |
| 27 | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-08T14:52:55.000000 | -               |
| 30 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | down  | 2016-04-08T13:53:35.000000 | -               |
| 33 | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-08T14:52:50.000000 | -               |
| 36 | nova-compute     | localhost                          | nova     | enabled | up    | 2016-04-08T14:52:48.000000 | -               |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+

Comment 7 Marius Cornea 2016-04-08 15:18:16 UTC
[heat-admin@overcloud-compute-0 ~]$ hostname -f
localhost
[heat-admin@overcloud-compute-0 ~]$ facter fqdn
overcloud-compute-0.localdomain

Comment 8 Marius Cornea 2016-04-08 16:25:58 UTC
I see that there's a naming difference between versions for the nova compute hosts:

nova service-list | grep nova-compute
 30 | nova-compute     | overcloud-compute-0.localdomain     | nova     | enabled | down  | 2016-04-08T13:53:35.000000 | -               |
| 36 | nova-compute     | localhost                           | nova     | enabled | up    | 2016-04-08T16:22:38.000000 | -               |
| 39 | nova-compute     | overcloud-novacompute-1.localdomain | nova     | enabled | up    | 2016-04-08T16:22:32.000000 | -               |


overcloud-compute-0.localdomain was part of the initial OSP7 deployment. After upgrade it became localhost. 

overcloud-novacompute-1.localdomain is a new compute node added to the deployment post upgrade.

Comment 9 Marios Andreou 2016-04-11 09:48:42 UTC
(In reply to Marius Cornea from comment #8)
> I see that there's a naming difference between versions for the nova compute
> hosts:
> 
> nova service-list | grep nova-compute
>  30 | nova-compute     | overcloud-compute-0.localdomain     | nova     |
> enabled | down  | 2016-04-08T13:53:35.000000 | -               |
> | 36 | nova-compute     | localhost                           | nova     |
> enabled | up    | 2016-04-08T16:22:38.000000 | -               |
> | 39 | nova-compute     | overcloud-novacompute-1.localdomain | nova     |
> enabled | up    | 2016-04-08T16:22:32.000000 | -               |
> 
> 
> overcloud-compute-0.localdomain was part of the initial OSP7 deployment.
> After upgrade it became localhost. 
> 
> overcloud-novacompute-1.localdomain is a new compute node added to the
> deployment post upgrade.

I am trying to understand more about what is happening here. I am also using another environment which has just been upgraded and which has hit BZ 1325536 (may be related but not sure yet) - thanks to mcornea.

I see that there is indeed a small diff in the /etc/hosts after the upgrade and it may be the root of the issue here.  On the post-upgrade env, on the compute (also on a controller), node it looks like:


    # HEAT_HOSTS_START - Do not edit manually within this section!
    fd00:fd00:fd00:2000::13 overcloud-novacompute-0.localdomain overcloud-novacompute-0

and indeed from the undercloud, the compute node is called "overcloud-novacompute-0"


| 43f4b683-6c63-418a-ab6a-49b1832224fd | overcloud-novacompute-0 | ACTIVE | -          | Running     | ctlplane=192.0.2.9  |


However, we previously had the compute node called just 'overcloud-compute-0'. After editing /etc/hosts and changing :%s/novacompute/compute/gc hostname -f returns "overcloud-compute-0.localdomain".

Also, predictably, changing /etc/hostname to "overcloud-novacompute-0.localdomain" (in fact on the upgraded environment we still have in /etc/hostname "overcloud-compute-0.localdomain")
lets hostname -f return a correct value. Still poking.

Comment 10 Marios Andreou 2016-04-11 10:42:17 UTC
Update - this looks like another case of something landed downstream a little differently to upstream. The review which adds the NodeHostnameFormat params downstream is at https://code.engineering.redhat.com/gerrit/#/c/50813/1/overcloud-without-mergepy.yaml and this sets the compute node to '%stackname%-compute-%index%'

I think it is because that ^^^ was cherrypicked from the upstream at version 7 https://review.openstack.org/#/c/191726/7/overcloud-without-mergepy.yaml . However, along came gfidente. And so there was v8, which makes it "novacompute-%index%" https://review.openstack.org/#/c/191726/8/overcloud-without-mergepy.yaml

I think we should be able to pass ComputeHostnameFormat: '%stackname%-compute-%index%' in the upgrades environment files - I am still testing this though and still not sure it is the right fix...

Comment 11 Marius Cornea 2016-04-11 13:18:13 UTC
*** Bug 1325536 has been marked as a duplicate of this bug. ***

Comment 12 Mike Burns 2016-04-11 13:36:33 UTC
*** Bug 1325475 has been marked as a duplicate of this bug. ***

Comment 13 Mike Burns 2016-04-11 13:39:56 UTC
*** Bug 1325476 has been marked as a duplicate of this bug. ***

Comment 14 Alexander Chuzhoy 2016-04-11 15:01:42 UTC
Reproduced with IPv4.

Comment 15 Marios Andreou 2016-04-11 16:45:44 UTC
No solution yet but some debug info until I can pick up again tomorrow - tl;dr it's cloud-init doing the resetting of the hostname. I spent quite a while working out what was changing the hostname here, no matter how I set it explicitly, I tried:

 - hostnamectl set-hostname overcloud-novacompute-0.localdomain
 - editing /etc/hostname
 - editing /etc/sysconfig/network with HOSTNAME=overcloud-novacompute-0.localdomain
 
In all cases on reboot the compute node got "overcloud-compute-0.localdomain" in /etc/hostname and hostname -f "localhost" etc etc as described above.

I eventually found it was cloud-init doing the hostname setting. The only way to set 'overcloud-novacompute-0.localdomain' and get it to suvive a reboot was to add "preserve_hostname: true" in /etc/cloud/cloud.cfg (which tells cloud-init not to change the hostname). 

I also came across this upstream bug https://bugs.launchpad.net/cloud-init/+bug/1424710 which I think may be related; I can confirm the code at /usr/lib/python2.7/site-packages/cloudinit/distros/rhel.py looks like the 'problematic' code pointed out it that bug ( I have cloud-init-0.7.6-2.el7.x86_64 ):

    119     def _write_hostname(self, hostname, out_fn):
    120         if self._dist_uses_systemd():
    121             util.subp(['hostnamectl', 'set-hostname', str(hostname)])
    122         else:
    123             host_cfg = {
    124                 'HOSTNAME': hostname,
    125             }
    126             rhel_util.update_sysconfig_file(out_fn, host_cfg)

Comment 17 Mike Burns 2016-04-12 00:04:30 UTC
After testing with an additional environment file with a containing a section like this:

parameter_defaults:
  ComputeHostnameFormat: '%stackname%-compute-%index%'

the deployment would proceed and maintain the correct setting for nova and hostnames, etc.

This workaround will allow this bug to be moved to an async release since we have a simple workaround for any upgrade issues.

The real fix for this will be to change the default for this setting to compute (from novacompute).

Comment 18 Marius Cornea 2016-04-12 11:08:53 UTC
The provided environment file worked for me. After upgrade:

stack@instack:~>>> nova hypervisor-list
+----+---------------------------------+-------+---------+
| ID | Hypervisor hostname             | State | Status  |
+----+---------------------------------+-------+---------+
| 3  | overcloud-compute-0.localdomain | up    | enabled |
+----+---------------------------------+-------+---------+

stack@instack:~>>> nova service-list
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 3  | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-12T11:07:38.000000 | -               |
| 6  | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-12T11:07:38.000000 | -               |
| 9  | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-12T11:07:38.000000 | -               |
| 12 | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-12T11:07:36.000000 | -               |
| 15 | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-12T11:07:36.000000 | -               |
| 18 | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-12T11:07:36.000000 | -               |
| 21 | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-12T11:07:42.000000 | -               |
| 39 | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-12T11:07:42.000000 | -               |
| 45 | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-12T11:07:42.000000 | -               |
| 69 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | up    | 2016-04-12T11:07:40.000000 | -               |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+

Comment 19 Marios Andreou 2016-04-12 15:40:03 UTC
(In reply to Marius Cornea from comment #18)
> The provided environment file worked for me. After upgrade:
> 
> stack@instack:~>>> nova hypervisor-list
> +----+---------------------------------+-------+---------+
> | ID | Hypervisor hostname             | State | Status  |
> +----+---------------------------------+-------+---------+
> | 3  | overcloud-compute-0.localdomain | up    | enabled |
> +----+---------------------------------+-------+---------+
> 
> stack@instack:~>>> nova service-list
> +----+------------------+------------------------------------+----------+----
> -----+-------+----------------------------+-----------------+
> | Id | Binary           | Host                               | Zone     |
> Status  | State | Updated_at                 | Disabled Reason |
> +----+------------------+------------------------------------+----------+----
> -----+-------+----------------------------+-----------------+
> | 3  | nova-scheduler   | overcloud-controller-0.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:38.000000 | -               |
> | 6  | nova-scheduler   | overcloud-controller-1.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:38.000000 | -               |
> | 9  | nova-scheduler   | overcloud-controller-2.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:38.000000 | -               |
> | 12 | nova-consoleauth | overcloud-controller-1.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:36.000000 | -               |
> | 15 | nova-consoleauth | overcloud-controller-2.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:36.000000 | -               |
> | 18 | nova-consoleauth | overcloud-controller-0.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:36.000000 | -               |
> | 21 | nova-conductor   | overcloud-controller-1.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:42.000000 | -               |
> | 39 | nova-conductor   | overcloud-controller-0.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:42.000000 | -               |
> | 45 | nova-conductor   | overcloud-controller-2.localdomain | internal |
> enabled | up    | 2016-04-12T11:07:42.000000 | -               |
> | 69 | nova-compute     | overcloud-compute-0.localdomain    | nova     |
> enabled | up    | 2016-04-12T11:07:40.000000 | -               |
> +----+------------------+------------------------------------+----------+----
> -----+-------+----------------------------+-----------------+


+1 I also verified today with latest poodle hypervisor/service list as above - deploy and upgrade commands/env as below (I added the "ComputeHostnameFormat: '%stackname%-compute-%index%'" in my network_env.yaml parameter_defaults before running the UPGRADE_INIT step)


"DEPLOY OVERCLOUD 3/1 ha net-iso" openstack overcloud deploy --templates --control-scale 3 --compute-scale 1 --libvirt-type qemu -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e network_env.yaml --ntp-server "0.fedora.pool.ntp.org"

"UPGRADES INIT" openstack overcloud deploy --templates /usr/share/openstack-tripleo-heat-templates -e  /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e network_env.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-init.yaml -e rhos-release-8.yaml

"UPGRADE CONTROLLERS:" openstack overcloud deploy --templates --stack overcloud -e  /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e network_env.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker.yaml

[stack@instack ~]$ upgrade-non-controller.sh --upgrade overcloud-compute-0

"CONVERGE" openstack overcloud deploy --templates --stack overcloud -e  /usr/share/openstack-tripleo-heat-templates/overcloud-resource-registry-puppet.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/puppet-pacemaker.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/network-isolation.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/net-single-nic-with-vlans.yaml -e network_env.yaml -e /usr/share/openstack-tripleo-heat-templates/environments/major-upgrade-pacemaker-converge.yaml

#################
tripleo.sh -- Overcloud pingtest, SUCCESS

Comment 21 Marius Cornea 2016-04-18 16:43:49 UTC
stack@instack:~>>> nova hypervisor-list
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
+----+---------------------------------+-------+---------+
| ID | Hypervisor hostname             | State | Status  |
+----+---------------------------------+-------+---------+
| 3  | overcloud-compute-0.localdomain | up    | enabled |
| 6  | overcloud-compute-1.localdomain | up    | enabled |
+----+---------------------------------+-------+---------+
stack@instack:~>>> nova service-list
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
/usr/lib/python2.7/site-packages/requests/packages/urllib3/connection.py:251: SecurityWarning: Certificate has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
  SecurityWarning
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| Id | Binary           | Host                               | Zone     | Status  | State | Updated_at                 | Disabled Reason |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+
| 3  | nova-scheduler   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-18T16:43:26.000000 | -               |
| 6  | nova-scheduler   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-18T16:43:26.000000 | -               |
| 9  | nova-scheduler   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-18T16:43:21.000000 | -               |
| 12 | nova-consoleauth | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-18T16:43:24.000000 | -               |
| 15 | nova-consoleauth | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-18T16:43:24.000000 | -               |
| 18 | nova-consoleauth | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-18T16:43:25.000000 | -               |
| 21 | nova-conductor   | overcloud-controller-1.localdomain | internal | enabled | up    | 2016-04-18T16:43:27.000000 | -               |
| 45 | nova-conductor   | overcloud-controller-2.localdomain | internal | enabled | up    | 2016-04-18T16:43:27.000000 | -               |
| 54 | nova-conductor   | overcloud-controller-0.localdomain | internal | enabled | up    | 2016-04-18T16:43:27.000000 | -               |
| 69 | nova-compute     | overcloud-compute-0.localdomain    | nova     | enabled | up    | 2016-04-18T16:43:23.000000 | -               |
| 72 | nova-compute     | overcloud-compute-1.localdomain    | nova     | enabled | up    | 2016-04-18T16:43:29.000000 | -               |
+----+------------------+------------------------------------+----------+---------+-------+----------------------------+-----------------+

Comment 26 errata-xmlrpc 2016-04-20 13:04:44 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0653.html


Note You need to log in before you can comment on or make changes to this bug.