Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1722201

Summary: Sometimes new instances are scheduled only on 1 compute node while second remains empty
Product: Red Hat OpenStack Reporter: Roman Safronov <rsafrono>
Component: openstack-novaAssignee: smooney
Status: CLOSED DUPLICATE QA Contact: OSP DFG:Compute <osp-dfg-compute>
Severity: high Docs Contact:
Priority: unspecified    
Version: 15.0 (Stein)CC: dasmith, eglynn, jhakimra, kchamart, mbooth, sbauza, sgordon, smooney, vromanso
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-06-21 12:09:05 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Roman Safronov 2019-06-19 16:37:25 UTC
Description of problem:
While environment has 2 compute nodes all instances for some reason put on the same node and the second node remains unused.

Version-Release number of selected component (if applicable):
15.0-RHEL-8/RHOS_TRUNK-15.0-RHEL-8-20190527.n.0

How reproducible:
From time to time

Steps to Reproduce:
1. Create 2 networks, 2 subnets, router, connect router to external network. connect subnets to the router, create security group and keypair
2. Create instance on first network
3. Create instance on second network
4. Create one more instance on first network
5. Create one more instance on second network

Actual results:
All 4 instances are running on compute-1
0 instances runs on compute-0

(overcloud) [stack@undercloud-0 my]$ openstack server list --long
+--------------------------------------+-------------------------+--------+------------+-------------+----------------------------------------+---------------------+--------------------------------------+---------------+--------------------------------------+-------------------+-----------------------+------------+
| ID                                   | Name                    | Status | Task State | Power State | Networks                               | Image Name          | Image ID                             | Flavor Name   | Flavor ID                            | Availability Zone | Host                  | Properties |
+--------------------------------------+-------------------------+--------+------------+-------------+----------------------------------------+---------------------+--------------------------------------+---------------+--------------------------------------+-------------------+-----------------------+------------+
| 6fa97369-72c1-4e1b-83bf-761fa1a42f15 | cirros40_admin_vm2_net2 | ACTIVE | None       | Running     | admin_int_net_2=10.0.2.236             | cirros-0.4.0-x86_64 | b3591eda-c43a-4668-a2d7-065adf472978 | m1.nano       | 583b9886-3e64-4f60-be9e-9b76120144bc | nova              | compute-1.localdomain |            |
| 724373ca-30fb-4aea-a343-511d3168876e | cirros40_admin_vm2_net1 | ACTIVE | None       | Running     | admin_int_net_1=10.0.1.65, 10.0.0.216  | cirros-0.4.0-x86_64 | b3591eda-c43a-4668-a2d7-065adf472978 | m1.nano       | 583b9886-3e64-4f60-be9e-9b76120144bc | nova              | compute-1.localdomain |            |
| 070b8dba-4327-4f53-9682-7f36ccd04647 | cirros40_admin_vm1_net2 | ACTIVE | None       | Running     | admin_int_net_2=10.0.2.206, 10.0.0.210 | cirros40            | 770f527d-e530-4402-82d2-a72099497ea4 | cirros_flavor | 4113947b-6003-4fe5-9307-69fb0b70a0e0 | nova              | compute-1.localdomain |            |
| 1359d8e0-41c2-41c2-a9fd-a14d379d6dca | cirros40_admin_vm1_net1 | ACTIVE | None       | Running     | admin_int_net_1=10.0.1.94, 10.0.0.225  | cirros40            | 770f527d-e530-4402-82d2-a72099497ea4 | cirros_flavor | 4113947b-6003-4fe5-9307-69fb0b70a0e0 | nova              | compute-1.localdomain |            |
+--------------------------------------+-------------------------+--------+------------+-------------+----------------------------------------+---------------------+--------------------------------------+---------------+--------------------------------------+-------------------+-----------------------+------------+



(overcloud) [stack@undercloud-0 my]$ openstack compute service list
+----+------------------+--------------------------+----------+---------+-------+----------------------------+
| ID | Binary           | Host                     | Zone     | Status  | State | Updated At                 |
+----+------------------+--------------------------+----------+---------+-------+----------------------------+
|  5 | nova-conductor   | controller-0.localdomain | internal | enabled | up    | 2019-06-19T16:24:39.000000 |
| 11 | nova-conductor   | controller-1.localdomain | internal | enabled | up    | 2019-06-19T16:24:42.000000 |
| 14 | nova-consoleauth | controller-0.localdomain | internal | enabled | up    | 2019-06-19T16:24:35.000000 |
| 17 | nova-conductor   | controller-2.localdomain | internal | enabled | up    | 2019-06-19T16:24:40.000000 |
| 23 | nova-consoleauth | controller-1.localdomain | internal | enabled | up    | 2019-06-19T16:24:39.000000 |
| 26 | nova-consoleauth | controller-2.localdomain | internal | enabled | up    | 2019-06-19T16:24:38.000000 |
| 29 | nova-scheduler   | controller-0.localdomain | internal | enabled | up    | 2019-06-19T16:24:37.000000 |
| 38 | nova-scheduler   | controller-1.localdomain | internal | enabled | up    | 2019-06-19T16:24:39.000000 |
| 50 | nova-scheduler   | controller-2.localdomain | internal | enabled | up    | 2019-06-19T16:24:35.000000 |
| 62 | nova-compute     | compute-0.localdomain    | nova     | enabled | up    | 2019-06-19T16:24:33.000000 |
| 65 | nova-compute     | compute-1.localdomain    | nova     | enabled | up    | 2019-06-19T16:24:42.000000 |
+----+------------------+--------------------------+----------+---------+-------+----------------------------+


Expected results:
Amount of instances on both nodes are balanced, i.e. in case 4 instances I would expect 2 instances on compute-0 and 2 on compute-1

Additional info:

Both compute nodes are identical VMs with 32GB memory.

(overcloud) [stack@undercloud-0 my]$ com0
Last login: Wed Jun 19 15:16:09 2019 from 192.168.24.1
[heat-admin@compute-0 ~]$ free
              total        used        free      shared  buff/cache   available
Mem:       32771908     3849748    27136812      192736     1785348    28228340
Swap:             0           0           0
[heat-admin@compute-0 ~]$ exit
logout
Connection to compute-0 closed.
(overcloud) [stack@undercloud-0 my]$ com1
Last login: Wed Jun 19 16:19:03 2019 from 192.168.24.1
[heat-admin@compute-1 ~]$ free
              total        used        free      shared  buff/cache   available
Mem:       32771908     4449384    26464220      193272     1858304    27628604
Swap:             0           0           0

Comment 1 Roman Safronov 2019-06-19 16:47:27 UTC
logs location  http://rhos-release.virt.bos.redhat.com/log/bz1722201

Comment 2 Roman Safronov 2019-06-21 07:57:16 UTC
Note: after rebooting compute nodes new instances were scheduled properly.

Comment 3 Matthew Booth 2019-06-21 12:09:05 UTC

*** This bug has been marked as a duplicate of bug 1701334 ***