Bug 1121705

Summary: nova server-group-list doesn't show members of the group
Product: Red Hat OpenStack Reporter: david.costakos
Component: openstack-novaAssignee: Russell Bryant <rbryant>
Status: CLOSED ERRATA QA Contact: Sean Toner <stoner>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 5.0 (RHEL 7)CC: mlopes, ndipanov, sclewis, yeylon
Target Milestone: rc   
Target Release: 5.0 (RHEL 7)   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-09-02 18:21:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description david.costakos 2014-07-21 16:30:24 UTC
Description of problem:
Tracks with upstream bug https://bugs.launchpad.net/nova/+bug/1298494

There is no way, outside of directly querying the SQL database, to show relationships netween server groups in IceHouse and deployed servers.


Version-Release number of selected component (if applicable):
]# rpm -qa | grep nova
python-nova-2014.1-7.el7ost.noarch
openstack-nova-conductor-2014.1-7.el7ost.noarch
openstack-nova-scheduler-2014.1-7.el7ost.noarch
openstack-nova-api-2014.1-7.el7ost.noarch
openstack-nova-novncproxy-2014.1-7.el7ost.noarch
python-novaclient-2.17.0-2.el7ost.noarch
openstack-nova-common-2014.1-7.el7ost.noarch
openstack-nova-compute-2014.1-7.el7ost.noarch
openstack-nova-cert-2014.1-7.el7ost.noarch
openstack-nova-console-2014.1-7.el7ost.noarch




How reproducible:
Always


Steps to Reproduce:
1.Create a server group

# nova server-group-list
+--------------------------------------+------+--------------------+---------+----------+
| Id                                   | Name | Policies           | Members | Metadata |
+--------------------------------------+------+--------------------+---------+----------+
| ca989bce-ace5-483a-8844-b8320f55ea44 | anti | [u'anti-affinity'] | []      | {}       |
+--------------------------------------+------+--------------------+---------+----------+

2. Deploy one or more servers into that group:
# nova boot --image cirros --nic net-id=76bb7f2d-d37d-4f73-8cce-22eb9b9d0021 --flavor m1.tiny --hint group=ca989bce-ace5-483a-8844-b8320f55ea44 --num-instances 2 grouptest
...
]# nova list | grep grouptest
| a9d09b10-d63a-467d-82c2-0d444dd926d3 | grouptest-a9d09b10-d63a-467d-82c2-0d444dd926d3 | ACTIVE | -          | Running     | private=192.168.75.13                |
| bba64534-f9b5-491f-b6ac-5edc65b2a022 | grouptest-bba64534-f9b5-491f-b6ac-5edc65b2a022 | ACTIVE | -          | Running     | private=192.168.75.12                |
...



3. Verify that the anti-affinity (in this case) policy was applied:
# nova show a9d09b10-d63a-467d-82c2-0d444dd926d3 |egrep 'grouptest|hypervisor_hostname'
| OS-EXT-SRV-ATTR:hypervisor_hostname  | server162.costakos.com                                   |
| name                                 | grouptest-a9d09b10-d63a-467d-82c2-0d444dd926d3           |
# nova show bba64534-f9b5-491f-b6ac-5edc65b2a022 | egrep 'grouptest|hypervisor_hostname'
| OS-EXT-SRV-ATTR:hypervisor_hostname  | server161.costakos.com                                   |
| name                                 | grouptest-bba64534-f9b5-491f-b6ac-5edc65b2a022           |
# egrep 'a9d09b10-d63a-467d-82c2-0d444dd926d3|bba64534-f9b5-491f-b6ac-5edc65b2a022' /var/log/nova/nova-scheduler.log 
2014-07-21 09:19:20.449 12561 INFO nova.scheduler.filter_scheduler [req-6853f940-eb01-482f-af39-9ea927667d56 8e5ae3daa5a346648b8022e868ff1490 6765d72287984b598d104e5041b33a01] Attempting to build 2 instance(s) uuids: [u'bba64534-f9b5-491f-b6ac-5edc65b2a022', u'a9d09b10-d63a-467d-82c2-0d444dd926d3']
2014-07-21 09:19:20.486 12561 INFO nova.scheduler.filter_scheduler [req-6853f940-eb01-482f-af39-9ea927667d56 8e5ae3daa5a346648b8022e868ff1490 6765d72287984b598d104e5041b33a01] Choosing host WeighedHost [host: server161.costakos.com, weight: 1.0] for instance bba64534-f9b5-491f-b6ac-5edc65b2a022
2014-07-21 09:19:20.550 12561 INFO nova.scheduler.filter_scheduler [req-6853f940-eb01-482f-af39-9ea927667d56 8e5ae3daa5a346648b8022e868ff1490 6765d72287984b598d104e5041b33a01] Choosing host WeighedHost [host: server162.costakos.com, weight: 1.0] for instance a9d09b10-d63a-467d-82c2-0d444dd926d3

4. Now do a nova server-group-list and verify that the group has no members.

# nova server-group-list
+--------------------------------------+------+--------------------+---------+----------+
| Id                                   | Name | Policies           | Members | Metadata |
+--------------------------------------+------+--------------------+---------+----------+
| ca989bce-ace5-483a-8844-b8320f55ea44 | anti | [u'anti-affinity'] | []      | {}       |
+--------------------------------------+------+--------------------+---------+----------+
# nova server-group-get ca989bce-ace5-483a-8844-b8320f55ea44
+--------------------------------------+------+--------------------+---------+----------+
| Id                                   | Name | Policies           | Members | Metadata |
+--------------------------------------+------+--------------------+---------+----------+
| ca989bce-ace5-483a-8844-b8320f55ea44 | anti | [u'anti-affinity'] | []      | {}       |
+--------------------------------------+------+--------------------+---------+----------+

5. Verify via REST Nova calls that the group shows no members as well

...
<info>: nova-server-groups Entering method get_group_info ca989bce-ace5-483a-8844-b8320f55ea44
<info>: nova-server-groups Full Response {
  "server_group": {
    "members": [

    ],
    "metadata": {
    },
    "id": "ca989bce-ace5-483a-8844-b8320f55ea44",
    "policies": [
      "anti-affinity"
    ],
    "name": "anti"
  }
}

6. Verify that 'stuff' is in the nova/instance_group_members database:

MariaDB [nova]> select * from instance_group_member;
+---------------------+------------+------------+---------+----+--------------------------------------+----------+
| created_at          | updated_at | deleted_at | deleted | id | instance_id                          | group_id |
+---------------------+------------+------------+---------+----+--------------------------------------+----------+
| 2014-07-21 03:56:15 | NULL       | NULL       |       0 |  1 | 680f0e2d-71a3-4960-8bb5-78809e1bbaf9 |        1 |
| 2014-07-21 03:56:15 | NULL       | NULL       |       0 |  2 | 62ebb492-f3c5-467f-9030-9fd652f2f9ec |        1 |
| 2014-07-21 04:55:42 | NULL       | NULL       |       0 |  3 | 84c53e73-c047-4231-9700-3add2b4faac8 |        1 |
| 2014-07-21 04:55:42 | NULL       | NULL       |       0 |  4 | 99ea1e55-4380-4725-875e-7b1e0641ec3a |        1 |
| 2014-07-21 16:19:20 | NULL       | NULL       |       0 |  5 | bba64534-f9b5-491f-b6ac-5edc65b2a022 |        1 |
| 2014-07-21 16:19:20 | NULL       | NULL       |       0 |  6 | a9d09b10-d63a-467d-82c2-0d444dd926d3 |        1 |
+---------------------+------------+------------+---------+----+--------------------------------------+----------+



Actual results:

nova client calls to server-group-list/server-group-get show no group members
api calls to os-server-groups REST URLs return no members
database table entry exists in instance_group_member, but nothing is returned

Expected results:

nova returns the members of a particular nova server-group

Additional info:

Makes auditing, evacuation, live migration difficult for operators if they are unable to audit which instances are members of a particular server group with a policy applied.

Comment 2 Russell Bryant 2014-07-30 15:20:29 UTC
Based on the fix that merged upstream (and was backported to stable/icehouse), it looks like our current versions should have the fix for this since we recently rebased.

Comment 4 david.costakos 2014-08-01 17:38:32 UTC
This is confirmed to work for me after updating.

[root@server161 ~(keystone_admin)]# nova server-group-list
+--------------------------------------+------+--------------------+------------------------------------------------------------------------------------+----------+
| Id                                   | Name | Policies           | Members                                                                            | Metadata |
+--------------------------------------+------+--------------------+------------------------------------------------------------------------------------+----------+
| ca989bce-ace5-483a-8844-b8320f55ea44 | anti | [u'anti-affinity'] | [u'0cd08699-0028-4e68-937b-decc5e22cf52', u'69d48250-9d36-4758-b50e-d5b3ad9593c4'] | {}       |
+--------------------------------------+------+--------------------+------------------------------------------------------------------------------------+----------+

Comment 5 Sean Toner 2014-08-01 18:38:36 UTC
Step 1 should have included the means to create the server group:

from upstream: https://bugs.launchpad.net/nova/+bug/1298494, this is indicated as:

nova server-group-create --policy anti-affinity antiaffinitygroup

Comment 6 Sean Toner 2014-08-01 18:58:17 UTC
Verified for me as well

+--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+
| Id                                   | Name              | Policies           | Members                                                                            | Metadata |
+--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+
| 41014a2d-9272-44e1-8768-2550d44ba800 | antiaffinitygroup | [u'anti-affinity'] | [u'2cbfa234-5ca1-4ed9-a184-cfbc2d6c6024', u'1b460c87-2787-46f1-a665-013fc15fab1c'] | {}       |
+--------------------------------------+-------------------+--------------------+------------------------------------------------------------------------------------+----------+

Comment 12 errata-xmlrpc 2014-09-02 18:21:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1127.html