Bug 1383805

Summary:	[RFE] Dynamic Ansible inventory of overcloud nodes
Product:	Red Hat OpenStack	Reporter:	Mark McLoughlin <markmc>
Component:	openstack-tripleo-validations	Assignee:	Steven Hardy <shardy>
Status:	CLOSED ERRATA	QA Contact:	Amit Ugol <augol>
Severity:	unspecified	Docs Contact:
Priority:	low
Version:	11.0 (Ocata)	CC:	augol, dbecker, emacchi, fzdarsky, jcoufal, jjoyce, jschluet, jtaleric, lmadsen, mburns, mcornea, morazi, nlevinki, owalsh, rhel-osp-director-maint, rkharwar, sclewis, shardy, slinaber, tvignaud
Target Milestone:	Upstream M2	Keywords:	FutureFeature, Triaged
Target Release:	11.0 (Ocata)
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	openstack-tripleo-validations-5.1.0-5.el7ost	Doc Type:	If docs needed, set a value
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2017-05-17 19:34:42 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:
Bug Depends On:	1276482
Bug Blocks:	1426478

Description Mark McLoughlin 2016-10-11 19:49:12 UTC

In user story form, this RFE might be:

As a cloud operator, I want to run my Ansible playbooks from my own machine against my OSP-director managed overcloud controller, compute, and storage nodes so that I can debug and inspect my OpenStack environment.


The background is:

Ansible has a hosts inventory - this is where you list your hosts, and groups of hosts, so that you can run playbooks against them:

http://docs.ansible.com/ansible/intro_inventory.html

It also has the notion of a dynamic inventory:

http://docs.ansible.com/ansible/intro_dynamic_inventory.html

With dynamic inventory, rather than store your hosts in a text file, Ansible looks to some other system for information about hosts. And yes, there's an example there for an OpenStack dynamic inventory.

The idea is that OSP director users should be very easily to set up a dynamic inventory which talks to the undercloud and which would allow them to easily run ad-hoc playbooks against their overcloud nodes. They should first be able to easily distribute their SSH key to those nodes, also.

To be clear - the use case isn't about using Ansible to install and configure software on overcloud nodes.


There was one previous effort along these lines by Graeme:

https://review.openstack.org/277688

and I remember a brief "ansible on the undercloud" thread on this (the context was os-log-merger), which then spawn this thread:

http://lists.openstack.org/pipermail/openstack-dev/2016-February/087426.html

and some follow-on discussions in Austin. A key point of clarification for me is that the requirement should not be "ansible on the undercloud", but instead "ansible using the undercloud for dynamic inventory".

Comment 1 Frank Zdarsky 2016-11-15 15:06:49 UTC

Other related efforts:

https://github.com/openstack/tripleo-validations/blob/master/scripts/tripleo-ansible-inventory (dynamic inventory used by TripleO validations)

https://raw.githubusercontent.com/ansible/ansible/devel/contrib/inventory/openstack.py

Comment 2 Leif Madsen 2016-11-15 15:10:11 UTC

Also I had to read this a couple times to truly understand the ask. If I understand correctly, this is the primarily quote:

Quote:  

    A key point of clarification for me is that the requirement should not be
    "ansible on the undercloud", but instead "ansible using the undercloud for 
    dynamic inventory".


Basically, this request is to connect to the undercloud as the backing store from a laptop in order to provide post-deployment configuration to an overcloud.

undercloud
    ^
    |
  laptop  --> overcloud

Comment 4 Russell Bryant 2016-12-22 02:09:04 UTC

I've been doing some performance testing using Browbeat [1] and came across something similar to this.  Browbeat includes a script, generate_tripleo_hostfile.sh [2], which you run after deploying an overcloud.  It generates an ssh-config file and ansible hosts file.  You can then use that to run ansible against the overcloud.

The really handy thing is that this can be used from your laptop.  For example, I have an undercloud machine and 12 overcloud machines in a lab.  From my laptop, I just ran:

  $ ./generate_tripleo_hostfile.sh b09-h01-r620.rdu.openstack.engineering.redhat.com

and now I can easily run ansible against my overcloud from my laptop.  The ssh config it generates makes it so ansible will ssh through the undercloud machine to get to the overcloud machines on the management network.

For example, to run something on all of the controllers:

  $ ansible controller -i hosts -a "uname -r"
  overcloud-controller-2 | SUCCESS | rc=0 >>
  3.10.0-327.36.3.el7.x86_64

  overcloud-controller-1 | SUCCESS | rc=0 >>
  3.10.0-327.36.3.el7.x86_64

  overcloud-controller-0 | SUCCESS | rc=0 >>
  3.10.0-327.36.3.el7.x86_64

Similarly, I can run a command against all of the compute nodes using "ansible compute".

[1] https://github.com/openstack/browbeat
[2]  https://github.com/openstack/browbeat/blob/master/ansible/generate_tripleo_hostfile.sh

Comment 5 Joe Talerico 2016-12-22 12:56:02 UTC

Thanks Russell! We have started to talk about moving this bash script to python, anyone interested in helping out with this effort[1][2]

[1] https://blueprints.launchpad.net/openstack-browbeat/+spec/ansible-hosts
[2] https://etherpad.openstack.org/p/browbeat-ansible-hosts

We needed a solution a bit more abstracted so other clouds could take advantage of the host generation.

Comment 8 Steven Hardy 2017-01-04 17:28:11 UTC

$ rpm -qpl openstack-tripleo-validations-5.1.0-5.el7ost.noarch.rpm | grep inventory
/usr/bin/tripleo-ansible-inventory

Marking this as fixed in version openstack-tripleo-validations-5.1.0-5.el7ost

Comment 10 Marcos Garcia 2017-01-30 20:59:10 UTC

Just to confirm, this works OK on OSP10 (example executing a shell command on all overcloud nodes)

ansible overcloud -i /bin/tripleo-ansible-inventory  -u heat-admin --ssh-extra-args='-o StrictHostKeyChecking=no' -m shell -a "ip a | grep inet\ “ > output.txt

There are other hostgroups available such as compute, storage, controller, etc

Comment 14 errata-xmlrpc 2017-05-17 19:34:42 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:1245