Bug 1533271

Summary: Running the mistral workflow to rotate Fernet decryption keys in the overcloud Fails
Product: Red Hat OpenStack Reporter: Ken Savich <ksavich>
Component: openstack-tripleo-commonAssignee: Juan Antonio Osorio <josorior>
Status: CLOSED ERRATA QA Contact: Jeremy Agee <jagee>
Severity: high Docs Contact:
Priority: high    
Version: 12.0 (Pike)CC: david.costakos, dsorrent, emacchi, hrybacki, jamsmith, josorior, ksavich, mburns, nkinder, pkesavar, rhel-osp-director-maint, slinaber
Target Milestone: z3Keywords: Triaged, ZStream
Target Release: 12.0 (Pike)   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: openstack-tripleo-common-7.6.9-4.el7ost Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-20 12:58:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ken Savich 2018-01-10 23:00:44 UTC
Description of problem:

NOTE: my stack is named "sweatpants" here

Running the tripleo.fernet_keys.rotate_fernet_keys workflow from the undercloud as such:

openstack workflow execution create tripleo.fernet_keys.v1.rotate_fernet_keys '{"container": "sweatpants"}'

Will fail if your cloud is not named "overcloud"

Version-Release number of selected component (if applicable):


How reproducible:

100%

Steps to Reproduce:
1.Deploy a cloud named something other than "overcloud" (such as "sweatpants")
2.try rotate your fernet decrypt keys on the controllers with 

openstack workflow execution create tripleo.fernet_keys.v1.rotate_fernet_keys '{"container": "sweatpants"}'

3. look at the output of the workflow after a minute or so

openstack workflow execution output show <workflow id>


Actual results:

Look at your ferent keys in the contain from the controller node, notice that they haven't changed:

docker exec -ti keystone ls -l /etc/keystone/fernet-keys
'
Check the actual workflow output, you'll see something like the following:

{
    "status": "SUCCESS",
    "message": {
        "stderr": "\nPLAY [keystone] ****************************************************************\nskipping: no hosts matched\n\nPLAY RECAP *********************************************************************\n\n",
        "stdout": " [WARNING]: Could not match supplied host pattern, ignoring: keystone\n"
    }
}

The only way to get this working, is the following:

openstack workflow execution create tripleo.fernet_keys.v1.rotate_fernet_keys '{ "container": "sweatpants", "ansible_extra_env_variables": { "TRIPLEO_PLAN_NAME": "sweatpants", "ANSIBLE_HOST_KEY_CHECKING": "False" }}'

Note, additionally, that we have to pass the ANSIBLE_HOST_KEY_CHECKING variable because this gets overwritten by the workflow and isn't picked up if the plan name is the only variable passed into the workflow.

Expected results:

Expect to have rotated keys on all controller nodes

Additional info:

Comment 1 Nathan Kinder 2018-02-05 17:41:14 UTC
This has merged upstream:

  https://review.openstack.org/#/c/532808/

Comment 2 Harry Rybacki 2018-04-25 17:46:13 UTC
Patch fixed-in: openstack-tripleo-common-7.6.9-4.el7ost by rebase.

Comment 10 errata-xmlrpc 2018-08-20 12:58:39 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2331