Bug 1353354 - Upgrade from 3.1 to 3.2 overwrites AWS variables in /etc/sysconfig/atomic-openshift-master-*
Summary: Upgrade from 3.1 to 3.2 overwrites AWS variables in /etc/sysconfig/atomic-ope...
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Cluster Version Operator
Version: 3.2.0
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: ---
Assignee: Devan Goodwin
QA Contact: Anping Li
Depends On:
Blocks: 1370641
TreeView+ depends on / blocked
Reported: 2016-07-06 21:29 UTC by Steven Walter
Modified: 2017-03-08 18:26 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Previous versions allowed the user to specify AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in their /etc/sysconfig/ files for OpenShift services. During upgrade these files are updated according to a template, if the user had not yet switched to using the new cloud provider framework their pre-existing AWS variables would be overwritten. The upgrade process has been modified to preserve these variables if they are present during upgrade, and a cloud provider is not configured.
Clone Of:
: 1370641 (view as bug list)
Last Closed: 2016-09-27 09:39:20 UTC
Target Upstream Version:

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2016:1933 normal SHIPPED_LIVE Red Hat OpenShift Container Platform 3.3 Release Advisory 2016-09-27 13:24:36 UTC

Description Steven Walter 2016-07-06 21:29:41 UTC
Description of problem:
OpenShift 3.1 installed on AWS. Masters are on Atomic Host 7.2.4; Nodes are on RHEL. When upgrading to 3.2, the variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY disappear in /etc/sysconfig/atomic-openshift-master-* however they remain on the nodes in /etc/sysconfig/atomic-openshift-node

Version-Release number of selected component (if applicable):
OSE 3.1 -> 3.2

How reproducible:
Not yet reproduced

Actual results:
Expect the variables configured for AWS to remain

Expected results:
Files are possibly overwritten entirely, else some parts are removed.

Additional info:
Seems similar to https://bugzilla.redhat.com/show_bug.cgi?id=1345804 although with a different component

Comment 2 Alexander Koksharov 2016-07-08 13:37:21 UTC
Additional info:
After installation we also noticed that mounting of the /etc/origin/cloudprovider directory was not present in /etc/systemd/system/atomic-openshift-node.service
We had to add the following parameter manually: -v /etc/origin/cloudprovider:/etc/origin/cloudprovider

Also new configuration has not been added to master-config.yaml, we thought that specifying the following in the hosts files would create proper entries:

Comment 3 Steven Walter 2016-08-15 14:07:13 UTC
From the customer:

During the upgrade from to we faced the issue with missing AWS_ACCESS_KEY_ID and  AWS_SECRET_ACCESS_KEY in 

We have learned that they are missing when hosts file contains the variables:

We modified first the ansible templates
- atomic-openshift-master-api.j2
- atomic-openshift-master-controllers.j2

and removed the "if" block
{% if 'cloudprovider' in openshift and 'aws' in openshift.cloudprovider and 'kind' in openshift.cloudprovider and openshift.cloudprovider.kind == 'aws' and 'access_key' in openshift.cloudprovider.aws and 'secret_key' in openshift.cloudprovider.aws %}
AWS_ACCESS_KEY_ID={{ openshift.cloudprovider.aws.access_key }}
AWS_SECRET_ACCESS_KEY={{ openshift.cloudprovider.aws.secret_key }}
{% endif %}

leaving only:
AWS_ACCESS_KEY_ID={{ openshift.cloudprovider.aws.access_key }}
AWS_SECRET_ACCESS_KEY={{ openshift.cloudprovider.aws.secret_key }}

In this case the ansible upgrade script failed with:
fatal: [master01.pink.eu-central-1.aws.openpaas.axa-cloud.com]: FAILED! => {"changed": false, "failed": true, "msg": "AnsibleUndefinedVariable: 'dict object' has no attribute 'cloudprovider'"}

After hardcoding the values, the upgrade procedure worked.

Comment 6 Anping Li 2016-08-29 06:49:50 UTC
The keys can be found after upgraded so move bug to verified.

Comment 8 errata-xmlrpc 2016-09-27 09:39:20 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.


Note You need to log in before you can comment on or make changes to this bug.