Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1619731

Summary: AWS credentials lost during upgrade to 3.11
Product: OpenShift Container Platform Reporter: Mike Fiedler <mifiedle>
Component: Cluster Version OperatorAssignee: Vadim Rutkovsky <vrutkovs>
Status: CLOSED DUPLICATE QA Contact: liujia <jiajliu>
Severity: high Docs Contact:
Priority: unspecified    
Version: 3.11.0CC: aos-bugs, jokerman, mmccomas
Target Milestone: ---   
Target Release: 3.11.0   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-08-21 15:29:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
inventory
none
ansible -vvv log none

Description Mike Fiedler 2018-08-21 15:22:55 UTC
Created attachment 1477615 [details]
inventory

Description of problem:

The upgrade to 3.11 on a cluster with AWS as the cloud provider fails.   The node service on the master fails to restart with the following message in the log:

F0821 15:15:44.944704    8325 server.go:262] failed to run Kubelet: could not init cloud provider "aws": error finding instance i-061c93b1f9408ea6e: "error listing AWS instances: \"NoCredentialProviders: no valid providers in chain. Deprecated.\\n\\tFor verbose messaging see aws.Config.CredentialsChainVerboseErrors\""

At this point of the install, the contents of /etc/sysconfig/atomic-openshift-node has been reset:

OPTIONS=
DEBUG_LOGLEVEL=2
IMAGE_VERSION=v3.11


My inventory (attached) contains env vars for AWS credentials and the env vars are set correctly



Version-Release number of the following components:

root@ip-172-31-38-45: ~ # rpm -q openshift-ansible
openshift-ansible-3.11.0-0.19.0.git.0.ebd1bf9None.noarch
root@ip-172-31-38-45: ~ # rpm -q ansible
ansible-2.6.2-1.el7ae.noarch
root@ip-172-31-38-45: ~ # ansible --version
ansible 2.6.2
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jul 16 2018, 19:52:45) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]


How reproducible: Always with AWS as the cloud provider

Steps to Reproduce:
1.  Install cluster at 3.10.28 with properly configured AWS cloud provider
2.  Upgrade to 3.11.0-0.19.0


Actual results:
Upgrade fails - see attached ansible log and inventory

Comment 1 Mike Fiedler 2018-08-21 15:23:59 UTC
Created attachment 1477616 [details]
ansible -vvv log

Comment 2 Scott Dodson 2018-08-21 15:29:29 UTC

*** This bug has been marked as a duplicate of bug 1618420 ***