Bug 1876815 - Installer uses the environment variable OS_CLOUD for manifest generation despite explicit prompt
Summary: Installer uses the environment variable OS_CLOUD for manifest generation desp...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 4.6
Hardware: Unspecified
OS: Unspecified
low
low
Target Milestone: ---
: 4.7.0
Assignee: Mike Fedosin
QA Contact: weiwei jiang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-09-08 08:53 UTC by Pierre Prinetti
Modified: 2021-10-20 09:29 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-02-24 15:17:26 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift installer pull 4177 0 None closed Bug 1876815: unset OS_CLOUD during config generator 2020-10-09 02:41:23 UTC
Github openshift installer pull 4231 0 None closed Bug 1876815: OpenStack: unset OS_CLOUD 2020-10-10 02:22:02 UTC
Red Hat Product Errata RHSA-2020:5633 0 None None None 2021-02-24 15:17:46 UTC

Description Pierre Prinetti 2020-09-08 08:53:49 UTC
Description of problem:

If the environment variable OS_CLOUD is set while running `create manifests`, the Installer will use that for generating assets no matter what "Cloud" has been selected in the interactive prompt.

Steps to Reproduce:
$ export OS_CLOUD=blablabla
$ openshift-install create manifests
? SSH Public Key /home/user1/.ssh/id_rsa.pub
? Platform openstack
? Cloud moc
FATAL failed to fetch Master Machines: failed to fetch dependency of "Master Machines": failed to fetch dependency of "Cluster ID": failed to fetch dependency of "Install Config": failed to fetch dependency of "Base Domain": failed to generate asset "Platform": cloud blablabla does not exist in clouds.yaml


Expected results:
Given that a Cloud parameter has been explicitly passed, the value of the OS_CLOUD environment variable should be ignored/overridden.

Comment 4 weiwei jiang 2020-09-28 04:47:07 UTC
Verified with 4.6.0-0.nightly-2020-09-27-075304

$ export os_clouds=non-exist
$ ./openshift-install-4.6 create manifests --dir .
? SSH Public Key /home/stack/.ssh/id_rsa.pub
? Platform openstack
? Cloud shiftstack
? ExternalNetwork public
? APIFloatingIPAddress 10.46.43.177
? FlavorName m4.xlarge
? Base Domain example.com
? Cluster Name wjtest
? Pull Secret [? for help] *************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
INFO Credentials loaded from file "/home/stack/wjiang/clouds.yaml" 
INFO Manifests created in: manifests and openshift

Comment 5 weiwei jiang 2020-09-28 04:51:02 UTC
Ignore above.

verified with 4.6.0-0.nightly-2020-09-27-075304

$ export OS_CLOUD=non-exist
$ ./openshift-install-4.6 create manifests --dir .
? SSH Public Key /home/stack/.ssh/id_rsa.pub
? Platform openstack
? Cloud shiftstack
? ExternalNetwork public
? APIFloatingIPAddress 10.46.43.177
? FlavorName m4.xlarge
? Base Domain example.com
? Cluster Name wjtest
? Pull Secret [? for help] *************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
INFO Credentials loaded from file "/home/stack/wjiang/clouds.yaml" 
INFO Manifests created in: manifests and openshift

Comment 6 weiwei jiang 2020-09-28 06:27:54 UTC
Failed with destroy the cluster

$ export OS_CLOUD=non-exist
$ ./openshift-install-4.6 destroy cluster --dir . --log-level debug
DEBUG OpenShift Installer 4.6.0-0.nightly-2020-09-27-075304 
DEBUG Built from commit 527e3beb3609be50551b4019282ece14cce1ecce 
DEBUG Removing interfaces from custom router       
DEBUG Deleting openstack networks                  
DEBUG Deleting openstack subnets                   
DEBUG Deleting openstack containers                
DEBUG Deleting openstack base image                
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack networks          
DEBUG Deleting openstack routers                   
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack containers        
ERROR cloud non-exist does not exist in clouds.yaml 
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack base image        
DEBUG Deleting openstack servers                   
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack routers           
DEBUG Exiting removal of interfaces from custom router 
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack subnets           
DEBUG Deleting OpenStack volumes                   
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting OpenStack volumes           
DEBUG Deleting openstack security-groups           
DEBUG Deleting openstack floating ips              
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack security-groups   
DEBUG Deleting openstack server groups             
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack floating ips      
DEBUG Deleting openstack load balancers            
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack server groups     
DEBUG Deleting openstack trunks                    
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack load balancers    
DEBUG Deleting openstack ports                     
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack trunks            
DEBUG Deleting openstack subnet-pools              
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack ports             
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack subnet-pools      
ERROR cloud non-exist does not exist in clouds.yaml 
DEBUG Exiting deleting openstack servers

Comment 7 weiwei jiang 2020-09-28 06:33:11 UTC
And After I create the manifests with interactive way, then I try to create the cluster, also got error:

$ ./openshift-install-4.6 create cluster --dir . 
WARNING Discarding the Terraform Variables that was provided in the target directory because its dependencies are dirty and it needs to be regenerated 
WARNING Found override for release image. Please be warned, this is not advised 
INFO Credentials loaded from file "/home/stack/wjiang/clouds.yaml" 
FATAL failed to fetch Terraform Variables: failed to generate asset "Terraform Variables": failed to get cloud config for openstack: cloud non-exist does not exist in clouds.yaml

Comment 8 Mike Fedosin 2020-10-01 10:58:43 UTC
hello weiwei!

-----
with regards to:

$ export OS_CLOUD=non-exist
$ ./openshift-install-4.6 destroy cluster --dir . --log-level debug

Setting the env variable is the only way to tell the installer what `cloud` to use. If you set a wrong value there, you see `ERROR cloud non-exist does not exist in clouds.yaml`. And this is the expected behavior.
On the other hand I agree that it's not good that it got stuck in the infinite loop. This situation is partially handled in another PR: https://github.com/openshift/installer/pull/3350 where the execution will stop after 25 immediate attempts.
The correct solution here would be to check the credentials before we start deleting the cluster, and fail immediately if they are invalid.

But this is another issue and it's not related to `Installer uses the environment variable OS_CLOUD for manifest generation despite explicit prompt`. Probably we can create another bz to track it?
-----


For the second part I confirm that the original fix didn't solve the problem completely, and we have to unset this variable in several other places. This PR should handle it: https://github.com/openshift/installer/pull/4231

Comment 11 weiwei jiang 2020-10-10 02:23:57 UTC
(In reply to Mike Fedosin from comment #8)
> hello weiwei!
> 
> -----
> with regards to:
> 
> $ export OS_CLOUD=non-exist
> $ ./openshift-install-4.6 destroy cluster --dir . --log-level debug
> 
> Setting the env variable is the only way to tell the installer what `cloud`
> to use. If you set a wrong value there, you see `ERROR cloud non-exist does
> not exist in clouds.yaml`. And this is the expected behavior.
> On the other hand I agree that it's not good that it got stuck in the
> infinite loop. This situation is partially handled in another PR:
> https://github.com/openshift/installer/pull/3350 where the execution will
> stop after 25 immediate attempts.
> The correct solution here would be to check the credentials before we start
> deleting the cluster, and fail immediately if they are invalid.
> 
> But this is another issue and it's not related to `Installer uses the
> environment variable OS_CLOUD for manifest generation despite explicit
> prompt`. Probably we can create another bz to track it?
> -----
> 
> 
> For the second part I confirm that the original fix didn't solve the problem
> completely, and we have to unset this variable in several other places. This
> PR should handle it: https://github.com/openshift/installer/pull/4231


Thanks for the explaination, then move to verified.

Comment 14 errata-xmlrpc 2021-02-24 15:17:26 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: OpenShift Container Platform 4.7.0 security, bug fix, and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2020:5633

Comment 15 Pierre Prinetti 2021-10-20 09:29:26 UTC
For the record, here are a couple follow-ups:

Bug 1813949 - openshift-install doesn't use env variables for OS_* for some of API endpoints

Bug 2015837 - OS_CLOUD overwrites install-config's platform.openstack.cloud


Note You need to log in before you can comment on or make changes to this bug.