Bug 1468174 - Ensure that the undercloud-passwords.conf file is present before doing an undercloud upgrade
Summary: Ensure that the undercloud-passwords.conf file is present before doing an und...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: instack-undercloud
Version: 10.0 (Newton)
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ga
: 12.0 (Pike)
Assignee: James Slagle
QA Contact: Gurenko Alex
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-07-06 07:37 UTC by Chaitanya Shastri
Modified: 2020-08-13 09:38 UTC (History)
9 users (show)

Fixed In Version: instack-undercloud-7.2.1-0.20170821194210.el7ost
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-12-13 21:39:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Launchpad 1702709 0 None None None 2017-07-07 08:23:15 UTC
OpenStack gerrit 481122 0 None MERGED Fail-fast if the undercloud-passwords.conf file is missing 2020-12-10 06:06:23 UTC
Red Hat Product Errata RHEA-2017:3462 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 12.0 Enhancement Advisory 2018-02-16 01:43:25 UTC

Description Chaitanya Shastri 2017-07-06 07:37:15 UTC
Description of problem:
One should not be able to run 'openstack undercloud upgrade' or 'openstack undercloud install' if undercloud-passwords.conf file is missing / renamed from its default location which is '/home/stack' on the undercloud node.

If there are permission issues with undercloud-passwords.conf file, a permission denied error is thrown which is fine, but no validation is made if its missing:

~~~
[stack@instack ~]$ ls -l
total 44
-rw-r--r--. 1 stack stack 13695 Jul  6 06:56 instackenv.json
-rw-------. 1 stack stack   479 Jul  6 09:34 stackrc
drwxrwxr-x. 2 stack stack  4096 Jul  6 09:29 test
-rw-r--r--. 1 stack stack 10673 Jul  6 06:56 undercloud.conf
-r--------. 1 root  root   1650 Jul  6 09:29 undercloud-passwords.conf

[stack@instack ~]$ openstack undercloud upgrade
Logging to /home/stack/.instack/install-undercloud.log
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1207, in install
    _load_config()
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 477, in _load_config
    CONF(conf_params)
  File "/usr/lib/python2.7/site-packages/oslo_config/cfg.py", line 2255, in __call__
    self._namespace._files_permission_denied)
oslo_config.cfg.ConfigFilesPermissionDeniedError: Failed to open some config files: /home/stack/undercloud-passwords.conf
Command 'instack-install-undercloud' returned non-zero exit status 1
~~~

If the undercloud-passwwords.conf file is missing, a new password file is generated which changes the values for all the services and the undercloud upgrade fails eventually and leaves undercloud in a broken state:

~~~
[stack@instack ~]$ ls -l
total 44
-rw-r--r--. 1 stack stack 13695 Jul  6 06:56 instackenv.json
-rw-------. 1 stack stack   479 Jul  6 09:34 stackrc
drwxrwxr-x. 2 stack stack  4096 Jul  6 09:29 test
-rw-r--r--. 1 stack stack 10673 Jul  6 06:56 undercloud.conf
-r--------. 1 root  root   1650 Jul  6 09:29 undercloud-passwords.conf.root

[stack@instack ~]$ openstack undercloud upgrade
Logging to /home/stack/.instack/install-undercloud.log
Checking for a FQDN hostname...
Static hostname detected as instack.localdomain
Transient hostname detected as instack.localdomain
Generated new password for undercloud_db_password
Generated new password for undercloud_admin_token
Generated new password for undercloud_admin_password
Generated new password for undercloud_glance_password
Generated new password for undercloud_heat_encryption_key
Generated new password for undercloud_heat_password
Generated new password for undercloud_neutron_password
Generated new password for undercloud_nova_password
Generated new password for undercloud_ironic_password
Generated new password for undercloud_aodh_password
Generated new password for undercloud_ceilometer_password
Generated new password for undercloud_ceilometer_metering_secret
Generated new password for undercloud_ceilometer_snmpd_password
Generated new password for undercloud_swift_password
Generated new password for undercloud_mistral_password
Generated new password for undercloud_rabbit_cookie
Generated new password for undercloud_rabbit_password
Generated new password for undercloud_rabbit_username
Generated new password for undercloud_heat_stack_domain_admin_password
Generated new password for undercloud_swift_hash_suffix
Generated new password for undercloud_haproxy_stats_password
Generated new password for undercloud_zaqar_password
Generated new password for undercloud_horizon_secret_key
Running yum update
[...]

2017-07-06 09:48:43 - Notice: /Stage[main]/Heat::Deps/Anchor[heat::service::end]: Triggered 'refresh' from 3 events
2017-07-06 09:48:44 - Notice: Finished catalog run in 440.88 seconds
+ rc=6
+ set -e
+ echo 'puppet apply exited with exit code 6'
puppet apply exited with exit code 6
+ '[' 6 '!=' 2 -a 6 '!=' 0 ']'
+ exit 6
[2017-07-06 09:48:49,480] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]

[2017-07-06 09:48:49,480] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1214, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 1003, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 519, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1

[stack@instack ~]$ ls -l
total 48
-rw-r--r--. 1 stack stack 13695 Jul  6 06:56 instackenv.json
-rw-------. 1 stack stack   479 Jul  6 09:34 stackrc
drwxrwxr-x. 2 stack stack  4096 Jul  6 09:29 test
-rw-r--r--. 1 stack stack 10673 Jul  6 06:56 undercloud.conf
-r--------. 1 root  root   1650 Jul  6 09:29 undercloud-passwords.conf.root
~~~

The consequesces of it are that we will not be able to perform minor update and can lead to some other issues which I am unaware of at the moment.

Version-Release number of selected component (if applicable):
RHOSP 10

How reproducible:
Always

Steps to Reproduce:
1.Install undercloud 
2.Install overcloud nodes
3.Move / rename the undercloud-passwords.conf file to something / somewhere else.
4.Run 'openstack undercloud upgrade' command.

Actual results:
'openstack undercloud upgrade' or 'openstack undercloud install' will continue to run even if undercloud-passwords.conf file is renamed / missing.

Expected results:
One should not be able to run 'openstack undercloud upgrade' or 'openstack undercloud install' if undercloud-passwords.conf file is missing / renamed.

Additional info:
Proative solution to this issue is to take undercloud backup before performing undercloud upgrade.

Comment 1 Ben Nemec 2017-07-13 18:57:10 UTC
Upstream review has merged.

Comment 8 Gurenko Alex 2017-12-12 16:13:08 UTC
Verified on build 2017-12-06.2. Getting undercloud update failed with following validation as expected:

instack_undercloud.validator.FailedValidation: The /home/stack/undercloud-passwords.conf file is missing.  This will cause all service passwords to change and break the existing undercloud.
Command 'instack-upgrade-undercloud' returned non-zero exit status 1

Comment 11 errata-xmlrpc 2017-12-13 21:39:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2017:3462


Note You need to log in before you can comment on or make changes to this bug.