Bug 1292182 - Error: nova_admin_password must be set. at /etc/puppet/modules/neutron/manifests/server/notifications.pp:80
Error: nova_admin_password must be set. at /etc/puppet/modules/neutron/manife...
Status: CLOSED ERRATA
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-puppet-modules (Show other bugs)
8.0 (Liberty)
Unspecified Unspecified
urgent Severity high
: ga
: 8.0 (Liberty)
Assigned To: Jason Guiditta
Prasanth Anbalagan
: Automation, AutomationBlocker, Triaged
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-12-16 11:47 EST by wes hayutin
Modified: 2016-04-07 17:19 EDT (History)
13 users (show)

See Also:
Fixed In Version: openstack-puppet-modules-7.0.3-1.el7ost
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-07 17:19:22 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
Overcloud-Deployment-Error.txt (8.82 KB, text/plain)
2016-01-05 17:32 EST, Donny Davis
no flags Details
QA logs for undercloud installation (352.12 KB, text/plain)
2016-01-25 15:11 EST, Prasanth Anbalagan
no flags Details
QA Verification logs for undercloud Installation (472.04 KB, text/plain)
2016-01-26 16:34 EST, Prasanth Anbalagan
no flags Details

  None (edit)
Description wes hayutin 2015-12-16 11:47:30 EST
Description of problem:

instack undercloud fails to build w/ the following error.

class ::nova::vncproxy has not been evaluated[0m
[1;31mError: nova_admin_password must be set. at /etc/puppet/modules/neutron/manifests/server/notifications.pp:80 on node instack[0m
[1;31mError: nova_admin_password must be set. at /etc/puppet/modules/neutron/manifests/server/notifications.pp:80 on node instack[0m
+ rc=1
+ set -e
+ echo 'puppet apply exited with exit code 1'
puppet apply exited with exit code 1
+ '[' 1 '!=' 2 -a 1 '!=' 0 ']'
+ exit 1
[2015-12-15 12:18:42,846] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]
Comment 2 Jason Guiditta 2015-12-17 12:05:45 EST
This error no longer exists in upstream code, so the next build should correct the problem.
Comment 5 Donny Davis 2016-01-05 17:32 EST
Created attachment 1111963 [details]
Overcloud-Deployment-Error.txt
Comment 6 Donny Davis 2016-01-05 17:33:22 EST
This bug also effects OSP 7 overcloud deployment, please see attached.
Comment 7 Jason Guiditta 2016-01-07 10:22:54 EST
(In reply to Donny Davis from comment #6)
> This bug also effects OSP 7 overcloud deployment, please see attached.
If that is the case, can you please clone this bug to OSP7?  I do see the line in question in the puppet manifest from that version, but there also does not appear to have been any changes to this component recently, which makes it rather baffling that this would not have always been a problem.  Regardless, a repeat of this problem in an older OSP needs to be tracked separately to avoid confusion
Comment 8 Qasim Sarfraz 2016-01-07 13:36:25 EST
I am hitting this issue on osp7 deployment. Is there a workaround for this?
Comment 9 Donny Davis 2016-01-07 18:36:11 EST
I will open a ticket for OSP 7 if the updated images produce the same result tomorrow. 

If the issue is not using the latest image, then I will update this ticket with the result. 

In that case, please disregard my comments in this bug.
Comment 11 Donny Davis 2016-01-12 14:46:26 EST
My comments are not relevant to this bug, my issues were related to not using the most current deployment images.
Comment 12 Jason Guiditta 2016-01-12 15:05:50 EST
(In reply to Donny Davis from comment #11)
> My comments are not relevant to this bug, my issues were related to not
> using the most current deployment images.

Thank you for the update Donny. Qasim, is there any chance you have the same situation?  If not, my comment #7 applies here as well.
Comment 13 Qasim Sarfraz 2016-01-12 15:14:08 EST
(In reply to Jason Guiditta from comment #12)
> (In reply to Donny Davis from comment #11)
> > My comments are not relevant to this bug, my issues were related to not
> > using the most current deployment images.
> 
> Thank you for the update Donny. Qasim, is there any chance you have the same
> situation?  If not, my comment #7 applies here as well.

I haven't tested with latest images (OSP 7.2) but I have the same situation i.e I have a RHEL_OSP 7.2 director but overcloud image of 7.1. This seems like an issue.

Also, Shouldn't this be part of documentation that if a user upgrades the RHEL-OSP director packages. Overcloud images should be upgraded as well else things will break.
Comment 14 Jason Guiditta 2016-01-19 08:48:09 EST
Wes, can you tell Prasanth how to replicate this?  I was merely verifying change of existing code versus what was seen in testing, not actually installing ospd.
Comment 15 wes hayutin 2016-01-19 09:15:49 EST
Not punting on this, but following the doc up until 

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux_OpenStack_Platform/7/html-single/Director_Installation_and_Usage/index.html#sect-Installing_the_Director_Packages

 openstack undercloud install  <--- should hit the bug here
Comment 16 Prasanth Anbalagan 2016-01-25 15:11 EST
Created attachment 1118172 [details]
QA logs for undercloud installation
Comment 17 Prasanth Anbalagan 2016-01-25 15:15:14 EST
Undercloud installation still throws the same error. Please see below for logs. Hence moving the bug back to assigned.

Version
========

[stack@instack ~]$ yum list installed | grep openstack-puppet-modules
openstack-puppet-modules.noarch       1:7.0.3-1.el7ost    @rhelosp-8.0-puddle   

Logs
=====
*********************************************************
[For full logs, please see attachment undercloud-logs]
*********************************************************

[stack@instack ~]$ openstack undercloud install
Logging to /home/stack/.instack/install-undercloud.log
Checking for a FQDN hostname...
Static hostname detected as instack.localdomain
..
..
...
+ rc=1
+ set -e
+ echo 'puppet apply exited with exit code 1'
puppet apply exited with exit code 1
+ '[' 1 '!=' 2 -a 1 '!=' 0 ']'
+ exit 1
[2016-01-25 14:57:44,824] (os-refresh-config) [ERROR] during configure phase. [Command '['dib-run-parts', '/usr/libexec/os-refresh-config/configure.d']' returned non-zero exit status 1]

[2016-01-25 14:57:44,824] (os-refresh-config) [ERROR] Aborting...
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 562, in install
    _run_orc(instack_env)
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 494, in _run_orc
    _run_live_command(args, instack_env, 'os-refresh-config')
  File "/usr/lib/python2.7/site-packages/instack_undercloud/undercloud.py", line 325, in _run_live_command
    raise RuntimeError('%s failed. See log for details.' % name)
RuntimeError: os-refresh-config failed. See log for details.
Command 'instack-install-undercloud' returned non-zero exit status 1
Comment 18 Jason Guiditta 2016-01-25 15:21:33 EST
The error I see in the log you attached says:

Error: Could not find resource 'Keystone_domain[heat_domain]' for relationship from 'Class[Keystone::Roles::Admin]' on node instack.localdomain
Error: Could not find resource 'Keystone_domain[heat_domain]' for relationship from 'Class[Keystone::Roles::Admin]' on node instack.localdomain

Which is not the Bug this BZ tracks, so I am not sure it is fair to fail it for some other issue.
Comment 19 Prasanth Anbalagan 2016-01-26 06:57:39 EST
Looks like the BZ is https://bugzilla.redhat.com/show_bug.cgi?id=1298189.
Marking the BZ as BLOCKED by 1298189.
Comment 20 Prasanth Anbalagan 2016-01-26 16:34 EST
Created attachment 1118616 [details]
QA Verification logs for undercloud Installation
Comment 21 Prasanth Anbalagan 2016-01-26 16:38:48 EST
After applying the workaround for BZ#1298189 (Comment 12), Undercloud installation completed successfully. Hence moving the bug to verified

Version
=======

[stack@instack ~]$ yum list installed | grep openstack-puppet-modules
openstack-puppet-modules.noarch   1:7.0.3-1.el7ost        @rhelosp-8.0-puddle   
[stack@instack ~]$ 


Logs
=====
Please see attachment 'undercloud-logs_passed' (link below) for more details
https://bugzilla.redhat.com/attachment.cgi?id=1118616


[stack@instack ~]$ 
[stack@instack ~]$ openstack --debug --log-file=undercloud_install.log undercloud install
START with options: ['--debug', '--log-file=undercloud_install.log', 'undercloud', 'install']
options: Namespace(access_token_endpoint='', auth_type='', auth_url='', cacert='', client_id='', client_secret='', cloud='', debug=True, default_domain='default', deferred_help=False, domain_id='', domain_name='', endpoint='', identity_provider='', identity_provider_url='', insecure=None, inspector_api_version='1', inspector_url=None, interface='', log_file='undercloud_install.log', os_compute_api_version='', os_identity_api_version='', os_image_api_version='', os_network_api_version='', os_object_api_version='', os_project_id=None, os_project_name=None, os_tripleoclient_api_version='1', os_volume_api_version='', password='', project_domain_id='', project_domain_name='', project_id='', project_name='', protocol='', region_name='', scope='', service_provider_endpoint='', timing=False, token='', trust_id='', url='', user_domain_id='', user_domain_name='', user_id='', username='', verbose_level=3, verify=None)..
..
...
....
.....
dib-run-parts Tue Jan 26 16:29:04 EST 2016 Running /usr/libexec/os-refresh-config/post-configure.d/99-refresh-completed
++ os-apply-config --key completion-handle --type raw --key-default ''
[2016/01/26 04:29:04 PM] [WARNING] DEPRECATED: falling back to /var/run/os-collect-config/os_config_files.json
+ HANDLE=
++ os-apply-config --key completion-signal --type raw --key-default ''
[2016/01/26 04:29:04 PM] [WARNING] DEPRECATED: falling back to /var/run/os-collect-config/os_config_files.json
+ SIGNAL=
++ os-apply-config --key instance-id --type raw --key-default ''
[2016/01/26 04:29:05 PM] [WARNING] DEPRECATED: falling back to /var/run/os-collect-config/os_config_files.json
+ ID=
+ '[' -n '' ']'
+ exit 0
dib-run-parts Tue Jan 26 16:29:05 EST 2016 99-refresh-completed completed
----------------------- PROFILING -----------------------

Target: post-configure.d

Script                                     Seconds
---------------------------------------  ----------

10-iptables                                   1.028
68-ironic-conductor                          13.050
80-seedstack-masquerade                       0.226
98-undercloud-setup                          57.073
99-refresh-completed                          0.779

--------------------- END PROFILING ---------------------
[2016-01-26 16:29:05,259] (os-refresh-config) [INFO] Completed phase post-configure
os-refresh-config completed successfully
Generated new ssh key in ~/.ssh/id_rsa

#############################################################################
Undercloud install complete.

The file containing this installation's passwords is at
/home/stack/undercloud-passwords.conf.

There is also a stackrc file at /home/stack/stackrc.

These files are needed to interact with the OpenStack services, and should be
secured.

#############################################################################

clean_up InstallPlugin: 
END return value: 0
Comment 26 errata-xmlrpc 2016-04-07 17:19:22 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-0603.html

Note You need to log in before you can comment on or make changes to this bug.