Bug 1130724

Summary: packstack: Error appeared during Puppet run: <IP> mysql.pp Error: mysqladmin -u root fails
Product: [Community] RDO Reporter: b24warbaby
Component: openstack-packstackAssignee: Martin Magr <mmagr>
Status: CLOSED NOTABUG QA Contact: Ido Ovadia <iovadia>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: aortega, derekh, yeylon
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-08-17 17:04:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Last log file none

Description b24warbaby 2014-08-17 03:31:07 UTC
Created attachment 927401 [details]
Last log file

Description of problem:


Fedora 20 x64


 packstack --allinone


Welcome to Installer setup utility

Installing:
Clean Up                                             [ DONE ]
Setting up ssh keys                                  [ DONE ]
Discovering hosts' details                           [ DONE ]
Adding pre install manifest entries                  [ DONE ]
Adding MySQL manifest entries                        [ DONE ]
Adding QPID manifest entries                         [ DONE ]
Adding Keystone manifest entries                     [ DONE ]
Adding Glance Keystone manifest entries              [ DONE ]
Adding Glance manifest entries                       [ DONE ]
Installing dependencies for Cinder                   [ DONE ]
Adding Cinder Keystone manifest entries              [ DONE ]
Adding Cinder manifest entries                       [ DONE ]
Checking if the Cinder server has a cinder-volumes vg[ DONE ]
Adding Nova API manifest entries                     [ DONE ]
Adding Nova Keystone manifest entries                [ DONE ]
Adding Nova Cert manifest entries                    [ DONE ]
Adding Nova Conductor manifest entries               [ DONE ]
Adding Nova Compute manifest entries                 [ DONE ]
Adding Nova Scheduler manifest entries               [ DONE ]
Adding Nova VNC Proxy manifest entries               [ DONE ]
Adding Nova Common manifest entries                  [ DONE ]
Adding Openstack Network-related Nova manifest entries[ DONE ]
Adding Neutron API manifest entries                  [ DONE ]
Adding Neutron Keystone manifest entries             [ DONE ]
Adding Neutron L3 manifest entries                   [ DONE ]
Adding Neutron L2 Agent manifest entries             [ DONE ]
Adding Neutron DHCP Agent manifest entries           [ DONE ]
Adding Neutron LBaaS Agent manifest entries          [ DONE ]
Adding Neutron Metadata Agent manifest entries       [ DONE ]
Adding OpenStack Client manifest entries             [ DONE ]
Adding Horizon manifest entries                      [ DONE ]
Adding Swift Keystone manifest entries               [ DONE ]
Adding Swift builder manifest entries                [ DONE ]
Adding Swift proxy manifest entries                  [ DONE ]
Adding Swift storage manifest entries                [ DONE ]
Adding Swift common manifest entries                 [ DONE ]
Adding Provisioning manifest entries                 [ DONE ]
Adding Ceilometer manifest entries                   [ DONE ]
Adding Ceilometer Keystone manifest entries          [ DONE 
Adding Ceilometer Keystone manifest entries          [ DONE ]
Adding Nagios server manifest entries                [ DONE ]
Adding Nagios host manifest entries                  [ DONE ]
Adding post install manifest entries                 [ DONE ]
Preparing servers                                    [ DONE ]
Installing Dependencies    
Preparing servers                                    [ DONE ]
Installing Dependencies                              [ DONE ]
Copying Puppet modules and manifests                 [ DONE ]
Applying 192.168.1.126_prescript.pp
192.168.1.126_prescript.pp:                          [ DONE ]         
Applying 192.168.1.126_mysql.pp
Applying 192.168.1.126_qpid.pp
192.168.1.126_mysql.pp:                           [ ERROR ]       
Applying Puppet manifests                         [ ERROR ]

ERROR : Error appeared during Puppet run: 192.168.1.126_mysql.pp
Error: mysqladmin -u root  password '34a1ce3572424a9c' returned 1 instead of one of [0]
You will find full trace in log /var/tmp/packstack/20140816-220059-vzV2BX/manifests/192.168.1.126_mysql.pp.log
Please check log file /var/tmp/packstack/20140816-220059-vzV2BX/openstack-setup.log for more information



backtrace from log:

cd /usr/lib/python2.7/site-packages/packstack/puppet
cd /var/tmp/packstack/20140816-220059-vzV2BX/manifests
tar --dereference -cpzf - ../manifests | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.126 tar -C /var/tmp/packstack/6806be72d365465b8d61a3c5eba92fc6 -xpzf -
cd /usr/share/openstack-puppet/modules
tar --dereference -cpzf - apache ceilometer certmonger cinder concat firewall glance heat horizon inifile keystone memcached mongodb mysql neutron nova nssdb openstack packstack qpid rsync ssh stdlib swift sysctl tempest vcsrepo vlan vswitch xinetd | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.126 tar -C /var/tmp/packstack/6806be72d365465b8d61a3c5eba92fc6/modules -xpzf -

 
2014-08-16 22:02:27::ERROR::run_setup::891::root:: Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 872, in main
    single_step_aio_install(options)
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 656, in single_step_aio_install
    single_step_install(options)
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 689, in single_step_install
    _main(answerfilepath)
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 573, in _main
    runSequences()
  File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 552, in runSequences
    controller.runAllSequences()
  File "/usr/lib/python2.7/site-packages/packstack/installer/setup_controller.py", line 84, in runAllSequences
    sequence.run(self.CONF)
  File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 96, in run
    step.run(config=config)
  File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 43, in run
    raise SequenceError(str(ex))
SequenceError: Error appeared during Puppet run: 192.168.1.126_mysql.pp
Error: mysqladmin -u root  password '34a1ce3572424a9c' returned 1 instead of one of [0]^[[0m



I set SELinux to permissive,  rebooted Still happens

 sudo rpm -qf  `which packstack`

openstack-packstack-2013.2.1-0.36.dev1013.fc20.noarch

uname -a
Linux gibsonsg 3.15.9-200.fc20.x86_64 #1 SMP Sat Aug 9 09:02:55 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Comment 1 b24warbaby 2014-08-17 03:36:12 UTC

The dashboard webapge is not working so I am assuming the install completely failed.

Comment 2 b24warbaby 2014-08-17 12:13:40 UTC

I got past this error by removing mysql:

yum remove mysql mysql-server
rm -rf /var/lib/mysql

Now keystone is failing:



Adding post install manifest entries                 [ DONE ]
Preparing servers                                    [ DONE ]
Installing Dependencies                              [ DONE ]
Copying Puppet modules and manifests                 [ DONE ]
Applying 192.168.1.126_prescript.pp
192.168.1.126_prescript.pp:                          [ DONE ]         
Applying 192.168.1.126_mysql.pp
Applying 192.168.1.126_qpid.pp
192.168.1.126_mysql.pp:                              [ DONE ]     
192.168.1.126_qpid.pp:                               [ DONE ]     
Applying 192.168.1.126_keystone.pp
Applying 192.168.1.126_glance.pp
Applying 192.168.1.126_cinder.pp
192.168.1.126_keystone.pp:                        [ ERROR ]          
Applying Puppet manifests                         [ ERROR ]


ERROR : Error appeared during Puppet run: 192.168.1.126_keystone.pp
Error: Could not start Service[keystone]: Execution of '/sbin/service openstack-keystone start' returned 1: 
You will find full trace in log /var/tmp/packstack/20140817-065740-Pelo2F/manifests/192.168.1.126_keystone.pp.log
Please check log file /var/tmp/packstack/20140817-065740-Pelo2F/openstack-setup.log for more information

Additional information:
 * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
 * Did not create a cinder volume group, one already existed
 * File /root/keystonerc_admin has been created on OpenStack client host 192.168.1.126. To use the command line tools you need to source the file.
 * Copy of keystonerc_admin file has been created for non-root user in /home/jpd.
 * To access the OpenStack Dashboard browse to http://192.168.1.126/dashboard.
Please, find your login credentials stored in the keystonerc_admin in your home directory.
 * To use Nagios, browse to http://192.168.1.126/nagios username : nagiosadmin, password : 848ae55280ab47e4
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ sudo systemctl status openstack-keystone.service
openstack-keystone.service - OpenStack Identity Service (code-named Keystone)
   Loaded: loaded (/usr/lib/systemd/system/openstack-keystone.service; disabled)
   Active: failed (Result: start-limit) since Sun 2014-08-17 06:59:50 CDT; 7min ago
 Main PID: 15428 (code=exited, status=1/FAILURE)
   CGroup: /system.slice/openstack-keystone.service

Aug 17 06:59:50 gibsonsg systemd[1]: Failed to start OpenStack Identity Ser...).
Aug 17 06:59:50 gibsonsg systemd[1]: Unit openstack-keystone.service entere...e.
Aug 17 06:59:50 gibsonsg systemd[1]: openstack-keystone.service holdoff tim...t.
Aug 17 06:59:50 gibsonsg systemd[1]: Stopping OpenStack Identity Service (c.....
Aug 17 06:59:50 gibsonsg systemd[1]: Starting OpenStack Identity Service (c.....
Aug 17 06:59:50 gibsonsg systemd[1]: openstack-keystone.service start reque...t.
Aug 17 06:59:50 gibsonsg systemd[1]: Failed to start OpenStack Identity Ser...).
Aug 17 06:59:50 gibsonsg systemd[1]: Unit openstack-keystone.service entere...e.
Hint: Some lines were ellipsized, use -l to show in full.
[jpd@gibsonsg ~]$ sudo systemctl restart openstack-keystone.service
Job for openstack-keystone.service failed. See 'systemctl status openstack-keystone.service' and 'journalctl -xn' for details.
[jpd@gibsonsg ~]$ sudo systemctl status openstack-keystone.service
openstack-keystone.service - OpenStack Identity Service (code-named Keystone)
   Loaded: loaded (/usr/lib/systemd/system/openstack-keystone.service; disabled)
   Active: failed (Result: start-limit) since Sun 2014-08-17 07:07:13 CDT; 13s ago
  Process: 19116 ExecStart=/usr/bin/keystone-all --config-file /usr/share/keystone/keystone-dist.conf --config-file /etc/keystone/keystone.conf (code=exited, status=1/FAILURE)
 Main PID: 19116 (code=exited, status=1/FAILURE)

Aug 17 07:07:13 gibsonsg systemd[1]: Failed to start OpenStack Identity Ser...).
Aug 17 07:07:13 gibsonsg systemd[1]: Unit openstack-keystone.service entere...e.
Aug 17 07:07:13 gibsonsg systemd[1]: openstack-keystone.service holdoff tim...t.
Aug 17 07:07:13 gibsonsg systemd[1]: Stopping OpenStack Identity Service (c.....
Aug 17 07:07:13 gibsonsg systemd[1]: Starting OpenStack Identity Service (c.....
Aug 17 07:07:13 gibsonsg systemd[1]: openstack-keystone.service start reque...t.
Aug 17 07:07:13 gibsonsg systemd[1]: Failed to start OpenStack Identity Ser...).
Aug 17 07:07:13 gibsonsg systemd[1]: Unit openstack-keystone.service entere...e.
Hint: Some lines were ellipsized, use -l to show in full.
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ sudo systemctl status -l openstack-keystone.service
openstack-keystone.service - OpenStack Identity Service (code-named Keystone)
   Loaded: loaded (/usr/lib/systemd/system/openstack-keystone.service; disabled)
   Active: failed (Result: start-limit) since Sun 2014-08-17 07:07:13 CDT; 32s ago
  Process: 19116 ExecStart=/usr/bin/keystone-all --config-file /usr/share/keystone/keystone-dist.conf --config-file /etc/keystone/keystone.conf (code=exited, status=1/FAILURE)
 Main PID: 19116 (code=exited, status=1/FAILURE)

Aug 17 07:07:13 gibsonsg systemd[1]: Failed to start OpenStack Identity Service (code-named Keystone).
Aug 17 07:07:13 gibsonsg systemd[1]: Unit openstack-keystone.service entered failed state.
Aug 17 07:07:13 gibsonsg systemd[1]: openstack-keystone.service holdoff time over, scheduling restart.
Aug 17 07:07:13 gibsonsg systemd[1]: Stopping OpenStack Identity Service (code-named Keystone)...
Aug 17 07:07:13 gibsonsg systemd[1]: Starting OpenStack Identity Service (code-named Keystone)...
Aug 17 07:07:13 gibsonsg systemd[1]: openstack-keystone.service start request repeated too quickly, refusing to start.
Aug 17 07:07:13 gibsonsg systemd[1]: Failed to start OpenStack Identity Service (code-named Keystone).
Aug 17 07:07:13 gibsonsg systemd[1]: Unit openstack-keystone.service entered failed state.
[jpd@gibsonsg ~]$ 
[jpd@gibsonsg ~]$ sudo journalctl -xn
-- Logs begin at Sat 2014-08-16 04:53:35 CDT, end at Sun 2014-08-17 07:08:18 CDT
Aug 17 07:07:22 gibsonsg setroubleshoot[19067]: KeyboardInterrupt in RunFaultSer
Aug 17 07:07:22 gibsonsg setroubleshoot[19067]: writing database (/var/lib/setro
Aug 17 07:07:22 gibsonsg dbus-daemon[790]: 'list' object has no attribute 'split
Aug 17 07:07:22 gibsonsg dbus-daemon[790]: string index out of range
Aug 17 07:07:26 gibsonsg sudo[19124]: jpd : TTY=pts/0 ; PWD=/home/jpd ; USER=roo
Aug 17 07:07:45 gibsonsg sudo[19133]: jpd : TTY=pts/0 ; PWD=/home/jpd ; USER=roo
Aug 17 07:08:01 gibsonsg systemd[1]: Starting Session 1129 of user root.
-- Subject: Unit session-1129.scope has begun with start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit session-1129.scope has begun starting up.
Aug 17 07:08:01 gibsonsg systemd[1]: Started Session 1129 of user root.
-- Subject: Unit session-1129.scope has finished start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit session-1129.scope has finished starting up.
-- 
-- The start-up result is done.
Aug 17 07:08:01 gibsonsg CROND[19140]: (root) CMD (/usr/bin/keystone-manage toke
Aug 17 07:08:18 gibsonsg sudo[19152]: jpd : TTY=pts/0 ; PWD=/home/jpd ; USER=roo
[jpd@gibsonsg ~]$

Comment 3 b24warbaby 2014-08-17 17:03:56 UTC

I started with a fresh F20 install and immediatley installed packstack;

It appears to work ;