Description of problem: packstack --allinone fails with 192.168.1.158_mariadb.pp: [ DONE ] Applying 192.168.1.158_keystone.pp Applying 192.168.1.158_glance.pp Applying 192.168.1.158_cinder.pp 192.168.1.158_keystone.pp: [ ERROR ] Applying Puppet manifests [ ERROR ] ERROR : Error appeared during Puppet run: 192.168.1.158_keystone.pp Error: Failed to apply catalog: Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: Internal Server Error (HTTP 500) Version-Release number of selected component (if applicable): Used fresh installation of Fedora 22-3 Workstation. Fetched the latest version of OpenStack as described on https://www.rdoproject.org/install/quickstart/ How reproducible: Just followed the steps in https://www.rdoproject.org/install/quickstart/ Steps to Reproduce: 1. follow the steps in https://www.rdoproject.org/install/quickstart/ 2. run packstack --allinone Actual results: 192.168.1.158_keystone.pp: [ ERROR ] Applying Puppet manifests [ ERROR ] ERROR : Error appeared during Puppet run: 192.168.1.158_keystone.pp Error: Failed to apply catalog: Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: Internal Server Error (HTTP 500) Expected results: No installation errors Additional info: [root@so ~]# cat /var/tmp/packstack/20151027-193859-XXZqoI/openstack-setup.log 2015-10-27 19:38:59::INFO::shell::94::root:: [localhost] Executing script: DEVICE=($(ip -o address show to 192.168.1.158 | cut -f 2 -d ' ')) if [ ! -z ${DISPLAY[1]} ]; then false; fi ip link show "$DEVICE" > /dev/null echo $DEVICE 2015-10-27 19:39:00::INFO::shell::94::root:: [localhost] Executing script: rm -rf /var/tmp/packstack/20151027-193859-XXZqoI/manifests/*pp 2015-10-27 19:39:00::INFO::shell::94::root:: [localhost] Executing script: mkdir -p ~/.ssh chmod 500 ~/.ssh grep 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYj1Sz7MS/JZcjTTMvkUnDqcCYoxfR1nganMdiyM3cnrjYLVv+PWgpIEY413njcdn1ampm6cHyctTAwiWCtYVTuRm/OCIuts0n9uWegcBU/qjCU/lX55GOWFhClpwMEZSEmPTig1Ks3hEXeRpRJ71as+mxk3KbF9gxG6PNAmPGyZnFMewKbadJbNEyJGnqVF8+j2cIQVYThhAQGgzNzICSBwDuEFvcAwWxYPS6tHukG6p2puuJesopKqiK5pW+sxDWex1yjd6UQy9Z8dcWBrdVLqM/rlaUwo5kpC6Tvzrc5n9rIMoUTG4sPq1gDWbK+ijP0dSTx1ft72aiHBfyKzMT root.se' ~/.ssh/authorized_keys > /dev/null 2>&1 || echo ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYj1Sz7MS/JZcjTTMvkUnDqcCYoxfR1nganMdiyM3cnrjYLVv+PWgpIEY413njcdn1ampm6cHyctTAwiWCtYVTuRm/OCIuts0n9uWegcBU/qjCU/lX55GOWFhClpwMEZSEmPTig1Ks3hEXeRpRJ71as+mxk3KbF9gxG6PNAmPGyZnFMewKbadJbNEyJGnqVF8+j2cIQVYThhAQGgzNzICSBwDuEFvcAwWxYPS6tHukG6p2puuJesopKqiK5pW+sxDWex1yjd6UQy9Z8dcWBrdVLqM/rlaUwo5kpC6Tvzrc5n9rIMoUTG4sPq1gDWbK+ijP0dSTx1ft72aiHBfyKzMT root.se >> ~/.ssh/authorized_keys chmod 400 ~/.ssh/authorized_keys restorecon -r ~/.ssh 2015-10-27 19:39:00::INFO::shell::94::root:: [192.168.1.158] Executing script: rpm -q --whatprovides yum-utils || yum install -y yum-utils 2015-10-27 19:39:01::INFO::shell::48::root:: Executing command: rpm -q rdo-release --qf='%{version}-%{release}.%{arch} ' 2015-10-27 19:39:02::INFO::shell::94::root:: [192.168.1.158] Executing script: (rpm -q 'rdo-release-liberty' || yum install -y --nogpg http://rdo.fedorapeople.org/openstack/openstack-liberty/rdo-release-liberty-2.noarch.rpm) || true 2015-10-27 19:39:04::INFO::shell::94::root:: [192.168.1.158] Executing script: yum-config-manager --enable openstack-liberty 2015-10-27 19:39:05::INFO::shell::94::root:: [192.168.1.158] Executing script: python -c "import platform; print platform.linux_distribution(full_distribution_name=0)[0]+','+platform.linux_distribution()[1]" 2015-10-27 19:39:05::INFO::shell::94::root:: [192.168.1.158] Executing script: rpm -q --whatprovides yum-utils || yum install -y yum-utils yum clean metadata 2015-10-27 19:39:07::INFO::shell::94::root:: [localhost] Executing script: rpm -q --requires openstack-puppet-modules | egrep -v "^(rpmlib|\/|perl)" 2015-10-27 19:39:08::INFO::shell::94::root:: [localhost] Executing script: rpm -q --requires openstack-packstack-puppet | egrep -v "^(rpmlib|\/|perl)" 2015-10-27 19:39:09::INFO::shell::94::root:: [192.168.1.158] Executing script: yum install -y puppet hiera openssh-clients tar nc rubygem-json yum update -y puppet hiera openssh-clients tar nc rubygem-json rpm -q --whatprovides puppet rpm -q --whatprovides hiera rpm -q --whatprovides openssh-clients rpm -q --whatprovides tar rpm -q --whatprovides nc rpm -q --whatprovides rubygem-json 2015-10-27 19:40:41::INFO::shell::94::root:: [192.168.1.158] Executing script: mkdir -p /var/tmp/packstack mkdir --mode 0700 /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 mkdir --mode 0700 /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/modules mkdir --mode 0700 /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/resources 2015-10-27 19:40:41::INFO::shell::94::root:: [192.168.1.158] Executing script: facter -p 2015-10-27 19:40:43::INFO::shell::94::root:: [192.168.1.158] Executing script: [[ ! -L /etc/puppet/hiera.yaml ]] && ln -s /etc/hiera.yaml /etc/puppet/hiera.yaml || echo "hiera.yaml symlink already created" sed -i 's;:datadir:.*;:datadir: /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/hieradata;g' /etc/puppet/hiera.yaml 2015-10-27 19:40:43::INFO::shell::94::root:: [192.168.1.158] Executing script: vgdisplay cinder-volumes 2015-10-27 19:40:44::INFO::shell::94::root:: [localhost] Executing script: ssh-keygen -t rsa -b 2048 -f "/var/tmp/packstack/20151027-193859-XXZqoI/nova_migration_key" -N "" 2015-10-27 19:40:44::INFO::shell::94::root:: [localhost] Executing script: ssh-keyscan 192.168.1.158 2015-10-27 19:40:44::INFO::shell::94::root:: [192.168.1.158] Executing script: systemctl 2015-10-27 19:40:45::INFO::shell::94::root:: [192.168.1.158] Executing script: systemctl is-enabled NetworkManager 2015-10-27 19:40:45::INFO::shell::94::root:: [192.168.1.158] Executing script: systemctl is-active NetworkManager 2015-10-27 19:40:45::INFO::shell::94::root:: [192.168.1.158] Executing script: echo $HOME 2015-10-27 19:40:46::INFO::shell::94::root:: [localhost] Executing script: cd /var/tmp/packstack/20151027-193859-XXZqoI/hieradata tar --dereference -cpzf - ../hieradata | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.158 tar -C /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 -xpzf - cd /usr/lib/python2.7/site-packages/packstack/puppet cd /var/tmp/packstack/20151027-193859-XXZqoI/manifests tar --dereference -cpzf - ../manifests | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.158 tar -C /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 -xpzf - cd /usr/share/openstack-puppet/modules tar --dereference -cpzf - apache ceilometer certmonger cinder concat firewall glance galera heat horizon inifile ironic keystone manila memcached mongodb mysql neutron nova nssdb openstack packstack qpid rabbitmq redis remote rsync sahara ssh stdlib swift sysctl tempest trove vcsrepo vlan vswitch xinetd openstacklib | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.158 tar -C /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/modules -xpzf - 2015-10-27 19:41:42::ERROR::run_setup::1017::root:: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 992, in main single_step_aio_install(options, logFile) File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 760, in single_step_aio_install single_step_install(options, logFile) File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 791, in single_step_install _main(options, answerfilepath, logFile) File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 660, in _main runSequences() File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 627, in runSequences controller.runAllSequences() File "/usr/lib/python2.7/site-packages/packstack/installer/setup_controller.py", line 81, in runAllSequences sequence.run(config=self.CONF, messages=self.MESSAGES) File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 109, in run step.run(config=config, messages=messages) File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 50, in run self.function(config, messages) File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 209, in apply_puppet_manifest wait_for_puppet(currently_running, messages) File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 123, in wait_for_puppet validate_logfile(log) File "/usr/lib/python2.7/site-packages/packstack/modules/puppet.py", line 107, in validate_logfile raise PuppetError(message) PuppetError: Error appeared during Puppet run: 192.168.1.158_keystone.pp Error: Failed to apply catalog: Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: Internal Server Error (HTTP 500) You will find full trace in log /var/tmp/packstack/20151027-193859-XXZqoI/manifests/192.168.1.158_keystone.pp.log 2015-10-27 19:41:42::INFO::shell::94::root:: [192.168.1.158] Executing script: rm -rf /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 [root@so ~]# cat /var/tmp/packstack/20151027-193859-XXZqoI/openstack-setup.log 2015-10-27 19:38:59::INFO::shell::94::root:: [localhost] Executing script: DEVICE=($(ip -o address show to 192.168.1.158 | cut -f 2 -d ' ')) if [ ! -z ${DISPLAY[1]} ]; then false; fi ip link show "$DEVICE" > /dev/null echo $DEVICE 2015-10-27 19:39:00::INFO::shell::94::root:: [localhost] Executing script: rm -rf /var/tmp/packstack/20151027-193859-XXZqoI/manifests/*pp 2015-10-27 19:39:00::INFO::shell::94::root:: [localhost] Executing script: mkdir -p ~/.ssh chmod 500 ~/.ssh grep 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYj1Sz7MS/JZcjTTMvkUnDqcCYoxfR1nganMdiyM3cnrjYLVv+PWgpIEY413njcdn1ampm6cHyctTAwiWCtYVTuRm/OCIuts0n9uWegcBU/qjCU/lX55GOWFhClpwMEZSEmPTig1Ks3hEXeRpRJ71as+mxk3KbF9gxG6PNAmPGyZnFMewKbadJbNEyJGnqVF8+j2cIQVYThhAQGgzNzICSBwDuEFvcAwWxYPS6tHukG6p2puuJesopKqiK5pW+sxDWex1yjd6UQy9Z8dcWBrdVLqM/rlaUwo5kpC6Tvzrc5n9rIMoUTG4sPq1gDWbK+ijP0dSTx1ft72aiHBfyKzMT root.se' ~/.ssh/authorized_keys > /dev/null 2>&1 || echo ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYj1Sz7MS/JZcjTTMvkUnDqcCYoxfR1nganMdiyM3cnrjYLVv+PWgpIEY413njcdn1ampm6cHyctTAwiWCtYVTuRm/OCIuts0n9uWegcBU/qjCU/lX55GOWFhClpwMEZSEmPTig1Ks3hEXeRpRJ71as+mxk3KbF9gxG6PNAmPGyZnFMewKbadJbNEyJGnqVF8+j2cIQVYThhAQGgzNzICSBwDuEFvcAwWxYPS6tHukG6p2puuJesopKqiK5pW+sxDWex1yjd6UQy9Z8dcWBrdVLqM/rlaUwo5kpC6Tvzrc5n9rIMoUTG4sPq1gDWbK+ijP0dSTx1ft72aiHBfyKzMT root.se >> ~/.ssh/authorized_keys chmod 400 ~/.ssh/authorized_keys restorecon -r ~/.ssh 2015-10-27 19:39:00::INFO::shell::94::root:: [192.168.1.158] Executing script: rpm -q --whatprovides yum-utils || yum install -y yum-utils 2015-10-27 19:39:01::INFO::shell::48::root:: Executing command: rpm -q rdo-release --qf='%{version}-%{release}.%{arch} ' 2015-10-27 19:39:02::INFO::shell::94::root:: [192.168.1.158] Executing script: (rpm -q 'rdo-release-liberty' || yum install -y --nogpg http://rdo.fedorapeople.org/openstack/openstack-liberty/rdo-release-liberty-2.noarch.rpm) || true 2015-10-27 19:39:04::INFO::shell::94::root:: [192.168.1.158] Executing script: yum-config-manager --enable openstack-liberty 2015-10-27 19:39:05::INFO::shell::94::root:: [192.168.1.158] Executing script: python -c "import platform; print platform.linux_distribution(full_distribution_name=0)[0]+','+platform.linux_distribution()[1]" 2015-10-27 19:39:05::INFO::shell::94::root:: [192.168.1.158] Executing script: rpm -q --whatprovides yum-utils || yum install -y yum-utils yum clean metadata 2015-10-27 19:39:07::INFO::shell::94::root:: [localhost] Executing script: rpm -q --requires openstack-puppet-modules | egrep -v "^(rpmlib|\/|perl)" 2015-10-27 19:39:08::INFO::shell::94::root:: [localhost] Executing script: rpm -q --requires openstack-packstack-puppet | egrep -v "^(rpmlib|\/|perl)" 2015-10-27 19:39:09::INFO::shell::94::root:: [192.168.1.158] Executing script: yum install -y puppet hiera openssh-clients tar nc rubygem-json yum update -y puppet hiera openssh-clients tar nc rubygem-json rpm -q --whatprovides puppet rpm -q --whatprovides hiera rpm -q --whatprovides openssh-clients rpm -q --whatprovides tar rpm -q --whatprovides nc rpm -q --whatprovides rubygem-json 2015-10-27 19:40:41::INFO::shell::94::root:: [192.168.1.158] Executing script: mkdir -p /var/tmp/packstack mkdir --mode 0700 /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 mkdir --mode 0700 /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/modules mkdir --mode 0700 /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/resources 2015-10-27 19:40:41::INFO::shell::94::root:: [192.168.1.158] Executing script: facter -p 2015-10-27 19:40:43::INFO::shell::94::root:: [192.168.1.158] Executing script: [[ ! -L /etc/puppet/hiera.yaml ]] && ln -s /etc/hiera.yaml /etc/puppet/hiera.yaml || echo "hiera.yaml symlink already created" sed -i 's;:datadir:.*;:datadir: /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/hieradata;g' /etc/puppet/hiera.yaml 2015-10-27 19:40:43::INFO::shell::94::root:: [192.168.1.158] Executing script: vgdisplay cinder-volumes 2015-10-27 19:40:44::INFO::shell::94::root:: [localhost] Executing script: ssh-keygen -t rsa -b 2048 -f "/var/tmp/packstack/20151027-193859-XXZqoI/nova_migration_key" -N "" 2015-10-27 19:40:44::INFO::shell::94::root:: [localhost] Executing script: ssh-keyscan 192.168.1.158 2015-10-27 19:40:44::INFO::shell::94::root:: [192.168.1.158] Executing script: systemctl 2015-10-27 19:40:45::INFO::shell::94::root:: [192.168.1.158] Executing script: systemctl is-enabled NetworkManager 2015-10-27 19:40:45::INFO::shell::94::root:: [192.168.1.158] Executing script: systemctl is-active NetworkManager 2015-10-27 19:40:45::INFO::shell::94::root:: [192.168.1.158] Executing script: echo $HOME 2015-10-27 19:40:46::INFO::shell::94::root:: [localhost] Executing script: cd /var/tmp/packstack/20151027-193859-XXZqoI/hieradata tar --dereference -cpzf - ../hieradata | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.158 tar -C /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 -xpzf - cd /usr/lib/python2.7/site-packages/packstack/puppet cd /var/tmp/packstack/20151027-193859-XXZqoI/manifests tar --dereference -cpzf - ../manifests | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.158 tar -C /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1 -xpzf - cd /usr/share/openstack-puppet/modules tar --dereference -cpzf - apache ceilometer certmonger cinder concat firewall glance galera heat horizon inifile ironic keystone manila memcached mongodb mysql neutron nova nssdb openstack packstack qpid rabbitmq redis remote rsync sahara ssh stdlib swift sysctl tempest trove vcsrepo vlan vswitch xinetd openstacklib | ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null root.1.158 tar -C /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1/modules -xpzf - 2015-10-27 19:41:42::ERROR::run_setup::1017::root:: Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 992, in main single_step_aio_install(options, logFile) File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 760, in single_step_aio_install single_step_install(options, logFile) File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 791, in single_step_install _main(options, answerfilepath, logFile) File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 660, in _main runSequences() File "/usr/lib/python2.7/site-packages/packstack/installer/run_setup.py", line 627, in runSequences controller.runAllSequences() File "/usr/lib/python2.7/site-packages/packstack/installer/setup_controller.py", line 81, in runAllSequences sequence.run(config=self.CONF, messages=self.MESSAGES) File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 109, in run step.run(config=config, messages=messages) File "/usr/lib/python2.7/site-packages/packstack/installer/core/sequences.py", line 50, in run self.function(config, messages) File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 209, in apply_puppet_manifest wait_for_puppet(currently_running, messages) File "/usr/lib/python2.7/site-packages/packstack/plugins/puppet_950.py", line 123, in wait_for_puppet validate_logfile(log) File "/usr/lib/python2.7/site-packages/packstack/modules/puppet.py", line 107, in validate_logfile raise PuppetError(message) PuppetError: Error appeared during Puppet run: 192.168.1.158_keystone.pp Error: Failed to apply catalog: Execution of '/usr/bin/openstack project list --quiet --format csv --long' returned 1: Internal Server Error (HTTP 500) You will find full trace in log /var/tmp/packstack/20151027-193859-XXZqoI/manifests/192.168.1.158_keystone.pp.log 2015-10-27 19:41:42::INFO::shell::94::root:: [192.168.1.158] Executing script: rm -rf /var/tmp/packstack/34436bb2b9014b688f05d141f2f44ac1
Can confirm this error today on fedora 23
This bug is against a Version which has reached End of Life. If it's still present in supported release (http://releases.openstack.org), please update Version and reopen.