Description of problem: ----------------------- Installation of packstack --allinone is failing on Fedora20 ERROR : Error appeared during Puppet run: 10.35.64.158_neutron.pp Error: Could not start Service[ovs-cleanup-service]: Execution of '/sbin/service neutron-ovs-cleanup start' returned 1: You will find full trace in log /var/tmp/packstack/20140325-144031-XD3KyR/manifests/10.35.64.158_neutron.pp.log Please check log file /var/tmp/packstack/20140325-144031-XD3KyR/openstack-setup.log for more information Version-Release number of selected component (if applicable): openstack-packstack-2014.1.1-0.4.dev1018.fc21.noarch How reproducible: Steps to Reproduce: ------------------- 1. sudo yum install -y openstack-packstack 2. packstack --allinone Actual results: --------------- Full output of packstack --allinone is mentioned below: ====================================================================== packstack --allinone Welcome to Installer setup utility Packstack changed given value to required value /root/.ssh/id_rsa.pub Installing: Clean Up [ DONE ] root.64.158's password: Setting up ssh keys [ DONE ] Discovering hosts' details [ DONE ] Adding pre install manifest entries [ DONE ] Adding MySQL manifest entries [ DONE ] Adding AMQP manifest entries [ DONE ] Adding Keystone manifest entries [ DONE ] Adding Glance Keystone manifest entries [ DONE ] Adding Glance manifest entries [ DONE ] Installing dependencies for Cinder [ DONE ] Adding Cinder Keystone manifest entries [ DONE ] Adding Cinder manifest entries [ DONE ] Checking if the Cinder server has a cinder-volumes vg[ DONE ] Adding Nova API manifest entries [ DONE ] Adding Nova Keystone manifest entries [ DONE ] Adding Nova Cert manifest entries [ DONE ] Adding Nova Conductor manifest entries [ DONE ] Adding Nova Compute manifest entries [ DONE ] Adding Nova Scheduler manifest entries [ DONE ] Adding Nova VNC Proxy manifest entries [ DONE ] Adding Nova Common manifest entries [ DONE ] Adding Openstack Network-related Nova manifest entries[ DONE ] Adding Neutron API manifest entries [ DONE ] Adding Neutron Keystone manifest entries [ DONE ] Adding Neutron L3 manifest entries [ DONE ] Adding Neutron L2 Agent manifest entries [ DONE ] Adding Neutron DHCP Agent manifest entries [ DONE ] Adding Neutron LBaaS Agent manifest entries [ DONE ] Adding Neutron Metadata Agent manifest entries [ DONE ] Adding OpenStack Client manifest entries [ DONE ] Adding Horizon manifest entries [ DONE ] Adding Provisioning manifest entries [ DONE ] Adding Ceilometer manifest entries [ DONE ] Adding Ceilometer Keystone manifest entries [ DONE ] Adding Nagios server manifest entries [ DONE ] Adding Nagios host manifest entries [ DONE ] Adding post install manifest entries [ DONE ] Preparing servers [ DONE ] Installing Dependencies [ DONE ] Copying Puppet modules and manifests [ DONE ] Applying 10.35.64.158_prescript.pp 10.35.64.158_prescript.pp: [ DONE ] Applying 10.35.64.158_mysql.pp Applying 10.35.64.158_amqp.pp 10.35.64.158_mysql.pp: [ DONE ] 10.35.64.158_amqp.pp: [ DONE ] Applying 10.35.64.158_keystone.pp Applying 10.35.64.158_glance.pp Applying 10.35.64.158_cinder.pp 10.35.64.158_keystone.pp: [ DONE ] 10.35.64.158_glance.pp: [ DONE ] 10.35.64.158_cinder.pp: [ DONE ] Applying 10.35.64.158_api_nova.pp 10.35.64.158_api_nova.pp: [ DONE ] Applying 10.35.64.158_nova.pp 10.35.64.158_nova.pp: [ DONE ] Applying 10.35.64.158_neutron.pp 10.35.64.158_neutron.pp: [ ERROR ] Applying Puppet manifests [ ERROR ] ERROR : Error appeared during Puppet run: 10.35.64.158_neutron.pp Error: Could not start Service[ovs-cleanup-service]: Execution of '/sbin/service neutron-ovs-cleanup start' returned 1: You will find full trace in log /var/tmp/packstack/20140325-144031-XD3KyR/manifests/10.35.64.158_neutron.pp.log Please check log file /var/tmp/packstack/20140325-144031-XD3KyR/openstack-setup.log for more information ====================================================================== Expected results: ----------------- Packstack should install and bringup openstack successfully. Additional info: --------------- The following is the content of: /var/tmp/packstack/20140325-145608-rshEpx/manifests/10.35.64.158_neutron.pp.log ====================================================================== connect: No such file or directory Please make sure that the zfs-fuse daemon is running. internal error: failed to initialize ZFS library connect: No such file or directory Please make sure that the zfs-fuse daemon is running. internal error: failed to initialize ZFS library ^[[1;31mWarning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults^[[0m ^[[1;31mWarning: Scope(Class[Neutron::Server]): sql_connection deprecated for connection^[[0m ^[[mNotice: Compiled catalog for aqua-vds9.qa.lab.tlv.redhat.com in environment production in 0.98 seconds^[[0m connect: No such file or directory Please make sure that the zfs-fuse daemon is running. internal error: failed to initialize ZFS library connect: No such file or directory Please make sure that the zfs-fuse daemon is running. internal error: failed to initialize ZFS library ^[[1;31mError: Could not start Service[ovs-cleanup-service]: Execution of '/sbin/service neutron-ovs-cleanup start' returned 1: ^[[0m ^[[1;31mError: /Stage[main]/Neutron::Agents::Ovs/Service[ovs-cleanup-service]/ensure: change from stopped to running failed: Could not start Service[ovs-cleanup-service]: Execution of '/sbin/service neutron-ovs-cleanup start' returned 1: ^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: Traceback (most recent call last):^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/bin/neutron-db-manage", line 10, in <module>^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: sys.exit(main())^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py", line 138, in main^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: CONF()^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/oslo/config/cfg.py", line 1599, in __call__^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: raise ConfigFilesNotFoundError(self._namespace.files_not_found)^[[0m ^[[mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: oslo.config.cfg.ConfigFilesNotFoundError: Failed to read some config files: /etc/neutron/plugin.ini^[[0m ^[[1;31mError: neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini upgrade head returned 1 instead of one of [0]^[[0m ^[[1;31mError: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: change from notrun to 0 failed: neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini upgrade head returned 1 instead of one of [0]^[[0m ^[[mNotice: /Stage[main]/Neutron::Agents::Ovs/Service[neutron-plugin-ovs-service]/ensure: ensure changed 'stopped' to 'running'^[[0m ^[[mNotice: /Stage[main]/Neutron::Agents::Dhcp/Service[neutron-dhcp-service]/ensure: ensure changed 'stopped' to 'running'^[[0m ^[[mNotice: /Stage[main]/Neutron::Server/Service[neutron-server]: Dependency Exec[neutron-db-manage upgrade] has failures: true^[[0m ^[[1;31mWarning: /Stage[main]/Neutron::Server/Service[neutron-server]: Skipping because of failed dependencies^[[0m ^[[mNotice: Finished catalog run in 1.29 seconds^[[0m ====================================================================== OS: Fedora20 SELINUX=enforcing
Created attachment 878454 [details] packstack install logs for recreate recreated the issue.. see log file
From the log in description: oslo.config.cfg.ConfigFilesNotFoundError: Failed to read some config files: /etc/neutron/plugin.ini That's was also seen by eglynn and was due to the permissions causing /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini being unreadable by the neutron user. My guess is that openstack-puppet-modules is replacing that file (with wrong owner)
I confirm that this was issue on Martin Magr's machine while hitting this issue. Changing ownership of /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini to neutron:neutron fixed the issue.
The package sets that file to root:neutron Something is intermittently setting it to root:root That might happen if (puppet?) logic runs against that file before the package is installed
*** Bug 1080560 has been marked as a duplicate of this bug. ***
I don't think this is puppet specific. See the steps i've used to reproduce in the bug i marked as a duplicate of this one above. Has something to do with the neutron group not existing on the very first install of the package.
<mmagr> pixelb, so you were write, puppet-neutron's neutron_plugin_ovs provider uses ini_file resource from inifile module which simply opens config file with 'w', which is what I think rewrites permission on the config file ... since all openstack related puppet modules are using ini_file to modify configs we will have this problem with all components ... <mmagr> pixelb, will implement fix for pupet-neutron, but we will have to fix that on ini_file level in future
Submitted patch is to make sure ownership is in state it is supposed to be. Bug #1080560 should be reopened so Neutron guys can fix that on packaging side ... if it is problem in Neutron package.
I think this is just a neutron packaging bug Martin and so the packstack workaround might best be left out.
Neutron side fixed in bug 1080560