Description of problem: install via packstack fails: Notice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: sqlalchemy.exc.OperationalError: (OperationalError) (1054, "Unknown column 'id' in 'field list'") 'DELETE routerl3agentbindings FROM routerl3agentbindings LEFT OUTER JOIN (SELECT MIN(id) as id, router_id, l3_agent_id FROM routerl3agentbindings GROUP BY router_id, l3_agent_id) AS temp ON routerl3agentbindings.id = temp.id WHERE temp.id is NULL;' () From log file: [1;31mWarning: Config file /etc/puppet/hiera.yaml not found, using Hiera defaults[0m [1;31mWarning: Scope(Class[Neutron::Server]): The mysql_module parameter is deprecated. The latest 2.x mysql module will be used.[0m [1;31mWarning: Scope(Class[Neutron::Server]): The sql_connection parameter is deprecated, use database_connection instead.[0m [1;31mWarning: Scope(Class[Neutron::Agents::L3]): The neutron::l3_agent::newtork_device_mtu parameter is deprecated, use neutron::newtork_device_mtu instead.[0m [mNotice: Compiled catalog for turing.berg.ol in environment production in 1.72 seconds[0m [1;31mWarning: The package type's allow_virtual parameter will be changing its default value from false to true in a future release. If you do not want to allow virtual packages, please explicitly set allow_virtual to false. (at /usr/share/ruby/vendor_ruby/puppet/type.rb:816:in `set_default')[0m [mNotice: /Stage[main]/Packstack::Neutron::Bridge/Exec[sysctl_refresh]/returns: executed successfully[0m [mNotice: /Stage[main]/Neutron::Server::Notifications/Nova_admin_tenant_id_setter[nova_admin_tenant_id]/ensure: created[0m [mNotice: /Stage[main]/Neutron::Agents::Ml2::Ovs/Service[neutron-ovs-agent-service]/ensure: ensure changed 'stopped' to 'running'[0m [mNotice: /Stage[main]/Main/Vs_bridge[br-ex]/ensure: created[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: INFO [alembic.migration] Context impl MySQLImpl.[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: INFO [alembic.migration] Will assume non-transactional DDL.[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: INFO [alembic.migration] Running upgrade 37f322991f59 -> 31d7f831a591, add constraint for routerid[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: Traceback (most recent call last):[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/bin/neutron-db-manage", line 10, in <module>[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: sys.exit(main())[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py", line 173, in main[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: CONF.command.func(config, CONF.command.name)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py", line 83, in do_upgrade_downgrade[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: do_alembic_command(config, cmd, revision, sql=CONF.command.sql)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/cli.py", line 61, in do_alembic_command[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: getattr(alembic_command, cmd)(config, *args, **kwargs)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/command.py", line 125, in upgrade[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: script.run_env()[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/script.py", line 203, in run_env[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: util.load_python_file(self.dir, 'env.py')[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/util.py", line 215, in load_python_file[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: module = load_module_py(module_id, path)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/compat.py", line 58, in load_module_py[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: mod = imp.load_source(module_id, path, fp)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/alembic_migrations/env.py", line 108, in <module>[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: run_migrations_online()[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/alembic_migrations/env.py", line 100, in run_migrations_online[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: context.run_migrations()[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "<string>", line 7, in run_migrations[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/environment.py", line 689, in run_migrations[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: self.get_context().run_migrations(**kw)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/migration.py", line 263, in run_migrations[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: change(**kw)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/neutron/db/migration/alembic_migrations/versions/31d7f831a591_add_constraint_for_routerid.py", line 65, in upgrade[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: % {'table': TABLE_NAME})[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "<string>", line 7, in execute[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/operations.py", line 1024, in execute[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: execution_options=execution_options)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/ddl/impl.py", line 79, in execute[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: self._exec(sql, execution_options)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib/python2.7/site-packages/alembic/ddl/impl.py", line 76, in _exec[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: conn.execute(construct, *multiparams, **params)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 729, in execute[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: return meth(self, multiparams, params)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: return connection._execute_clauseelement(self, multiparams, params)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: compiled_sql, distilled_params[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 958, in _execute_context[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: context)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1155, in _handle_dbapi_exception[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: util.raise_from_cause(newraise, exc_info)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: reraise(type(exception), exception, tb=exc_tb)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 951, in _execute_context[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: context)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/default.py", line 436, in do_execute[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: cursor.execute(statement, parameters)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 174, in execute[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: self.errorhandler(self, exc, value)[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: raise errorclass, errorvalue[0m [mNotice: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: sqlalchemy.exc.OperationalError: (OperationalError) (1054, "Unknown column 'id' in 'field list'") 'DELETE routerl3agentbindings FROM routerl3agentbindings LEFT OUTER JOIN (SELECT MIN(id) as id, router_id, l3_agent_id FROM routerl3agentbindings GROUP BY router_id, l3_agent_id) AS temp ON routerl3agentbindings.id = temp.id WHERE temp.id is NULL;' ()[0m [1;31mError: neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini upgrade head returned 1 instead of one of [0][0m [1;31mError: /Stage[main]/Main/Exec[neutron-db-manage upgrade]/returns: change from notrun to 0 failed: neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini upgrade head returned 1 instead of one of [0][0m [mNotice: /Stage[main]/Neutron::Agents::L3/Service[neutron-l3]/ensure: ensure changed 'stopped' to 'running'[0m [mNotice: /Stage[main]/Neutron::Server/Service[neutron-server]: Dependency Exec[neutron-db-manage upgrade] has failures: true[0m [1;31mWarning: /Stage[main]/Neutron::Server/Service[neutron-server]: Skipping because of failed dependencies[0m [mNotice: /Stage[main]/Neutron::Server/Service[neutron-server]: Triggered 'refresh' from 1 events[0m [mNotice: Finished catalog run in 5.11 seconds[0m Version-Release number of selected component (if applicable): openstack-packstack-2014.2-0.4.dev1266.g63d9c50.fc22.noarch openstack-puppet-modules-2014.2.1-0.5.fc22.noarch openstack-neutron-2014.2-1.fc22.noarch
This was tested on f21 while RDO CI runs currently on f20 since f21 image was not stable yet NB database on f21 is mariadb-galera-server-10.0.13-6.fc21.x86_64 while f20 has 5.5
I can not reproduce this on f20.
it seems to be a change in database table routerl3agentbindings. The following query fails: DELETE routerl3agentbindings FROM routerl3agentbindings LEFT OUTER JOIN (SELECT MIN(id) as id, router_id, l3_agent_id FROM routerl3agentbindings GROUP BY router_id, l3_agent_id) AS temp ON routerl3agentbindings.id = temp.id WHERE temp.id is NULL; on f20, f21, juno: MariaDB [neutron]> describe routerl3agentbindings; +-------------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------------+-------------+------+-----+---------+-------+ | router_id | varchar(36) | YES | MUL | NULL | | | l3_agent_id | varchar(36) | YES | MUL | NULL | | +-------------+-------------+------+-----+---------+-------+ where Icehouse has an 'id' field: MariaDB [neutron]> describe routerl3agentbindings; +-------------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------------+-------------+------+-----+---------+-------+ | id | varchar(36) | NO | PRI | NULL | | | router_id | varchar(36) | YES | MUL | NULL | | | l3_agent_id | varchar(36) | YES | MUL | NULL | | +-------------+-------------+------+-----+---------+-------+ 3 rows in set (0.00 sec) The question would be more: why f20 does not fail?
on f21: python-sqlalchemy-0.9.8-1.fc21.x86_64 python-alembic-0.6.6-1.fc21.noarch
neutron-db-manage --config-file /etc/neutron/neutron.conf upgrade head reproduced the problem for me
The problem since MySQL 5.6 forbids to alter column with FK constraint. I tested solution we drop both FKs (router_id, l3_agent_id), set new primary key for l3_agent_id and then re-create FKs and it works. I'll send patch soon.
*** Bug 1160009 has been marked as a duplicate of this bug. ***
*** Bug 1154702 has been marked as a duplicate of this bug. ***
https://review.openstack.org/#/c/132273/ has been merged, please build new RDO Juno openstack-neutron which includes this fix.
http://pkgs.fedoraproject.org/cgit/openstack-neutron.git/commit/?id=e498dc2bca59c50333ea40318d31b6c0a3ba0cd7