Description of problem: glance-manage db_sync silently fails to prepare the database: # glance-manage db_sync && echo ok ok # mysql glance -e "show tables" +------------------+ | Tables_in_glance | +------------------+ | migrate_version | +------------------+ Version-Release number of selected component (if applicable): openstack-glance-2014.1-1.el6.noarch How reproducible: 100% Steps to Reproduce: 1. connection = mysql://glance:glance@localhost/glance 2. openstack-db --init --service glance Actual results: # openstack-db --init --service glance && echo ok Verified connectivity to MySQL. Creating 'glance' database. Initializing the glance database, please wait... Complete! ok # mysql glance -e "show tables" +------------------+ | Tables_in_glance | +------------------+ | migrate_version | +------------------+ Expected results: The glance database should be ready to be used or an error should be clearly reported. Additional info: Adding some debugging it's possible to see that the actual error is: # glance-manage db_sync && echo ok Tables "migrate_version" have non utf8 collation, please make sure all tables are CHARSET=utf8 ok If CHARSET=utf8 is another issue that should be fixed separately (and not just something related to my env) feel free to split the bug.
As a workaround, one can set: db_enforce_mysql_charset=False in /etc/glance/glance.conf This will get you past this error.
Sorry... /etc/glance/glance-api.conf
(In reply to Brad P. Crochet from comment #1) > As a workaround, one can set: > > db_enforce_mysql_charset=False > > in /etc/glance/glance.conf > > This will get you past this error. The biggest problem is that the error is not printed anywhere (I had to manually add debugging to the code) so no one can tell if this is the issue they're having.
I agree a proper error is necessary.
I can't replicate this issue :/ Running `glance-manage` without setting db_enforce_mysql_charset=False` runs and fails with an error code when `ValueError` is raised. Is there a way I can test this issue in the same environment where you hit it?
To find the issue I had to modify DbCommands.sync() in glance/cmd/manage.py adding a try/except and a print: try: migration.db_sync(db_api.get_engine(), db_migration.MIGRATE_REPO_PATH, version, sanity_check=self._need_sanity_check()) except Exception as e: print(e) These are some of the packages I am using: python-sqlalchemy-0.5.5-3.el6_2.noarch openstack-swift-1.13.1-1.el6.noarch openstack-swift-plugin-swift3-1.7-1.el6.noarch python-sqlalchemy0.7-0.7.8-1.el6ost.x86_64 openstack-keystone-2014.1-2.el6.noarch openstack-glance-2014.1-1.el6.noarch openstack-utils-2013.2-3.el6.noarch
Ahhh, that clear things up. Do you have the glance-api logs? I *think* the whole exception (pre- try/except) may have been logged to the glance log file. `glance-manage` reads /etc/glance/glance-api.conf and it will use the logging configurations found there. If you have the log files, I'll check everything. Otherwise, I'll try to replicate it.
/var/log/glance/api.log: 2014-05-07 17:18:57.679 16348 CRITICAL glance [-] ValueError: Tables "migrate_version" have non utf8 collation, please make sure all tables are CHARSET=utf8 2014-05-07 17:18:57.679 16348 TRACE glance Traceback (most recent call last): 2014-05-07 17:18:57.679 16348 TRACE glance File "/usr/bin/glance-manage", line 10, in <module> 2014-05-07 17:18:57.679 16348 TRACE glance sys.exit(main()) 2014-05-07 17:18:57.679 16348 TRACE glance File "/usr/lib/python2.6/site-packages/glance/cmd/manage.py", line 259, in main 2014-05-07 17:18:57.679 16348 TRACE glance return CONF.command.action_fn() 2014-05-07 17:18:57.679 16348 TRACE glance File "/usr/lib/python2.6/site-packages/glance/cmd/manage.py", line 160, in sync 2014-05-07 17:18:57.679 16348 TRACE glance CONF.command.current_version) 2014-05-07 17:18:57.679 16348 TRACE glance File "/usr/lib/python2.6/site-packages/glance/cmd/manage.py", line 137, in sync 2014-05-07 17:18:57.679 16348 TRACE glance sanity_check=self._need_sanity_check()) 2014-05-07 17:18:57.679 16348 TRACE glance File "/usr/lib/python2.6/site-packages/glance/openstack/common/db/sqlalchemy/migration.py", line 195, in db_sync 2014-05-07 17:18:57.679 16348 TRACE glance _db_schema_sanity_check(engine) 2014-05-07 17:18:57.679 16348 TRACE glance File "/usr/lib/python2.6/site-packages/glance/openstack/common/db/sqlalchemy/migration.py", line 221, in _db_schema_sanity_check 2014-05-07 17:18:57.679 16348 TRACE glance ) % ','.join(table_names)) 2014-05-07 17:18:57.679 16348 TRACE glance ValueError: Tables "migrate_version" have non utf8 collation, please make sure all tables are CHARSET=utf8 As a side note: installing keystone on the same mysql server wasn't a problem.
The UTF8 issue is known. That is, Glance was the only service still using non-UTF8 tables and those need to be migrated. With that said, I think this bug could be fixed by making `openstack-db` use other logging values on the CLI. I'll move this bug under `openstack-utils` and close Glance's upstream bug since there's no bug here from a Glance stand point. Thanks Fede.
As per comment #5, glance-manage prints out the ValueError? If not it should. There have been long term issues with the early startup issues in glance not being logged. openstack-db doesn't suppress any error messages from glance-manage, and glance-manage should log appropriately given the current logging setting of the system given not only openstack-db may be calling it.
It prints the ValueError but it logs it in the log file. This happens because glance-manage is reading the global config which defaults to logging to the log file. There are 2 ways to fix this that I can think OTOH: 1. Changing Glance's default (use_stderr = True) 2. Make openstack-db explicitly specify std(out|err) as the preferred logging output, hence the move to openstack-utils. I'd prefer changing this in openstack-utils rather than making `use_stderr=True` the default value. This also applies to other projects. openstack-db could do something like: $ glance-manage -d -v --log-file= db_sync The above unsets the log file and forces glance-manage to use stdout.
Ah sorry I thought Federico needed to patch glance to get the log, when in fact it was logged to the standard location. So as for changing glance-manage to log to stderr rather than to logs, I'm a bit reluctant, as there are often superfluous things logged to openstack logs, and having everything logged to a single location can have its advantages too. I suppose the best thing would be to bolster the sanity check done in openstack-db, or otherwise honor/diagnose the exit status of the glance-manage command. I presume it does exit with failure code as you said in comment 5. Hopefully this patch addresses things appropriately: https://github.com/redhat-openstack/openstack-utils/commit/db3ca160