Bug 890180

Summary: postgres support for database - glance
Product: Red Hat OpenStack Reporter: Perry Myers <pmyers>
Component: openstack-glanceAssignee: RHOS Maint <rhos-maint>
Status: CLOSED CURRENTRELEASE QA Contact: Attila Fazekas <afazekas>
Severity: medium Docs Contact:
Priority: medium    
Version: 1.0 (Essex)CC: apevec, fpercoco, markmc, pbrady
Target Milestone: gaKeywords: FutureFeature, TestOnly
Target Release: 2.1   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Enhancement
Doc Text:
Story Points: ---
Clone Of: 816316 Environment:
Last Closed: 2013-06-28 14:41:15 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 892384, 905082    
Bug Blocks: 816316    

Description Perry Myers 2012-12-25 15:03:56 UTC
+++ This bug was initially created as a clone of Bug #816316 +++

Description of problem:
Presently upstream OpenStack primarily utilizes mysql for its database.  We'd like to expand this to include postgres in addition.  There are probably some integration issues/small problems with running Postgres w/ Essex today, and these bugs will need to be tracked and individually resolved separately.

This bug is specifically for tracking the overall support of Postgres, and as such is a TestOnly bug.

--- Additional comment from Perry Myers on 2012-04-25 15:14:39 EDT ---

Related bugs:
https://bugs.launchpad.net/glance/+bug/953159
https://bugs.launchpad.net/glance/+bugs?field.tag=postgresql

Comment 1 Attila Fazekas 2013-01-25 14:43:52 UTC
https://bugzilla.redhat.com/show_bug.cgi?id=879387 describes the incompatibility between sqlalchemy and sqlalchemy-migrate.
When I installed it from pip it passed:

[root@node-01 cinder(keystone_admin)]$ pip-python install sqlalchemy-migrate==0.7.1
Downloading/unpacking sqlalchemy-migrate==0.7.1
  Downloading sqlalchemy-migrate-0.7.1.tar.gz (101Kb): 101Kb downloaded
  Running setup.py egg_info for package sqlalchemy-migrate
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*pyc' found anywhere in distribution
    warning: no previously-included files found matching '.hgtags'
    warning: no previously-included files matching '*' found under directory 'docs/_build'
Requirement already satisfied (use --upgrade to upgrade): SQLAlchemy>=0.5 in /usr/lib/python2.6/site-packages (from sqlalchemy-migrate==0.7.1)
Requirement already satisfied (use --upgrade to upgrade): decorator in /usr/lib/python2.6/site-packages (from sqlalchemy-migrate==0.7.1)
Requirement already satisfied (use --upgrade to upgrade): Tempita>=0.4 in /usr/lib/python2.6/site-packages (from sqlalchemy-migrate==0.7.1)
Requirement already satisfied (use --upgrade to upgrade): distribute in /usr/lib/python2.6/site-packages (from sqlalchemy-migrate==0.7.1)
Installing collected packages: sqlalchemy-migrate
  Found existing installation: sqlalchemy-migrate 0.6
    Uninstalling sqlalchemy-migrate:
      Successfully uninstalled sqlalchemy-migrate
  Running setup.py install for sqlalchemy-migrate
    warning: no files found matching '*' under directory 'tests'
    warning: no previously-included files matching '*pyc' found anywhere in distribution
    warning: no previously-included files found matching '.hgtags'
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    Installing migrate script to /usr/bin
    Installing migrate-repository script to /usr/bin
Successfully installed sqlalchemy-migrate
Cleaning up...
[root@node-01 cinder(keystone_admin)]$ su -s /bin/sh glance -c "glance-manage db_sync"[root@node-01 cinder(keystone_admin)]$ su - postgres -c "psql"
psql (8.4.13)
Type "help" for help.

postgres=# DROP DATABASE glance;
DROP DATABASE
postgres=#      CREATE DATABASE glance OWNER glance;
CREATE DATABASE
postgres=# \q
[root@node-01 cinder(keystone_admin)]$ su -s /bin/sh glance -c "glance-manage db_sync"
/usr/lib/python2.6/site-packages/migrate/changeset/schema.py:553: SAWarning: <type 'NoneType'> being associated with <class 'sqlalchemy.schema.Column'> object after the <class 'sqlalchemy.schema.Column'> has already been used in a SQL generation; previously generated constructs may contain stale state.
  self.table = None
/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migrate_repo/versions/003_add_disk_format.py:46: SADeprecationWarning: useexisting is deprecated.  Use extend_existing.
  useexisting=True)
[root@node-01 cinder(keystone_admin)]$ 
#(Just warning)

According to #879387 the python-migrate-0.7.1 has build time test issues with the old alchemy.

I assume the #892384 is related to the version issues.

Comment 2 Attila Fazekas 2013-02-06 14:58:26 UTC
The current status:

[root@node-01 ~]# su -s /bin/sh glance -c "glance-manage db_sync"
Traceback (most recent call last):
  File "/usr/bin/glance-manage", line 132, in <module>
    main()
  File "/usr/bin/glance-manage", line 128, in main
    dispatch_cmd(args)
  File "/usr/bin/glance-manage", line 98, in dispatch_cmd
    cmd_func(args)
  File "/usr/bin/glance-manage", line 86, in do_db_sync
    glance.db.sqlalchemy.migration.db_sync(version, current_version)
  File "/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migration.py", line 127, in db_sync
    upgrade(version=version)
  File "/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migration.py", line 67, in upgrade
    return versioning_api.upgrade(sql_connection, repo_path, version)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 185, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File "/usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
    return f(*a, **kw)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 364, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 90, in runchange
    change.run(self.engine, step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/script/py.py", line 145, in run
    script_func(engine)
  File "/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migrate_repo/versions/006_key_to_name.py", line 75, in upgrade
    index.rename('ix_image_properties_image_id_name')
  File "/usr/lib/python2.6/site-packages/migrate/changeset/schema.py", line 620, in rename
    engine._run_visitor(visitorcallable, self, connection, **kwargs)
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 2303, in _run_visitor
    conn._run_visitor(visitorcallable, element, **kwargs)
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1973, in _run_visitor
    **kwargs).traverse_single(element)
  File "/usr/lib/python2.6/site-packages/migrate/changeset/ansisql.py", line 55, in traverse_single
    ret = super(AlterTableVisitor, self).traverse_single(elem)
  File "/usr/lib64/python2.6/site-packages/SQLAlchemy-0.7.8-py2.6-linux-x86_64.egg/sqlalchemy/sql/visitors.py", line 106, in traverse_single
    return meth(obj, **kw)
  File "/usr/lib/python2.6/site-packages/migrate/changeset/ansisql.py", line 172, in visit_index
    (self.preparer.quote(self._validate_identifier(index.name,
AttributeError: 'PGSchemaChanger' object has no attribute '_validate_identifier'

The sqlalchemy-migrate package needs to be updated as well.