Bug 879387

Thanks Attila for noticing the upstream bug,
and trying out the new version, with PostgreSQL.

While there is no mention of PostgreSQL in the 0.6.1 release notes,
I'll take your word that it fixes the bug :)

I'll prepare an update ASAP unless I hear otherwise.

BTW I'm a bit worried by the release notes mention of
"migrate.versioning.exceptions to migrate.exceptions"
as I needed to patch nova to handle that explicitly like:

  try:
    # Try the more specific path first (migrate <= 0.6)
    from migrate.versioning import exceptions as versioning_exceptions
  except ImportError:
    # Use the newer path (migrate >= 0.7)
    from migrate import exceptions as versioning_exceptions

So nova will be fine with either, but existing packages
expecting migrate.versioning.exceptions may fail?

Attila could you try this on your pip installed version:

python -c "from migrate.versioning import exceptions; print exceptions.__file__"

I expect it might error.

Looks like it just jumps that particular issue when I tried with 0.6.1 :(. It is just working (return 0) after you tried with the older version first.
I was also able to use it.

Just 0.6.1 wont be enough.

With 0.6.1 I got this:
python -c "from migrate.versioning import exceptions; print exceptions.__file__"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: cannot import name exceptions


After drop and create database again round I got a new error message:
Traceback (most recent call last):
  File "/usr/bin/glance-manage", line 126, in <module>
    main()
  File "/usr/bin/glance-manage", line 122, in main
    dispatch_cmd(args)
  File "/usr/bin/glance-manage", line 98, in dispatch_cmd
    cmd_func(args)
  File "/usr/bin/glance-manage", line 86, in do_db_sync
    glance.db.sqlalchemy.migration.db_sync(version, current_version)
  File "/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migration.py", line 127, in db_sync
    upgrade(version=version)
  File "/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migration.py", line 67, in upgrade
    return versioning_api.upgrade(sql_connection, repo_path, version)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 186, in upgrade
    return _migrate(url, repository, version, upgrade=True, err=err, **opts)
  File "<string>", line 2, in _migrate
  File "/usr/lib/python2.6/site-packages/migrate/versioning/util/__init__.py", line 159, in with_engine
    return f(*a, **kw)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 365, in _migrate
    schema.runchange(ver, change, changeset.step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 84, in runchange
    change.run(self.engine, step)
  File "/usr/lib/python2.6/site-packages/migrate/versioning/script/py.py", line 145, in run
    script_func(engine)
  File "/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migrate_repo/versions/003_add_disk_format.py", line 114, in upgrade
    images.columns['type'].drop()
  File "/usr/lib/python2.6/site-packages/migrate/changeset/schema.py", line 552, in drop
    self.remove_from_table(self.table, unset_table=False)
  File "/usr/lib/python2.6/site-packages/migrate/changeset/schema.py", line 593, in remove_from_table
    table.c.remove(self)
  File "/usr/lib/python2.6/site-packages/sqlalchemy/util/_collections.py", line 38, in _immutable
    raise TypeError("%s object is immutable" % self.__class__.__name__)
TypeError: ImmutableColumnCollection object is immutable

I had to upgrade it a much newer version (tried 0.7.2) in order to work from the beginning.

The http://code.google.com/p/sqlalchemy-migrate/ says the "SQLAlchemy-migrate 0.7.1 provides support for SQLAlchemy versions from 0.5.x to 0.7.x."

With 0.7.1 I was able to sync keystone, nova, glance and cinder.


Note:
I got the usual warning message too.
/usr/lib/python2.6/site-packages/glance/db/sqlalchemy/migrate_repo/versions/003_add_disk_format.py:46: SADeprecationWarning: useexisting is deprecated.  Use extend_existing.
  useexisting=True)

Pádraig, is insta||able python-migrate0.7-0.7.2 doable?
PostgreSQL support seems important enough to justify it and it would avoid sqlalchemy/migrate combination nobody else uses.

Parallel installable is doable but parallel installable packages are best avoided if possible. I'd be inclined to update to 0.7.1 in this case.

So python-migrate=0.7.2 is what's generally in use,
and required by nova, etc. That dependency was done upstream
as there were issues seen with v 0.7.1 in
https://bugs.launchpad.net/nova/+bug/928637

Though Attila in comment #3 said all projects sync fine with 0.7.1
so maybe the code at issue with 0.7.1 was changed upstream in the meantime?

Given 0.7.2 is documented to not be compatible with python-sqlalchemy-0.5
I'll avoid that for now.

Though Attila why is this the only report of this?
Is there something specific about your environment?

Anyway I'll prepare an update for testing, which I won't
push to stable until it gets appropriate karma.

So that was not successful. Seems like contrary to the docs,
0.7.1 is not in fact compat with sqlalchemy 0.5 as the build
% check fails with lots of instances of:

File "/builddir/build/BUILD/sqlalchemy-migrate-0.7.1/migrate/versioning/schemadiff.py", line 72, in __init__
    self.affinity_A = self.type_A._type_affinity
AttributeError: 'Text' object has no attribute '_type_affinity'

http://kojipkgs.fedoraproject.org//work/tasks/4523/4894523/build.log

I presume _type_affinity was introduced to sqlalchemy sometime
after the version currently in RHEL.

So the _type_affinity code was added since sqlalchemy 0.5.7.
So the version 0.5.5 in RHEL 6.4 (released Jul 2009)
is not supported by migrate 0.7.1.

To have a non parallel installable python-migrate-0.7.1 would therefore
require rebasing python-sqlalchemy in RHEL 6 to 0.5.8 (released Jul 2010)

Another report from Havana 3 test that with Postgres and migrate 0.6 you get:

glance AttributeError: 'PGSchemaChanger' object has no attribute '_validate_identifier'

The other option now available is to make a non parallel installable python-migrate-0.7.2 available to RDO and if postgres support required RHOS (LPC).
So given the risk/delay of updating the RHEL/EPEL components as described in comment 10, I'll take this non parallel approach.

I've built an updated python-migrate-0.7.2 that depends on sqlalchemy0.7 at:
http://kojipkgs.fedoraproject.org/packages/python-migrate/0.7.2/8.el6/

It was tricky to avoid an (already) parallel installed sqlalchemy-0.5 in all cases, but I think it should work.
Details of the changes specific to the parallel sqlalchemy0.7 are:
http://pkgs.fedoraproject.org/cgit/python-migrate.git/log/?h=el6-havana&qt=range&q=8c077ebb...bcad4f5d

Pushed to RDO Havana EL6 repo