Bug 1104239
| Summary: | nova-conductor Deprecation Warning | |||
|---|---|---|---|---|
| Product: | Red Hat OpenStack | Reporter: | Stephen Gordon <sgordon> | |
| Component: | python-migrate | Assignee: | Brent Eagles <beagles> | |
| Status: | CLOSED ERRATA | QA Contact: | Toure Dunnon <tdunnon> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 5.0 (RHEL 7) | CC: | beagles, jtaleric, ndipanov, oblaut, sclewis, sgordon, tdunnon, yeylon | |
| Target Milestone: | rc | |||
| Target Release: | 5.0 (RHEL 7) | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | python-migrate-0.8.2-2.el7ost | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | 1093305 | |||
| : | 1109402 (view as bug list) | Environment: | ||
| Last Closed: | 2014-07-08 15:14:02 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1093305 | |||
| Bug Blocks: | 1109402 | |||
|
Description
Stephen Gordon
2014-06-03 15:16:17 UTC
Apparently these are frequent enough to be flooding the logs: https://gist.github.com/jtaleric/015c8842a9e11355849a To be clear, the gist above is from a RHEL OSP5 install on RHEL7. Nova packages: openstack-nova-conductor-2014.1-2.el7ost.noarch openstack-nova-common-2014.1-2.el7ost.noarch openstack-nova-novncproxy-2014.1-2.el7ost.noarch openstack-nova-api-2014.1-2.el7ost.noarch openstack-nova-scheduler-2014.1-2.el7ost.noarch openstack-nova-console-2014.1-2.el7ost.noarch openstack-nova-cert-2014.1-2.el7ost.noar Apparently the increase in the visibility of the deprecation message is deliberate. See: https://github.com/celery/kombu/issues/296 and https://pypi.python.org/pypi/amqp/1.4.5 A discussion of auto_delete in rabbitmq and the way we use it: http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2014-May/036031.html A little contradictory in that it claims the feature isn't going away. The amqp lib in pypi seems to be the source of this warning. What is interesting is that it shouldn't be showing up if you are using python 2.7 (which seems to be the case above) as the DeprecationWarning has been defaulted to "ignore" in the warnings module in 2.7. Maybe a bogus py[co] file kicking around? If not then there is probably another issue to be reported against the python packages. (setting NEEDINFO to get it checked out) It *is* a nuisance for folks stuck on python 2.6 though since it kicks up an exception. The workaround would be to use the "warnings" module's filtering mechanism to suppress the exception. As a heads up, we don't get this with 6.5 because it uses amqplib, not amqp.py. As it so often turns out, the amqp.py thing is not the entire story. On a RHEL 6.5 system with python 2.6.6, I hacked up my amqplib.py to see if it would display the deprecation warning and it did. I tried disabling it with a call to warnings.simplefilter() and it still appeared. A simple check reveals that the python environment on the host exhibiting the issue has the warning disabled by default, so that was not the source of the issue. It remained then to hunt down what actually was enabling it. Hacking warnings.py to print a stack trace when the filter manipulators were called revealed that
/usr/lib/python2.6/site-packages/migrate/changeset/__init__.py
contains:
warnings.simplefilter('always', DeprecationWarning)
This is arguably a bug in that migrate module. Instead of restrict the filter to its own modules, it applies it globally.
Full stack trace causing import follows:
always <type 'exceptions.DeprecationWarning'>
File "/usr/lib/python2.6/site-packages/eventlet/greenthread.py", line 194, in main
result = function(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/nova/openstack/common/service.py", line 483, in run_service
service.start()
File "/usr/lib/python2.6/site-packages/nova/service.py", line 168, in start
self.host, self.binary)
File "/usr/lib/python2.6/site-packages/nova/conductor/api.py", line 193, in service_get_by_args
binary=binary)
File "/usr/lib/python2.6/site-packages/nova/utils.py", line 966, in wrapper
return func(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/oslo/messaging/rpc/server.py", line 139, in inner
return func(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/nova/conductor/manager.py", line 441, in service_get_all_by
result = self.db.service_get_by_args(context, host, binary)
File "/usr/lib/python2.6/site-packages/nova/db/api.py", line 134, in service_get_by_args
return IMPL.service_get_by_args(context, host, binary)
File "/usr/lib/python2.6/site-packages/nova/openstack/common/db/api.py", line 147, in __getattr__
self._load_backend()
File "/usr/lib/python2.6/site-packages/nova/openstack/common/db/api.py", line 142, in _load_backend
backend_mod = importutils.import_module(backend_path)
File "/usr/lib/python2.6/site-packages/nova/openstack/common/importutils.py", line 57, in import_module
__import__(import_str)
File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/api.py", line 57, in <module>
from nova.openstack.common.db.sqlalchemy import utils as sqlalchemyutils
File "/usr/lib/python2.6/site-packages/nova/openstack/common/db/sqlalchemy/utils.py", line 22, in <module>
from migrate.changeset import UniqueConstraint
File "/usr/lib/python2.6/site-packages/migrate/__init__.py", line 31, in <module>
from migrate.changeset import *
File "/usr/lib/python2.6/site-packages/migrate/changeset/__init__.py", line 13, in <module>
warnings.simplefilter('always', DeprecationWarning)
File "/usr/lib64/python2.6/warnings.py", line 76, in simplefilter
Seeing that there is a root cause identified - I assume we can patch this in RHOS for the 5.0 release and do the proper fix upstream and drop the patch once that's in? Nikola - I agree. We are moving to fix this at the "source" (i.e. sqlalchemy-migrate) instead of the previously posted workarounds. This should satisfy the concerns posted against the original approach. So to be clear, we'd be patching python-sqlalchemy and not nova or oslo, etc. Indicated wrong package in Comment 7 - should have been python-migrate. To verify: Scenario A: existing openstack deployment - Before updating packages, inspect /var/log/messages and check for reports similar to the ones indicated in this bug report (e.g. auto_delete exchanges are deprecated, etc). - Shutdown openstack services (openstack-service stop) - Verify there are no openstack services running (ps ax | grep python, etc) - Install updated python-migrate package. You can update all RPMs, but for the sake of direct verification it would be better to update only this package. - Start openstack services (openstack-service start) - Inspect or tail /var/log/messages. You should not see any new warnings about auto_delete exchanges. Scenario B: new openstack deployment on fresh system - Verify openstack services are running - Inspect and/or tail /var/log/messages. You should not see deprecation warnings about auto_delete exchanges. It is not clear that additional errata information is required as this was an internally reported issue and would not otherwise impact users unless they were dependent on DeprecationWarnings being "enabled-by-side-effect" in their apps. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-0845.html |