Bug 1349350 - Unknown column 'severity' in 'field list' on undercloud
Summary: Unknown column 'severity' in 'field list' on undercloud
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat OpenStack
Classification: Red Hat
Component: openstack-ceilometer
Version: 9.0 (Mitaka)
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ga
: 9.0 (Mitaka)
Assignee: Pradeep Kilambi
QA Contact: Yurii Prokulevych
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-06-23 09:56 UTC by Yurii Prokulevych
Modified: 2016-08-11 12:27 UTC (History)
12 users (show)

Fixed In Version: openstack-ceilometer-6.1.3-2.el7ost
Doc Type: Bug Fix
Doc Text:
Telemetry (ceilometer) dbsync creates an old alarming table, as a result of still running the `sqlalchemy-migrate` code. Consequently, Aodh dbsync sees no reference to Alembic, and has SQLAlchemy create the necessary tables. However, the tables are already present, so nothing is done, and the database is stamped to the latest version of Aodh Alembic. You can avoid this issue by not creating alarm tables in ceilometer dbsync.
Clone Of:
Environment:
Last Closed: 2016-08-11 12:27:43 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
OpenStack gerrit 333397 0 None None None 2016-06-23 15:05:03 UTC
Red Hat Product Errata RHEA-2016:1597 0 normal SHIPPED_LIVE Red Hat OpenStack Platform 9 Release Candidate Advisory 2016-08-11 16:06:52 UTC

Description Yurii Prokulevych 2016-06-23 09:56:06 UTC
Description of problem:
aodh alarm create \
 --type threshold \
 --name 'AClient-Threshold-2' \
 --description 'AodhClient-Threshold-Alarm-1' \
 --severity low \
 --enabled True \
 --alarm-action 'log://' \
 --ok-action 'log://' \
 --insufficient-data-action 'log://' \
 --repeat-actions False \
 --comparison-operator ge \
 --evaluation-periods 3 \
 --meter-name CustomMeter-1 \
 --period 60 \
 --statistic avg \
 --threshold 4 \
 --query 'resource_id=CustomMeter-Resource-1'

Failed with next error:
-----------------------
(pymysql.err.InternalError) (1054, u"Unknown column 'severity' in 'field list'") [SQL: u'INSERT INTO alarm_history (event_id, alarm_id, on_behalf_of, project_id, user_id, type, detail, timestamp, severity) VALUE
S (%s, %s, %s, %s, %s, %s, %s, %s, %s)'] [parameters: ('74000a6a-1cfe-461a-aacd-5af30acdb2fc', u'2bbd8cb8-d056-4cef-b255-96d88892b180', u'ac06a9ae73384967a05fbacb7f6a5a5b', u'ac06a9ae73384967a05fbacb7f6a5a5b', u
'20c6b773b8514c1b8b7b68200df1ea20', 'creation', '{"alarm_actions": ["log://"], "user_id": "20c6b773b8514c1b8b7b68200df1ea20", "name": "AClient-Threshold-2", "state": "insufficient data", "timestamp": "2016-06-23
T07:08:15.055917", "description": "AodhClient-Threshold-Alarm-1", "enabled": true, "state_timestamp": "2016-06-23T07:08:15.055917", "rule": {"meter_name": "CustomMeter-1", "evaluation_periods": 3, "period": 60,
"statistic": "avg", "threshold": 4.0, "query": [{"field": "resource_id", "type": "", "value": "CustomMeter-Resource-1", "op": "eq"}], "comparison_operator": "ge", "exclude_outliers": false}, "alarm_id": "2bbd8cb
8-d056-4cef-b255-96d88892b180", "time_constraints": [], "insufficient_data_actions": ["log://"], "repeat_actions": false, "ok_actions": ["log://"], "project_id": "ac06a9ae73384967a05fbacb7f6a5a5b", "type": "thre
shold", "severity": "low"}', Decimal('1466665695.055917'), 'low')] (HTTP 500) (Request-ID: req-13d48644-633b-4473-a0e5-1c857ee8d31c)

Indeed, alarm_history table missing 'severity' field:

> desc alarm_history;
+--------------+---------------+------+-----+---------+-------+
| Field        | Type          | Null | Key | Default | Extra |
+--------------+---------------+------+-----+---------+-------+
| event_id     | varchar(128)  | NO   | PRI | NULL    |       |
| alarm_id     | varchar(128)  | YES  | MUL | NULL    |       |
| on_behalf_of | varchar(255)  | YES  | MUL | NULL    |       |
| project_id   | varchar(255)  | YES  | MUL | NULL    |       |
| user_id      | varchar(255)  | YES  | MUL | NULL    |       |
| type         | varchar(20)   | YES  |     | NULL    |       |
| detail       | text          | YES  |     | NULL    |       |
| timestamp    | decimal(20,6) | YES  |     | NULL    |       |
+--------------+---------------+------+-----+---------+-------+

Though alembig_version shows the expected value:
> select * from alembic_version; 
+-------------+
| version_num |
+-------------+
| bb07adac380 |
+-------------+

After creating a new database and running aodh-dbsync the issue gone.

Version-Release number of selected component (if applicable):
-------------------------------------------------------------
mariadb-5.5.47-1.el7_2.x86_64
mariadb-libs-5.5.47-1.el7_2.x86_64
mariadb-server-5.5.47-1.el7_2.x86_64

python-aodhclient-0.5.0-1.el7ost.noarch
openstack-aodh-common-2.0.1-3.el7ost.noarch
openstack-aodh-listener-2.0.1-3.el7ost.noarch
openstack-aodh-notifier-2.0.1-3.el7ost.noarch
python-aodh-2.0.1-3.el7ost.noarch
openstack-aodh-api-2.0.1-3.el7ost.noarch
openstack-aodh-evaluator-2.0.1-3.el7ost.noarch

openstack-tripleo-heat-templates-2.0.0-11.el7ost.noarch
openstack-tripleo-heat-templates-kilo-2.0.0-11.el7ost.noarch
python-tripleoclient-2.0.0-1.0.4.el7ost.noarch
openstack-tripleo-common-2.0.0-6.el7ost.noarch
openstack-tripleo-0.0.8-0.2.d81bd6dgit.el7ost.noarch
openstack-tripleo-image-elements-0.9.9-5.el7ost.noarch
openstack-tripleo-puppet-elements-2.0.0-2.el7ost.noarch
openstack-tripleo-heat-templates-liberty-2.0.0-11.el7ost.noarch

Additional info:
VirtualSetup: 3Controllers +2Computes +1Ceph

Comment 2 Julien Danjou 2016-06-23 15:06:40 UTC
The problem is that on fresh install:

- Ceilometer dbsync create the old alarming table because it runs the sqlalchemy-migrate code
- Aodh dbsync see no reference to Alembic, so ask SQLAlchemy to create the tables; the tables are there, so it does nothing, and stamp the database to the latest version of Aodh Alembic

https://review.openstack.org/333397 should avoid creating the alarming table in Ceilometer on fresh installs.

Comment 6 Julien Danjou 2016-07-20 14:55:28 UTC
The fix has been backported to Mitaka upstream and should be in Ceilometer 6.1.3 once https://review.openstack.org/#/c/344701/ is approved.

Comment 8 Julien Danjou 2016-07-22 21:14:32 UTC
Package rebase needed to Ceilometer 6.1.3

Comment 10 Julien Danjou 2016-07-26 14:56:16 UTC
Upgrade fails because, based on my original scenario described earlier the following happens:

"- Aodh dbsync see no reference to Alembic, so ask SQLAlchemy to create the tables; the tables are there, so it does nothing, and stamp the database to the latest version of Aodh Alembic"

This has been fixed in master and backported to stable/mitaka in Aodh:
https://git.openstack.org/cgit/openstack/aodh/commit/?h=stable/mitaka&id=4a53f93f60a1a7dc9981801ca2bcee5a168949e1

I've pushed a review at https://review.openstack.org/#/c/347433/ to release Aodh 2.0.3 with this fix.

Comment 12 Yurii Prokulevych 2016-07-29 10:51:41 UTC
Fresh install of RHOS-9.
Alarm is successfully created and transitions between states.

Packages:
---------
instack-0.0.8-3.el7ost.noarch
instack-undercloud-4.0.0-10.el7ost.noarch

openstack-aodh-common-2.0.3-2.el7ost.noarch
openstack-aodh-notifier-2.0.3-2.el7ost.noarch
openstack-aodh-evaluator-2.0.3-2.el7ost.noarch
python-aodh-tests-2.0.3-2.el7ost.noarch
openstack-aodh-listener-2.0.3-2.el7ost.noarch
python-aodh-2.0.3-2.el7ost.noarch
python-aodhclient-0.5.0-1.el7ost.noarch
openstack-aodh-api-2.0.3-2.el7ost.noarch

openstack-ceilometer-collector-6.1.3-2.el7ost.noarch
openstack-ceilometer-central-6.1.3-2.el7ost.noarch
python-ceilometerclient-2.3.0-1.el7ost.noarch
python-ceilometer-tests-6.1.3-2.el7ost.noarch
python-ceilometer-6.1.3-2.el7ost.noarch
openstack-ceilometer-common-6.1.3-2.el7ost.noarch
openstack-ceilometer-notification-6.1.3-2.el7ost.noarch
openstack-ceilometer-polling-6.1.3-2.el7ost.noarch
openstack-ceilometer-api-6.1.3-2.el7ost.noarch

Comment 14 errata-xmlrpc 2016-08-11 12:27:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-1597.html


Note You need to log in before you can comment on or make changes to this bug.